switch HA-proxy tests to h1load client #69

Sashan · 2025-11-27T15:37:15Z

This change simplifies current HA-proxy test set up. Testing no longer requires apache/nginx server as backend. Instead of using siege as a client the test uses
h1load [1].

The pull request also install httpterm [2] http/1.1 server. It's unused currently.

The HA-proxy configuration for testing matches the configuration used in 'State of SSL stacks' write up.

The h1load client currently runs with options as follows :
h1load
-l \ # long results, output expected by h1load shell script
-P \ # report also percentiles for gathared data
-d ${TEST_TIME} \ # test duration, TEST_TIME is 10secs
-c 500 \ # 500 concurrent connections
-t ${THREAD_COUNT} \ # gather data for 1, 2, 4, 8, 16, 32, 64 threads
-u \ # use runtime instead of system time
${BASE_URL}${PORT} # url where to connect to

The options above is just the initial version.

[1] https://github.com/wtarreau/h1load

[2] https://github.com/wtarreau/httpterm

[3] https://www.haproxy.com/blog/state-of-ssl-stacks

nhorman

This looks fine to me, but I'm a bit confused as to how h1load gets setup with this test. Is it meant to be run by hand independently?

Sashan · 2025-12-08T07:49:22Z

This looks fine to me, but I'm a bit confused as to how h1load gets setup with this test. Is it meant to be run by hand independently?

it's run by bench_run_haproxy.sh this comes from run_test() function:

    RESULT=${RESULT_DIR}/h1load-dh-rsa-noreuse-${THREAD_COUNT}-${SSL_LIB}.out
    PORT=$(( ${PORT_RSA} + ${PROXY_CHAIN}  ))
    LD_LIBRARY_PATH=${OPENSSL_DIR}/lib ${H1LOAD} \
        -l \
        -P \
        -d ${TEST_TIME} \
        -c 500 \
        -t ${THREAD_COUNT} \
        -u \
        ${BASE_URL}${PORT} > ${RESULT} || exit 1

it's the H1LOAD variable which holds the path to h1load client. the client is linked with desired SSL library. I'm still verifying set up and figuring out the command line options to use. I've added also ability to use the siege [1] client just to cross check the results with other kind of tests.

the part up to collecting results is mostly done. I'm still working on gnuplot scripts to post-process data. I will include them to separate PR.

wtarreau · 2025-12-11T18:15:47Z

Sasha, before you spend too much time on gnuplot scripts, I'll share you some hints to ease your job (e.g. using -ll instead of -l to have raw numbers instead of human-friendly ones). I'm also finalizing a few small changes that ease selecting relevant lines if you want to compute averages. I'll ping you soon about this.

Sashan · 2025-12-12T10:21:43Z

Walter, thanks a lot. I will be also happy for tips on how to set the h1load arguments. to run benchmark tests.
What I'm looking for is to have a test which will run for certain period of time (let's say 10secs) trying to do as many requests/handshakes as possible using 1, 2, 4, 8, 16, 32, 64 threads. The more requests handled within the period of time the better. This is just very gross benchmark to get some basic metric. Once again thank you.

wtarreau · 2025-12-12T11:03:57Z

Agreed, that's what I want to show you because it's not quite hard. With -d you can set the duration of the test, with -s you can configure a slow ramp-up period (absolutely necessary to avoid measurement errors), and I'll also show you how to pick the relevant values to provide a meaningful measure. On the haproxy side, using taskset is an easy and convenient way to select the number of threads you want. I'll try to dedicate you some time next week to work on this. Today I'm busy chasing a few bugs.

jogme

First iteration - only cosmetics

bench-scripts/bench_config_haproxy.sh

bench-scripts/bench_run_haproxy.sh

bench-scripts/common_util.sh

bench-scripts/bench_config_haproxy.sh

t-j-h · 2026-01-26T10:59:42Z

You comment that you aren't using siege as a client - but I still see a pile of references there - is this expected? Is it still actually used? If so the PR comment should be updated.

Sashan · 2026-01-27T06:44:18Z

You comment that you aren't using siege as a client - but I still see a pile of references there - is this expected? Is it still actually used? If so the PR comment should be updated.

I will fix the comment. I think it is useful to keep both tools working just for a cross check.

This change also simplifies current HA-proxy test set up. Testing no longer requires apache/nginx server as backend. Test also uses h1load [1] client now. The pull request also installs httpterm [2] http/1.1 server. It's used as a backend for tests perfromed by siege [3]. The HA-proxy configuration for testing matches the configuration used in 'State of SSL stacks' [4] write up. The h1load client currently runs with options as follows : h1load -l \ # long results, output expected by h1load shell script -P \ # report also percentiles for gathared data -d ${TEST_TIME} \ # test duration, TEST_TIME is 10secs -c 500 \ # 500 concurrent connections -t ${THREAD_COUNT} \ # gather data for 1, 2, 4, 8, 16, 32, 64 threads -u \ # use runtime instead of system time ${BASE_URL}${PORT} # url where to connect to [1] https://github.com/wtarreau/h1load [2] https://github.com/wtarreau/httpterm [3] https://github.com/JoeDog/siege [4] https://www.haproxy.com/blog/state-of-ssl-stacks

mbroz · 2026-02-05T13:29:03Z

bench-scripts/bench_config_haproxy.sh

+
+function install_h1load {
+    typeset SSL_LIB=$1
+    typeset H1LOAD_REPO="https://github.com/sashan/h1load"


Why are you referencing own repo? Is there h1load upstream PR for changes you have?

I was using the fork because I had to modify the gnuplot scripts. but it looks like there is no reason to keep those modifications around.

BTW @Sashan, when you're forced to change stuff in h1load, do not hesitate to ping me for merging it. The program receives few modifications and most of them are totally relevant and more than welcome, because generally we all face similar limitations/difficulties that we all painfully work around using scripts. So don't be shy!

I think those ended up in the update to README. I'm going to give try the new h1load. And see how it will go. There will be follow up PR which will let gnuplot to generate charts. It will be gathering date from h1load output and also from siege. The h1load data are more detailed the siege is just for cross-check.

npajkovsky · 2026-02-09T08:17:04Z

bench-scripts/bench_config_haproxy.sh

+    # this fork adds -u option to keep time as uptime
+    #
+    cd "${WORKSPACE_ROOT}" || exit 1
+    git clone -b float "${H1LOAD_REPO}" "${DIRNAME}" || exit 1


fatal: Remote branch float not found in upstream origin

this looks like a leftover after sashan changed the repo from his fork to upstream

npajkovsky · 2026-02-09T10:38:46Z

bench-scripts/bench_run_haproxy.sh

+            -d ${TEST_TIME} \
+            -c 500 \
+            -t ${THREAD_COUNT} \
+            -u \


-u seems to be gone.

npajkovsky · 2026-02-09T11:25:41Z

bench-scripts/bench_config_haproxy.sh

+}
+
+check_env
+setup_tests


Should we do plot_result here?

this is a follow up PR

… tests

…A-proxy tests

…nt to HA-proxy tests

jogme

LGTM, thanks!

jogme · 2026-02-09T14:56:03Z

bench-scripts/bench_config_haproxy.sh

+    install_haproxy openssl-master
+    install_httpterm openssl-master
+    install_h1load openssl-master
+    install_siege openssl-master


note: I can't install siege on manjaro, but that's not an issue of this script.

It can't properly detect string.h - even the config says it founds it (fails on HAVE_STRCMP)

thanks, for review. I think ?Nikola? reported the siege does not build on ?suse? if I remember correct. Let's get it working on Ubuntu first.

It cannot be built anywhere. The last version is pretty broken.

ubuntu works fine

bench-scripts/bench_run_haproxy.sh

Sashan added this to Development Board Nov 27, 2025

Sashan moved this to Waiting Review in Development Board Nov 27, 2025

Sashan force-pushed the ha-proxy.h1load branch from e118d97 to 67ed709 Compare November 27, 2025 16:01

vavroch2010 moved this from Waiting Review to In Progress in Development Board Dec 1, 2025

vavroch2010 assigned Sashan Dec 2, 2025

nhorman reviewed Dec 7, 2025

View reviewed changes

Sashan force-pushed the ha-proxy.h1load branch 2 times, most recently from be2f9de to 7d10408 Compare December 17, 2025 14:33

Sashan moved this from In Progress to Waiting Review in Development Board Dec 17, 2025

vavroch2010 requested a review from jogme January 8, 2026 09:06

jogme requested changes Jan 19, 2026

View reviewed changes

vavroch2010 moved this from Waiting Review to In Progress in Development Board Jan 21, 2026

Sashan force-pushed the ha-proxy.h1load branch from 1c43370 to 7e377ad Compare January 27, 2026 07:09

vavroch2010 moved this from In Progress to Waiting Review in Development Board Jan 29, 2026

vavroch2010 requested review from mbroz and npajkovsky February 5, 2026 09:03

mbroz reviewed Feb 5, 2026

View reviewed changes

fixup! add support for h1load client to HA-proxy tests

1b38133

npajkovsky reviewed Feb 9, 2026

View reviewed changes

Sashan added 3 commits February 9, 2026 16:35

fixup! fixup! add support for h1load client to HA-proxy tests

2b16f3e

fixup! fixup! fixup! add support for h1load client to HA-proxy tests

e35e8f3

fixup! fixup! fixup! fixup! add support for h1load client to HA-proxy…

9fe53ab

… tests

npajkovsky approved these changes Feb 9, 2026

View reviewed changes

Sashan added 2 commits February 9, 2026 23:35

fixup! fixup! fixup! fixup! fixup! add support for h1load client to H…

1a8198b

…A-proxy tests

fixup! fixup! fixup! fixup! fixup! fixup! add support for h1load clie…

f23dbe8

…nt to HA-proxy tests

vavroch2010 moved this from Waiting Review to Waiting Merge in Development Board Feb 10, 2026

jogme approved these changes Feb 10, 2026

View reviewed changes

switch HA-proxy tests to h1load client #69

Are you sure you want to change the base?

switch HA-proxy tests to h1load client #69

Uh oh!

Conversation

Sashan commented Nov 27, 2025

Uh oh!

nhorman left a comment

Choose a reason for hiding this comment

Uh oh!

Sashan commented Dec 8, 2025

Uh oh!

wtarreau commented Dec 11, 2025

Uh oh!

Sashan commented Dec 12, 2025

Uh oh!

wtarreau commented Dec 12, 2025

Uh oh!

jogme left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

t-j-h commented Jan 26, 2026

Uh oh!

Sashan commented Jan 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jogme left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants