Skip to content

Instantly share code, notes, and snippets.

@haproxytechblog

haproxytechblog/blog20210405-01.sh

Created April 5, 2021 15:25
Show Gist options
  • Save haproxytechblog/1013c08a5bc0d24670c8b1901c6067fd to your computer and use it in GitHub Desktop.
Save haproxytechblog/1013c08a5bc0d24670c8b1901c6067fd to your computer and use it in GitHub Desktop.
Download ZIP
HAProxy Forwards Over 2 Million HTTP Requests per Second on a single AWS Arm Instance
Raw
blog20210405-01.sh
$ lscpu -e
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
0 0 0 0 0:0:0:0 yes
1 0 0 1 1:1:1:0 yes
2 0 0 2 2:2:2:0 yes
3 0 0 3 3:3:3:0 yes
...
60 0 0 60 60:60:60:0 yes
61 0 0 61 61:61:61:0 yes
62 0 0 62 62:62:62:0 yes
63 0 0 63 63:63:63:0 yes
Raw
blog20210405-02.sh
$ sudo systemctl stop irqbalance
$ sudo systemctl stop snapd
$ sudo systemctl stop cron
Raw
blog20210405-03.sh
$ sudo ~/dpbench/scripts/set-irq.sh ens5 16
Raw
blog20210405-04.sh
$ ulimit -n 100000
$ for i in {2..47}; do taskset -c 2-47 httpterm -D -L :8000;done
Raw
blog20210405-05.sh
$ ./haproxy -v
HA-Proxy version 2.3.7 2021/03/16 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2022.
Known bugs: http://www.haproxy.org/bugs/bugs-2.3.7.html
Running on: Linux 5.4.0-1038-aws #40-Ubuntu SMP Fri Feb 5 23:53:34 UTC 2021 aarch64
Raw
blog20210405-06.cfg
defaults
mode http
timeout client 60s
timeout server 60s
timeout connect 1s
listen px
bind :8000
balance random
server s1 172.31.36.194:8000
Raw
blog20210405-07.sh
$ ulimit -n 100000
$ taskset -c 2-47 haproxy -D -f test.cfg
Raw
blog20210405-08.sh
$ ulimit -n 100000
$ taskset -c 2-47 h1load -e -ll -P -t 46 -s 30 -d 120 -c 1150 http://172.31.37.79:8000/ | tee cli-run.dat
Raw
blog20210405-09.sh
$ ulimit -n 100000
$ taskset -c 1 h1load -e -ll -P -t 1 -s 30 -d 120 -R 1000 -c 128 http://172.31.36.194:8000/ | tee cli-ref.dat
Raw
blog20210405-10.sh
top - 05:43:31 up 29 min, 4 users, load average: 16.62, 12.76, 8.62
Tasks: 639 total, 2 running, 637 sleeping, 0 stopped, 0 zombie
%Cpu(s): 44.5 us, 33.2 sy, 0.0 ni, 22.2 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 126544.6 total, 125538.5 free, 568.9 used, 437.1 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 124966.6 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2044 ubuntu 20 0 3356916 27312 6260 R 4194 0.0 20:09.40 haproxy+
336 root 20 0 0 0 0 S 8.9 0.0 0:24.27 ksoftir+
355 root 20 0 0 0 0 S 5.9 0.0 0:23.12 ksoftir+
361 root 20 0 0 0 0 S 5.9 0.0 0:19.64 ksoftir+
409 root 20 0 0 0 0 S 5.9 0.0 0:23.90 ksoftir+
373 root 20 0 0 0 0 S 5.0 0.0 0:22.96 ksoftir+
379 root 20 0 0 0 0 S 5.0 0.0 0:18.61 ksoftir+
391 root 20 0 0 0 0 S 5.0 0.0 0:22.93 ksoftir+
415 root 20 0 0 0 0 S 5.0 0.0 0:17.42 ksoftir+
385 root 20 0 0 0 0 S 4.0 0.0 0:18.74 ksoftir+
397 root 20 0 0 0 0 S 4.0 0.0 0:15.33 ksoftir+
432 root 20 0 0 0 0 S 4.0 0.0 0:15.12 ksoftir+
450 root 20 0 0 0 0 S 4.0 0.0 0:17.04 ksoftir+
367 root 20 0 0 0 0 S 3.0 0.0 0:16.48 ksoftir+
403 root 20 0 0 0 0 S 3.0 0.0 0:17.66 ksoftir+
444 root 20 0 0 0 0 S 3.0 0.0 0:12.70 ksoftir+
Raw
blog20210405-11.sh
top - 06:42:08 up 1:28, 4 users, load average: 40.05, 20.99, 12.57
Tasks: 639 total, 2 running, 637 sleeping, 0 stopped, 0 zombie
%Cpu(s): 54.9 us, 31.3 sy, 0.0 ni, 13.7 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 126544.6 total, 125279.7 free, 626.9 used, 638.0 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 124893.0 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5433 ubuntu 20 0 3357232 39948 6428 R 4597 0.0 260:36.56 haproxy
373 root 20 0 0 0 0 S 8.9 0.0 1:32.24 ksoftir+
391 root 20 0 0 0 0 S 6.9 0.0 1:29.67 ksoftir+
409 root 20 0 0 0 0 S 5.9 0.0 1:26.27 ksoftir+
379 root 20 0 0 0 0 S 5.0 0.0 1:34.07 ksoftir+
415 root 20 0 0 0 0 S 5.0 0.0 1:20.57 ksoftir+
438 root 20 0 0 0 0 S 5.0 0.0 1:16.82 ksoftir+
355 root 20 0 0 0 0 S 4.0 0.0 1:27.25 ksoftir+
361 root 20 0 0 0 0 S 4.0 0.0 1:24.02 ksoftir+
385 root 20 0 0 0 0 S 4.0 0.0 1:19.53 ksoftir+
432 root 20 0 0 0 0 S 4.0 0.0 1:08.21 ksoftir+
450 root 20 0 0 0 0 S 4.0 0.0 1:13.43 ksoftir+
336 root 20 0 0 0 0 S 3.0 0.0 1:31.32 ksoftir+
367 root 20 0 0 0 0 S 3.0 0.0 1:20.86 ksoftir+
403 root 20 0 0 0 0 S 3.0 0.0 1:12.48 ksoftir+
397 root 20 0 0 0 0 S 2.0 0.0 1:10.06 ksoftir+
Raw
blog20210405-12.sh
> show profiling
Tasks activity:
function calls cpu_tot cpu_avg lat_tot lat_avg
h1_io_cb 3667960 9.089s 2.477us 12.88s 3.510us
si_cs_io_cb 3487273 4.971s 1.425us 12.44m 214.0us
process_stream 2330329 10.20s 4.378us 21.52m 554.1us
ssl_sock_io_cb 2134538 2.612m 73.43us 6.315h 10.65ms
h1_timeout_task 1553472 - - 6.881m 265.8us
accept_queue_process 320615 13.72m 2.568ms 8.094m 1.515ms
task_run_applet 58 3.897ms 67.19us 3.658ms 63.07us
session_expire_embryonic 1 - - 294.0ns 294.0ns
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment