The following is a very rough set of benchmarks to get an initial sense of the performance difference between up with the
nodejs shim and with using native Go support in Lambda. While not rigorous, the summary conclusion is that other than warmup
there is not a sigdifference between the two to warrent more rigorous testing.
This test utilized up 0.5.4 as well as a custom version that only utilizes
native Go support in Lambda. The testing was done with the simple Golang example app found in the documentation in eu-west-3.
An EC2 instance in the same region ran various tests using Vegeta using stock
AWS Ubuntu 16.04 image with the ulimit increated to 65000. After a clean up deploy, a 3 second warmup at 10RPS
was executed before recording the results.
Testing at low load (10 RPS)...
Requests [total, rate] 3000, 10.00
Duration [total, attack, wait] 4m59.932879955s, 4m59.899999532s, 32.880423ms
Latencies [mean, 50, 95, 99, max] 36.57345ms, 27.338771ms, 73.54883ms, 125.195094ms, 267.077039ms
Bytes In [total, mean] 60000, 20.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:3000
Error Set:
Testing at medium load (50RPS)...
Requests [total, rate] 15000, 50.00
Duration [total, attack, wait] 5m0.037250257s, 4m59.97999952s, 57.250737ms
Latencies [mean, 50, 95, 99, max] 34.028286ms, 23.363904ms, 46.34299ms, 104.183687ms, 9.919265127s
Bytes In [total, mean] 300000, 20.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:15000
Error Set:
Testing at highish load (250RPS)...
Requests [total, rate] 75000, 250.00
Duration [total, attack, wait] 5m0.04851071s, 4m59.995999523s, 52.511187ms
Latencies [mean, 50, 95, 99, max] 34.285ms, 21.837358ms, 50.349441ms, 87.630636ms, 11.828546582s
Bytes In [total, mean] 1500000, 20.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:75000
Testing at low load (10 RPS)...
Requests [total, rate] 3000, 10.00
Duration [total, attack, wait] 4m59.930329907s, 4m59.899999539s, 30.330368ms
Latencies [mean, 50, 95, 99, max] 44.19875ms, 31.187513ms, 70.141395ms, 118.423916ms, 7.837374018s
Bytes In [total, mean] 60000, 20.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:3000
Error Set:
Testing at medium load (50RPS)...
Requests [total, rate] 15000, 50.00
Duration [total, attack, wait] 5m0.024459414s, 4m59.979999547s, 44.459867ms
Latencies [mean, 50, 95, 99, max] 33.32933ms, 31.962925ms, 58.403387ms, 85.742701ms, 310.92098ms
Bytes In [total, mean] 300000, 20.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:15000
Error Set:
Testing at highish load (250RPS)...
Requests [total, rate] 75000, 250.00
Duration [total, attack, wait] 5m0.054307958s, 4m59.995999557s, 58.308401ms
Latencies [mean, 50, 95, 99, max] 31.110505ms, 26.176661ms, 54.719465ms, 89.843425ms, 1.412126078s
Bytes In [total, mean] 1500000, 20.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:75000
Error Set: