Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Apex Up Native Golang Benchmarks

Introduction

The following is a very rough set of benchmarks to get an initial sense of the performance difference between up with the nodejs shim and with using native Go support in Lambda. While not rigorous, the summary conclusion is that other than warmup there is not a sigdifference between the two to warrent more rigorous testing.

Methodology

This test utilized up 0.5.4 as well as a custom version that only utilizes native Go support in Lambda. The testing was done with the simple Golang example app found in the documentation in eu-west-3. An EC2 instance in the same region ran various tests using Vegeta using stock AWS Ubuntu 16.04 image with the ulimit increated to 65000. After a clean up deploy, a 3 second warmup at 10RPS was executed before recording the results.

Results

Up 0.5.4 (with nodejs shim)

Testing at low load (10 RPS)...
Requests      [total, rate]            3000, 10.00
Duration      [total, attack, wait]    4m59.932879955s, 4m59.899999532s, 32.880423ms
Latencies     [mean, 50, 95, 99, max]  36.57345ms, 27.338771ms, 73.54883ms, 125.195094ms, 267.077039ms
Bytes In      [total, mean]            60000, 20.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:3000  
Error Set:
Testing at medium load (50RPS)...
Requests      [total, rate]            15000, 50.00
Duration      [total, attack, wait]    5m0.037250257s, 4m59.97999952s, 57.250737ms
Latencies     [mean, 50, 95, 99, max]  34.028286ms, 23.363904ms, 46.34299ms, 104.183687ms, 9.919265127s
Bytes In      [total, mean]            300000, 20.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:15000  
Error Set:
Testing at highish load (250RPS)...
Requests      [total, rate]            75000, 250.00
Duration      [total, attack, wait]    5m0.04851071s, 4m59.995999523s, 52.511187ms
Latencies     [mean, 50, 95, 99, max]  34.285ms, 21.837358ms, 50.349441ms, 87.630636ms, 11.828546582s
Bytes In      [total, mean]            1500000, 20.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:75000  

Up Custom (golang native)

Testing at low load (10 RPS)...
Requests      [total, rate]            3000, 10.00
Duration      [total, attack, wait]    4m59.930329907s, 4m59.899999539s, 30.330368ms
Latencies     [mean, 50, 95, 99, max]  44.19875ms, 31.187513ms, 70.141395ms, 118.423916ms, 7.837374018s
Bytes In      [total, mean]            60000, 20.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:3000  
Error Set:
Testing at medium load (50RPS)...
Requests      [total, rate]            15000, 50.00
Duration      [total, attack, wait]    5m0.024459414s, 4m59.979999547s, 44.459867ms
Latencies     [mean, 50, 95, 99, max]  33.32933ms, 31.962925ms, 58.403387ms, 85.742701ms, 310.92098ms
Bytes In      [total, mean]            300000, 20.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:15000  
Error Set:
Testing at highish load (250RPS)...
Requests      [total, rate]            75000, 250.00
Duration      [total, attack, wait]    5m0.054307958s, 4m59.995999557s, 58.308401ms
Latencies     [mean, 50, 95, 99, max]  31.110505ms, 26.176661ms, 54.719465ms, 89.843425ms, 1.412126078s
Bytes In      [total, mean]            1500000, 20.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:75000  
Error Set:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.