Create a gist now

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Simple benchmark against STF Distribute Object Storage written in Perl and in Go

STF Distribute Object Storage (perl vs go)

This is a VERY limited benchmark. Take with a grain of salt. (go-stf-server)[https://github.com/stf-storage/go-stf-server] is still in half baked state, but can connect to the same database as the more mature (perl version)[https://github.com/stf-storage/stf], and handle GET requests.

And since the go version is still half-baked, this benchmark is for GET only.

Basic benchmark strategy

The original STF dispatcher PSGI application is running behind nginx reverse proxy on the same host, which handles the reproxying via X-Reproxy-URL. In this benchmark, I'm just sending GET requests directly to port 5000 (PSGI), and port 8080 (go) on the localhost via ab:

ab -n 10000 -c 100 $url

About GOMAXPROCS: The go version automatically adjusts runtime.GOMAXPROCS() to runtime.NumCPU(), and in this case GOMAXPROCS is set to "2"

About debugging output: Both versions have their debugging output ENABLED. Therefore, they are suffering some performance penalty. As an example, the go version can top at about 10~20% more requests per seconds when this output is disabled.

Results

Go version

  1. 733.65 req/sec
  2. 831.60 req/sec
  3. 866.82 req/sec
  4. 849.57 req/sec
  5. 737.27 req/sec
  6. 891.89 req/sec
  7. 790.60 req/sec
  8. 661.49 req/sec
  9. 776.47 req/sec
  10. 710.23 req/sec

(mean: 789.96 req/sec)

The dispatcher was compiled using go 1.2.

Some of the slower performance figures above seems to have resulted from reaching too high a load average during the benchmark. Load average went to about 89 during the benchmark When benchmarks are run during load average around 12, the results were around 800~900

Below is the list of results when benchmarks were run while load averages were < 2:

  1. 976.08 req/sec
  2. 900.81 req/sec
  3. 790.56 req/sec
  4. 997.57 req/sec
  5. 1014.03 req/sec

(mean: 935.81 req/sec)

Perl version

  1. 676.43 req/sec
  2. 684.24 req/sec
  3. 681.02 req/sec
  4. 634.54 req/sec
  5. 631.76 req/sec
  6. 626.48 req/sec
  7. 636.36 req/sec
  8. 645.37 req/sec
  9. 621.78 req/sec
  10. 623.42 req/sec

(mean: 646.14 req/sec)

The perl version uses per-5.12.2 (Yes, a bit old). The perl version seems to use much less CPU, as the load average would only go up to about 6, and seems to be a bit more stable. This may be because the PSGI version is run using a prefork server with number of jobs per process controlled more carefully.

Conclusion?

Well, no conclusions yet. But at first glance, the go version seems to outperform the perl version. The only thing that kind of eeks me at the moment is that the go version seems to use a lot more CPU. If there's a good way to control that, I'm all ears.

I'll get back to doing more benchmarks on when go-stf-server becomes more feature-complete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment