Skip to content

Instantly share code, notes, and snippets.

@joakimk

joakimk/gist:720660

Created Nov 29, 2010
Embed
What would you like to do?
Benchmark notes, testbot @ EC2

Most cost effective instance type?

I did a bit of benchmarking using our rspec suite and the most cost effective flavor seems to be "c1.medium". It's almost twice as fast as s1.small but still has the same per-core price. The micro instances were about 4 times slower than c1.medium.

The c1.xlarge instance did not perform better than c1.medium (actually worse) and costs as much per core.

I did not try the cluster nodes as they require you to create your own image to use it.

Cost

The spot instance price of c1.medium seems to be stable at $0.06 / hour ($0.17 normal). That would be about $100 a month for 10 instances running 9 hours a day, 20 days a month.

Benchmarks

Our ~30-35 minute test suite using:

  • 10 x c1.medium instances (20 cores): 2.8 minutes
  • 15 x c1.medium instances (30 cores): 2.3 minutes
  • 15 x c1.medium, and our ordinary hardware (57 cores): 1.8 minutes
  • Just the ordinary hardware (27 cores): 2.4 minutes

A few limiting factors here are:

  • Some of our spec files take up to 60 seconds to run
  • Testbot could do a better job of balancing test load
  • Syncing / communication (actually only about 15 secs)

In case you're curious, our ordinary hardware is:

  • 8 "old" computers (core 2 duo, 2.x ghz) that we PXE boot to not have to maintain separate installations.
  • 3 virtual computers using spare resources in our office and dev servers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment