Skip to content

Instantly share code, notes, and snippets.

@ngauthier
Created March 12, 2012 01:14
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ngauthier/2019132 to your computer and use it in GitHub Desktop.
Save ngauthier/2019132 to your computer and use it in GitHub Desktop.
Ruby Workflow Benchmark

Ruby Workflow System Benchmark

Why?

I've chatted with people that love working on their macbook airs, and honestly, they have a lot going for them. They're nice light machines with a surprising amount of performance for their form factor.

I always argue that they're way slower than much cheaper machines, like a good workstation. Ergonomics aside, I wanted to get some performance numbers.

I've attached a script to this gist that will grab Sinatra and time how long it takes to bundle it, run the full tests, and run a single test.

This is a pretty lightweight project, so times should be short all-around. I'd like to do this with a rails app as well, but couldn't quickly find a medium size open source rails app.

OK, what should I do?

Please run the script, and in the comments fill out the following:

  • Computer type (Macbook Air, custom workstation, Thinkpad, etc)
  • Operating System
  • Hard disk (or SSD) type (and throughput if you know it)
  • Processor type and speed
  • Amount of RAM
  • Anything else you think contributes to the performance.

Get your results

Then, include the time line for each command. It should look something like:

0.56user 0.07system 0:00.64elapsed 98%CPU (0avgtext+0avgdata 103648maxresident)k

The first number is the most important here.

Compute your score

Now, for a completely arbitrary score, do the following computation:

(Bundle Time) + (Suite Time * 10) + (Test Time * 100)

To reflect a rough workflow of bundling once, running the suite before commits, and running individual files while developing.

Final Note

This benchmark is probably pretty flawed. Before complaining about potential bias or problems, come up with your own benchmark and let's try it.

#!/bin/sh
# Please use ruby-1.9.3-p0-falcon for best accuracy
git clone https://github.com/sinatra/sinatra.git
cd sinatra
git checkout 1.3.2
time bundle --path .bundle/gems
time bundle exec rake
time bundle exec ruby test/base_test.rb
@ngauthier
Copy link
Author

@MattRogish oops yeah. Sorry time changes formats when run as a script and I got confused. Updated.

@ngauthier
Copy link
Author

Yeah good call on bundler.

I'm going to try to setup a better benchmark with a medium size rails project if I can find one. This is preliminary :-)

@abachman
Copy link

real    0m26.267s
user    0m12.111s
sys   0m3.117s

real    0m4.661s
user    0m3.542s
sys   0m0.449s

real    0m0.450s
user    0m0.356s
sys   0m0.086s

Total score: 26.27 + (4.66 * 10) + (0.45 * 100) =

117.87

April 2011 15" MacBook Pro

  • APPLE SSD TS128C
  • 8GB 1333MHz DDR3
  • 2 GHz Intel Core i7

Enjoy configuring your video drivers, Linux workstation guy.

;|

@ngauthier
Copy link
Author

I love configuring video cards

@jszmajda
Copy link

Netbook I use for fun:

  • Lenovo S12
  • Ubuntu 11.10 x32
  • Atom N270 @ 1.60GHz
  • Fujitsu MHZ2160BH 160Gb 5200 rpm HD
  • 2Gb RAM

bundle 72.50user 8.01system 1:29.32elapsed 90%CPU (0avgtext+0avgdata 210288maxresident)k
suite 18.30user 0.91system 0:21.44elapsed 89%CPU (0avgtext+0avgdata 205568maxresident)k
test 2.64user 0.25system 0:02.92elapsed 99%CPU (0avgtext+0avgdata 80896maxresident)k
#595.72 (lulz)

My machine at the office:

  • 1 year old custom-ish cyberpower desktop
  • Ubuntu 11.10 x64
  • Core 2 Duo i7 @ 3.07 Ghz
  • Kingston SNV425S264GB SSD (200 mbps read, 110 mbps write)
  • 12Gb RAM·

bundle 13.16user 2.00system 0:19.28elapsed 78%CPU (0avgtext+0avgdata 281280maxresident)k
suite 3.02user 0.17system 0:03.54elapsed 90%CPU (0avgtext+0avgdata 332832maxresident)k
test 0.43user 0.04system 0:00.49elapsed 97%CPU (0avgtext+0avgdata 103680maxresident)k
#103.68

Does that mean I win? ;)

@bryanl
Copy link

bryanl commented Mar 12, 2012

On my i5 Macbook Air: 27.75 + (5.66_10) + (0.90_100) = 174.35
On my i7 2600 desktop (intel ssd + 8GB of memory): 13.6 + (2.64_10) + (0.38_100) = 78

Not sure what this proves. My desktop is way faster than my macbook air.

@ngauthier
Copy link
Author

ngauthier commented Mar 12, 2012 via email

@ngauthier
Copy link
Author

My 2 year old $400 workstation. This is my "work" computer since it has a higher processor and ssd (and it lives at work).

  • Ubuntu 11.10
  • AMD Phenom(tm) II X4 965 Processor @ 3.4ghz
  • 8GB RAM
  • SSD

16.06user 2.90system 0:25.09elapsed 75%CPU (0avgtext+0avgdata 269536maxresident)k
3.55user 0.20system 0:04.23elapsed 88%CPU (0avgtext+0avgdata 304736maxresident)k
0.53user 0.04system 0:00.57elapsed 99%CPU (0avgtext+0avgdata 103664maxresident)k

Score: 124.39

@MattRogish
Copy link

My work computer:

  • 15" MacBook Pro - Mid 2010
  • 2.4GHz i5
  • 4GB RAM
  • 256GB SSD

I got creamed in the install which I hope was network related (work network is inconsistent).

real 1m38.231s user 0m27.003s sys 0m4.128s

real 0m5.291s user 0m4.045s sys 0m0.499s

real 0m0.875s user 0m0.722s sys 0m0.140s
#98.231 + (5.291 * 10) + (0.875 * 100) = 238.641

@ngauthier
Copy link
Author

My new ivy bridge machine, overclocked!

  • Ubuntu 12.04
  • Intel Ivy Bridge i7-3770k OCed to 3.8ghz (4.2ghz turbo)
  • 16GB RAM
  • Intel SATA III 6gb/s SSD

6.90user 1.10system 0:12.35elapsed 64%CPU (0avgtext+0avgdata 267648maxresident)k
2.51user 0.17system 0:03.08elapsed 87%CPU (0avgtext+0avgdata 259088maxresident)k
0.32user 0.02system 0:00.35elapsed 96%CPU (0avgtext+0avgdata 109536maxresident)k
#78.15 :-D

@kmarsh
Copy link

kmarsh commented Jun 16, 2012

Fancy new base-level MacBook Pro with Retina Display. Mac OS X 10.7.4, ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin11.4.2], 2.3GHz quad-core Intel Core i7 (Ivy Bridge), 8GB RAM, 256GB SSD:

15.385 + (4.329_10) + (.526_100) = 111.275

Honestly not as impressive as I'd thought... the text in the terminal was nice and sharp, though! ;)

@MattRogish
Copy link

Here's my brand-new 2012 MBA w/2GHz Ivy Bridge, 8GB RAM, 256GB SSD:

real 0m13.498s
user 0m8.516s
sys 0m2.009s

real 0m4.146s
user 0m3.314s
sys 0m0.378s

real 0m0.591s
user 0m0.488s
sys 0m0.095s

13.498 + (4.146_10) + (0.591_100) = 114.83800

The bundle install numbers are likely skewed by network stuff - but clearly your test suite runs a lot faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment