Comparing Windows MRI 1.9.3, jRuby 1.7.0 RC, Linux MRI 1.9.3

rubies-on-windows-compared.md

Test Setup

• All tests run on same machine: Dell Studio XPS 16 Laptop running Windows 7 x64
• Same codebase except jRuby does use different sqlite adaptor (you cannot use the standard one with jRuby)
• Code has 4 unit tests (incl. 21 assertions) that all pass.
• Code is the example app from Agile Web Development with Rails

Goals

• I wanted to find the fastest way to run my rails tests on my laptop.

Notes

• rake environment and rake noop explained at http://www.ruby-forum.com/topic/121459 (they just test the setup portion of the rake execution time and a dummy rake task respectively)

Legend for ruby versions

• mri: ruby 1.9.3p194 (2012-04-20) [i386-mingw32]
• jruby: jruby 1.7.0.RC2 (1.9.3p203) 2012-10-09 142cc7b on Java HotSpot(TM) Client VM 1.7.0_07-b11 [Windows 7-x86]
• lin-mri: ruby 1.9.3p286 (2012-10-12 revision 37165) [i686-linux] run in Ubuntu 12 VM on Windows 7 Host OS
• cyg-mir: ruby 1.9.3p194 (2012-04-20) [i386-mingw32] run via cygwin zsh shell
• lin-jruby jurby 1.6.8 on Ubuntu 12.04 (installed via rvm)

Results

    rake test:units     mri         1m40s    1m43s    1m39s
    rake test:units     cyg-mri     1m40s    1m41s    1m47s
    rake test:units     jruby       1m12s    1m14s    1m13s
    rake test:units     lin-mri       2vs      29s      26s
    rake test:units     lin-jruby     30s      29s      27s (seemed to take a few runs to warm up)

    rake environment    mri           29s      28s      28s
    rake environment    cyg-mri       28s      28s      26s
    rake environment    jruby         27s      26s      25s
    rake environment    lin-mri       10s      10s      11s
    rake environment    lin-jruby     10s      11s      10s

    rake noop           mri           13s      13s      13s
    rake noop           cyg-mri       12s      13s      13s
    rake noop           jruby         18s      16s      16s
    rake noop           lin-mri        6s       6s       6s
    rake noop           lin-jruby      8s       8s       8s

Conclusion

• The fastest way to run tests seems to be via the Ubuntu VM.
• It doesn't seem to matter whether you run Windows MRI via Cygwin or cmd.exe
• jRuby is generally faster than MRI on Windows. Setting up rails for jRuby is more work (due to Java dependencies & needing the non-standard SQLite3 adaptor)

Todo

• I assume rake is I/O constrained? Maybe an SSD would help ...
• Try a rails project with a larger test suite and compare.