Skip to content

Instantly share code, notes, and snippets.

@burke
Forked from funny-falcon/cumulative_performance.patch
Created January 27, 2012 13:44
Show Gist options
  • Save burke/1688857 to your computer and use it in GitHub Desktop.
Save burke/1688857 to your computer and use it in GitHub Desktop.
ruby-1.9.3-p327 cumulative performance patch for rbenv

ruby-1.9.3-p327 cumulative performance patch for rbenv

This installs a patched ruby 1.9.3-p327 with various performance improvements and a backported COW-friendly GC, all courtesy of funny-falcon.

Requirements

You will also need a C Compiler. If you're on Linux, you probably already have one or know how to install one. On OS X, you should install XCode, and brew install autoconf using homebrew.

Ok, let's do this!

curl https://raw.github.com/gist/1688857/rbenv.sh | sh ; rbenv global 1.9.3-p327-perf

BONUS!

Putting the following in your shell config (eg. ~/.bash_profile) will make Rails even faster, but will increase its memory footprint. For most applications, this will probably work well for you:

export RUBY_GC_MALLOC_LIMIT=60000000
export RUBY_FREE_MIN=200000

If you have a particularly large rails app (say, >100k lines), this seems to improve it even more:

export RUBY_GC_MALLOC_LIMIT=1000000000
export RUBY_FREE_MIN=500000
export RUBY_HEAP_MIN_SLOTS=40000

More BONUS!

If you're feeling adventurous, you can set some additional compile flags before you run the command above. These may cause the build to fail. If so, you can try again without the march flag, and it'll probably work. This generally makes my ruby ~1.5% faster. See http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel for more guidance. I had to change it to -march=core2 on my Macbook Air 2011/11".

export CFLAGS="-march=native -O3 -pipe -fomit-frame-pointer"

This has to be run BEFORE you compile ruby.

build_package_combined_patch() {
local package_name="$1"
{
curl https://raw.github.com/gist/4136373/falcon-gc.diff | patch -p1
autoconf
./configure --prefix="$PREFIX_PATH" $CONFIGURE_OPTS
make -j 8
make install
} >&4 2>&1
}
install_package "yaml-0.1.4" "http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz"
install_package "ruby-1.9.3-p327" "http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p327.tar.gz" combined_patch
VERSION="1.9.3-p327"
curl https://raw.github.com/gist/1688857/2-$VERSION-patched.sh > /tmp/$VERSION-perf
rbenv install /tmp/$VERSION-perf
@dogweather
Copy link

The RVM recipe worked perfectly for me. Thanks for keeping it updated.

How can we help get this pulled into the master branch?

@julian7
Copy link

julian7 commented Aug 22, 2012

mountain lion + xcode4.4.1 + brew install autoconf + remove require_gcc == win

@burke
Copy link
Author

burke commented Aug 22, 2012

Thanks @samgranieri and @julian7, I forgot that line was in there. I've updated the gist.

@davefp
Copy link

davefp commented Aug 24, 2012

I would change the 'install XCode' to reference the Command Line Tools instead. See my fork here: https://gist.github.com/3453141/76972c7185b579f5542d12cf56eae8d068c908ca

@seangeo
Copy link

seangeo commented Sep 11, 2012

This patch is available in rvm stable, so if you don't want to live on the edge you can do rvm get stable instead.

@funny-falcon
Copy link

@arikfr
Copy link

arikfr commented Sep 25, 2012

@burke I think you should update your instructions to use RVM stable rather than head, because the patch is already available in RVM stable and at the moment head seems quite unstable. It took me lots of digging and trial and error until I figured that I failed to compile Ruby with this patch because I was on the head version rather stable.

@whalesalad
Copy link

VERY cool! Here are my before stats, the base 1.9.3-p194:

mwhalen at Ajax in ~/Sites/askherzl on master!
± time rails r "puts 1"
Connecting to database specified by database.yml
1
rails r "puts 1"  4.23s user 1.19s system 54% cpu 10.009 total

And here is the same command after installing the patch, adding the suggested lines to my ~/.zshrc and re-running bundle:

mwhalen at Ajax in ~/Sites/askherzl on master!
± time rails r "puts 1"
Connecting to database specified by database.yml
1
rails r "puts 1"  2.95s user 0.90s system 58% cpu 6.549 total

That's pretty much an exact 35% increase in performance. Installing gem's feels a great deal quicker as well.

@dvandersluis
Copy link

Any idea why execution would actually be slower with this patch? Using RVM on a retina MBP:

ruby-1.9.3-p194:

$ time rails runner 'puts 3'
real    0m7.454s
user    0m6.481s
sys 0m0.957s

ruby-1.9.3-p194-perf:

$ time rails runner 'puts 3'
real    0m12.410s
user    0m11.347s
sys 0m1.036s

ruby-1.9.3-p194-perf with exports

$ time rails runner 'puts 3'
real    0m9.609s
user    0m8.344s
sys 0m1.250s

@guange2015
Copy link

@dvandersluis
the same with you.

@ryansch
Copy link

ryansch commented Oct 1, 2012

Should we be using falcon's new patch now?

@dvandersluis
Copy link

I tried benchmarking falcon's new patch on top of vanilla 1.9.3-p194 against 1.9.3-p194-perf: (using avgtime):

$ rvm current && avgtime -q -r5 -d -p rails runner "puts 1"
ruby-1.9.3-p194-perf

------------------------
Total time (ms): 46419.8
Repetitions    : 5
Sample mode    : 9320 (1 ocurrences)
Median time    : 9282.5
Avg time       : 9283.95
Std dev.       : 23.6096
Minimum        : 9257.65
Maximum        : 9321.53
95% conf.int.  : [9237.68, 9330.23]  e = 46.2739
99% conf.int.  : [9223.14, 9344.77]  e = 60.8142
EstimatedAvg95%: [9263.26, 9304.65]  e = 20.6943
EstimatedAvg99%: [9256.76, 9311.15]  e = 27.1969
Sorted times   : 
    [9257.65, 9261.26, 9282.5, 9296.83, 9321.53]

$ rvm current && avgtime -q -r5 -d -p rails runner "puts 1"
ruby-1.9.3-p194-falcon-new

------------------------
Total time (ms): 35149.3
Repetitions    : 5
Sample mode    : 6970 (2 ocurrences)
Median time    : 7032.1
Avg time       : 7029.87
Std dev.       : 52.5191
Minimum        : 6970.93
Maximum        : 7113.02
95% conf.int.  : [6926.93, 7132.8]  e = 102.936
99% conf.int.  : [6894.59, 7165.15]  e = 135.28
EstimatedAvg95%: [6983.83, 7075.9]  e = 46.0342
EstimatedAvg99%: [6969.37, 7090.37]  e = 60.4991
Sorted times   : 
    [6970.93, 6977.6, 7032.1, 7055.69, 7113.02]

@aaronjensen
Copy link

There are some issues w/ RVM when installing these. If you rename the ruby (by calling it 1.9.3-perf) or whatever, it won't include the cflags patch. So you'll actually lose quite a bit of performance doing this. Be sure to include --patch cflags.patch Also it is best to include --disable-shared

@caarlos0
Copy link

kewl! will surely test it when I get home. Thanks for hard work.

@caarlos0
Copy link

kewl! will surely test it when I get home. Thanks for hard work.

@rwz
Copy link

rwz commented Nov 30, 2012

BUILD FAILED

Inspect or clean up the working tree at /var/folders/50/nmb34mx94js1fd1z_s5fl8pr0000gn/T/ruby-build.20121130224625.6476
Results logged to /var/folders/50/nmb34mx94js1fd1z_s5fl8pr0000gn/T/ruby-build.20121130224625.6476.log

Last 10 log lines:
ossl_x509store.c:520: warning: ‘X509_STORE_CTX_set_flags’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:464)
ossl_x509store.c: In function ‘ossl_x509stctx_set_purpose’:
ossl_x509store.c:532: warning: ‘X509_STORE_CTX_set_purpose’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:460)
ossl_x509store.c: In function ‘ossl_x509stctx_set_trust’:
ossl_x509store.c:544: warning: ‘X509_STORE_CTX_set_trust’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:461)
ossl_x509store.c: In function ‘ossl_x509stctx_set_time’:
ossl_x509store.c:561: warning: ‘X509_STORE_CTX_set_time’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:466)
linking shared-object dl/callback.bundle
linking shared-object openssl.bundle
make: *** [build-ext] Error 2

@cannikin
Copy link

Mother of God...

Running this patch against p327 is working. I'm running RVM. Here are my steps (pretty simple):

brew install autoconf automake
curl https://raw.github.com/gist/4136373/falcon-gc.diff > $rvm_path/patches/ruby/1.9.3/p327/falcon.patch
rvm install 1.9.3-p327 -n fast --patch falcon

Then recreate my gemsets and bundle of course.

Benchmarks?

p286:

time rake
...
real    0m48.761s
user    0m41.117s
sys     0m3.561s

time rails runner puts 'foobar'
....
real    0m4.939s
user    0m4.210s
sys     0m0.719s

p327 + patch:

time rake
...
real    0m20.775s
user    0m15.543s
sys     0m2.208s

time rails runner puts 'foobar'
...
real    0m2.610s
user    0m2.047s
sys     0m0.553s

Greatest patch ever!!

@duff
Copy link

duff commented Dec 1, 2012

This is really great. I ran a few benchmarks against 2 separate apps. We're seeing a 28% - 49% speed increase depending on the test!

@davidcelis
Copy link

The rbenv installation gets build failures on Mountain Lion:

~Code/goodbre.ws master % curl https://raw.github.com/gist/1688857/rbenv.sh | sh ; rbenv global 1.9.3-p327-perf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   138  100   138    0     0    180      0 --:--:-- --:--:-- --:--:--   467
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   448  100   448    0     0   1194      0 --:--:-- --:--:-- --:--:--  1539
Downloading yaml-0.1.4.tar.gz...
-> http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
Installing yaml-0.1.4...
Installed yaml-0.1.4 to /Users/david/.rbenv/versions/1.9.3-p327-perf

Downloading ruby-1.9.3-p327.tar.gz...
-> http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p327.tar.gz
Installing ruby-1.9.3-p327...

BUILD FAILED

Inspect or clean up the working tree at /var/folders/tt/0zgjrcdd4zn0pvfmmp4s9n600000gq/T/ruby-build.20121206132529.63012
Results logged to /var/folders/tt/0zgjrcdd4zn0pvfmmp4s9n600000gq/T/ruby-build.20121206132529.63012.log

Last 10 log lines:
ossl_x509store.c: In function ‘ossl_x509stctx_set_flags’:
ossl_x509store.c:520: warning: ‘X509_STORE_CTX_set_flags’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:464)
ossl_x509store.c: In function ‘ossl_x509stctx_set_purpose’:
ossl_x509store.c:532: warning: ‘X509_STORE_CTX_set_purpose’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:460)
ossl_x509store.c: In function ‘ossl_x509stctx_set_trust’:
ossl_x509store.c:544: warning: ‘X509_STORE_CTX_set_trust’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:461)
ossl_x509store.c: In function ‘ossl_x509stctx_set_time’:
ossl_x509store.c:561: warning: ‘X509_STORE_CTX_set_time’ is deprecated (declared at /usr/include/openssl/x509_vfy.h:466)
linking shared-object openssl.bundle
make: *** [build-ext] Error 2

@davidcelis
Copy link

Looks like the problem was that my CONFIGURE_OPTS weren't getting picked up. If you're getting build failures on OSX, install it with the following so that it gets compiled correctly with support for the right OpenSSL and Readline libs:

brew install openssl readline
curl https://raw.github.com/gist/1688857/2-$VERSION-patched.sh > /tmp/1.9.3-p327-perf
CONFIGURE_OPTS="--with-openssl-dir=`brew --prefix openssl` --with-readline-dir=`brew --prefix readline`" rbenv install /tmp/1.9.3-p327-perf

@davidcelis
Copy link

Boy I sure wish that gist comments were editable. That should have been:

brew install openssl readline
curl https://raw.github.com/gist/1688857/2-1.9.3-p327-patched.sh > /tmp/1.9.3-p327-perf
CONFIGURE_OPTS="--with-openssl-dir=`brew --prefix openssl` --with-readline-dir=`brew --prefix readline`" rbenv install /tmp/1.9.3-p327-perf

@jeroenj
Copy link

jeroenj commented Jan 2, 2013

Is this patch still relevant for 1.9.3-p362?

As expected applying it to p362 doesn't work.

@kaspergrubbe
Copy link

I got issues when compiling, and I would see this:

checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: C compiler cannot create executables

I tracked down the issue to the "-march=native"-flag. When removed or changed to "-march=nocona" (on my Late 2008 Macbook with a Intel Core 2 Duo running Mountain Lion) it worked just fine.

I hope others won't hit this bug by reading this, because it has costed me some time tracking it down.

@agis
Copy link

agis commented Feb 6, 2013

Mountain Lion+RVM users follow instructions by @cannikin.

@NielsKSchjoedt
Copy link

And if you are getting Error running 'make ' you might have to append --with-gcc=gcc so the line looks like: ``rvm install 1.9.3-p327 -n fast --patch falcon --with-gcc=gcc`

@krohrbaugh
Copy link

ruby-build has S3-based mirrors for the various dependencies, which you can make use of in this gist, for times when sites like pyyaml.org go down.

To do so, change line 14 and 15 of 2-1.9.3-p327-patched.sh to be:

install_package "yaml-0.1.4" "http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz#36c852831d02cf90508c29852361d01b"
install_package "ruby-1.9.3-p327" "http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p327.tar.gz#96118e856b502b5d7b3a4398e6c6e98c" combined_patch

(I'm simply pulling these out of ruby-build's relevant manifest files here.)

That will attempt to download from the source and then fall-back to the S3 mirror if needed.

@umtrey
Copy link

umtrey commented Nov 21, 2013

This hasn't been touched in a while, but with the new gist links, things will likely break. I've made a version of the rbenv definition file that should work, for the time being.
https://gist.github.com/umtrey/7592062

@m-shirshendu
Copy link

To get rid of the following error:

regparse.c:582:15: error: implicit conversion loses integer precision: 'st_index_t' (aka 'unsigned long') to 'int' [-Werror,-Wshorten-64-to-32]

In addition to https://gist.github.com/umtrey/7592062, I had to do the following first

brew update
brew tap homebrew/dupes
brew install apple-gcc42
export CC=gcc-4.2

Then rbenv install /tmp/1.9.3-p327-perf

@chuckbergeron
Copy link

Thanks @m-shirshendu! That did the trick.

@agis
Copy link

agis commented Jul 9, 2014

Trying to install with ruby-install I got the following error (Using GCC 4.2):

....
compiling proc.c
compiling file.c
make: *** No rule to make target `pool_alloc.inc.h', needed by `gc.o'.  Stop.
!!! Compiling ruby 1.9.3-p327 failed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment