test to install symfony deps using PHP and HHVM

test.sh

#!/bin/bash

# Instructions
# run: php composer.phar create-project symfony/framework-standard-edition test 2.4.3  --no-scripts
# cd test and download and run this script

if [ ! -f composer.phar ]; then
	curl -sS https://getcomposer.org/installer | php
fi

echo Deleting vendor...
rm -rf vendor

echo PHP test

time php composer.phar install --prefer-dist --no-scripts

echo Deleting vendor...
rm -rf vendor

echo HHVM test

time hhvm composer.phar install --prefer-dist --no-scripts

After the yesterday conversation in twitter, i just spend some time making more tests and investigations.

Speaking with @danslo dont exists any benefit in run code in CLI with HHVM except long term executions, because the JIT cache is just stored in ram and every run need to recreate it, the HHVM team is working in solving this but currently, looks like dont have seens execute composer under HHVM by performance reasons

In my Macbook Air under hhvm-vagrant and KnpIpsum repository, and a previous standard installation with all in cache, my results are:

PHP 5.3.10-1ubuntu3.11
rm -rf vendor/ && time php composer.phar install --prefer-dist --no-scripts > /dev/null

real    0m3.724s
real    0m3.485s
real    0m3.359s
real    0m3.234s
real    0m3.510s

HipHop VM 3.1.0-dev (rel)
rm -rf vendor/ && time hhvm composer.phar install --prefer-dist --no-scripts > /dev/null

real    0m3.727s
real    0m3.517s
real    0m3.549s
real    0m3.742s
real    0m3.668s

With --dry-run as @stof suggested, the results are very different:

PHP 5.3.10-1ubuntu3.11
rm -rf vendor/ && time php composer.phar install --prefer-dist --no-scripts --dry-run > /dev/null

real    0m0.573s
real    0m0.514s
real    0m0.531s
real    0m0.574s
real    0m0.529s

HipHop VM 3.1.0-dev (rel)
rm -rf vendor/ && time hhvm composer.phar install --prefer-dist --no-scripts --dry-run > /dev/null

real    0m1.789s
real    0m1.860s
real    0m1.863s
real    0m1.696s
real    0m1.709s

So, we agree that running composer install with HHVM results in slower execution than PHP. I understand the issue with the JIT cache on CLI. But in my tests, HHVM takes longer than PHP, much longer than the JIT initialization. In the best case scenario, HHVM should be the same or roughly the same speed as PHP, not many seconds slower. The bigger the number of deps, the larger the gap is.

My point is, lots of people recommend to run composer with HHVM which is fine if you are using it for doing the update. But if you are using it for install then it will run slower. So doing the aliasing to always run composer with HHVM is not a good idea at the moment, until the HHVM team improves the performance around whatever composer install is doing.

@stof adding --dry-run clearly shows a major improvement in execution time, but why would I do that? Nobody will run composer with --dry-run. I want to test a real use case, not a theoretic one.

Hack vs PHP (Computer Language Benchmarks Game):

http://benchmarksgame.alioth.debian.org/u64q/hack.html