Benchmark https://github.com/mtdowling/ClassPreloader on symfony2 standard edition homepage.

0desc.md

Note that the prod environment home page is a 404, and this is a micro/unreal benchmark

I also had to fix some bugs manually, they're not all documented here, pr/issue soon

The tests were run on an EC2 small instance, Ubuntu 11.10.

$ php -v
PHP 5.4.6-1ubuntu1.1 (cli) (built: Nov 15 2012 01:18:34) 
Copyright (c) 1997-2012 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies

1summary.md

apc.stat default (not 0)

• web/app.php 19.91 [#/sec] (mean)
• web/app_preloader.php 17.57 [#/sec] (mean)
• web/app_dev.php 9.23 [#/sec] (mean)
• web/app_dev_preloader.php 8.42 [#/sec] (mean)

apc.stat=0

• web/app.php 20.56 [#/sec] (mean)
• web/app_preloader.php 19.87 [#/sec] (mean)

2preloaderConfig.php

<?php

require __DIR__ . '/vendor/classpreloader/classpreloader/src/ClassPreloader/ClassLoader.php';

use ClassPreloader\ClassLoader;

$config = ClassLoader::getIncludes(function(ClassLoader $loader) {
    require __DIR__ . '/vendor/autoload.php';
    $loader->register();

    require_once __DIR__.'/app/AppKernel.php';

    $kernel = new AppKernel('dev', true);
    $kernel->boot();

    // This requires to run "composer dump-autoload --optimize"
    $autoloadClassmap = require __DIR__ . '/vendor/composer/autoload_classmap.php';
    foreach ($autoloadClassmap as $class => $file) {
        if (preg_match('#' . __DIR__ . '/src/#', $file)) {
            class_exists($class); // Trigger class autoloading
        }
    }
});

$config->addExclusiveFilter('/Bundle\.php$/');

return $config;

app_dev_preloader.php

<?php

use Symfony\Component\HttpFoundation\Request;

// If you don't want to setup permissions the proper way, just uncomment the following PHP line
// read http://symfony.com/doc/current/book/installation.html#configuration-and-setup for more information
//umask(0000);

// This check prevents access to debug front controllers that are deployed by accident to production servers.
// Feel free to remove this, extend it, or make something more sophisticated.
if (isset($_SERVER['HTTP_CLIENT_IP'])
    || isset($_SERVER['HTTP_X_FORWARDED_FOR'])
    || !in_array(@$_SERVER['REMOTE_ADDR'], array('127.0.0.1', 'fe80::1', '::1'))
) {
    //header('HTTP/1.0 403 Forbidden');
    //exit('You are not allowed to access this file. Check '.basename(__FILE__).' for more information.');
}

//$loader = require_once __DIR__.'/../app/bootstrap.php.cache';
$loader = require_once __DIR__.'/../vendor/autoload.php';
require_once __DIR__.'/../preloader.php';
require_once __DIR__.'/../app/AppKernel.php';

$kernel = new AppKernel('dev', true);
//$kernel->loadClassCache();
Request::enableHttpMethodParameterOverride();
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();
$kernel->terminate($request, $response);

app_preloader.php

<?php

use Symfony\Component\ClassLoader\ApcClassLoader;
use Symfony\Component\HttpFoundation\Request;

//$loader = require_once __DIR__.'/../app/bootstrap.php.cache';
$loader = require_once __DIR__.'/../vendor/autoload.php';
require_once __DIR__.'/../preloader.php';

// Use APC for autoloading to improve performance.
// Change 'sf2' to a unique prefix in order to prevent cache key conflicts
// with other applications also using APC.
/*
$loader = new ApcClassLoader('sf2', $loader);
$loader->register(true);
*/

require_once __DIR__.'/../app/AppKernel.php';
//require_once __DIR__.'/../app/AppCache.php';

$kernel = new AppKernel('prod', false);
//$kernel->loadClassCache();
//$kernel = new AppCache($kernel);
Request::enableHttpMethodParameterOverride();
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();
$kernel->terminate($request, $response);

test.sh

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ php bin/classpreloader.php compile --config=preloaderConfig.php --output=preloader.php

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ php preloader.php

# Fix the file, as there is a bug with namespace block and some twig classes

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ sudo service apache2 restart
 * Restarting web server apache2                                                                                                                                                                      ... waiting                                                                                                                                                                                  [ OK ]

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ for file in app.php app_preloader.php app_dev.php app_dev_preloader.php; do curl --silent http://localhost/framework-standard-edition/web/$file > /dev/null ; done

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ for file in app.php app_preloader.php app_dev.php app_dev_preloader.php; do ab -n 500 -c 10 http://localhost/framework-standard-edition/web/$file ; done
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /framework-standard-edition/web/app.php
Document Length:        553 bytes

Concurrency Level:      10
Time taken for tests:   25.114 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Non-2xx responses:      500
Total transferred:      419000 bytes
HTML transferred:       276500 bytes
Requests per second:    19.91 [#/sec] (mean)
Time per request:       502.283 [ms] (mean)
Time per request:       50.228 [ms] (mean, across all concurrent requests)
Transfer rate:          16.29 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.0      0       7
Processing:    46  500 444.4    262    2263
Waiting:       30  433 427.7    205    2161
Total:         47  500 444.4    263    2264

Percentage of the requests served within a certain time (ms)
  50%    263
  66%    434
  75%    830
  80%    897
  90%   1210
  95%   1380
  98%   1714
  99%   1854
 100%   2264 (longest request)
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /framework-standard-edition/web/app_preloader.php
Document Length:        553 bytes

Concurrency Level:      10
Time taken for tests:   28.460 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Non-2xx responses:      500
Total transferred:      419000 bytes
HTML transferred:       276500 bytes
Requests per second:    17.57 [#/sec] (mean)
Time per request:       569.195 [ms] (mean)
Time per request:       56.920 [ms] (mean, across all concurrent requests)
Transfer rate:          14.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.6      0       5
Processing:    54  567 511.1    284    2655
Waiting:       28  478 483.0    222    2565
Total:         54  567 511.1    284    2655

Percentage of the requests served within a certain time (ms)
  50%    284
  66%    764
  75%    877
  80%    927
  90%   1226
  95%   1781
  98%   1987
  99%   2212
 100%   2655 (longest request)
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /framework-standard-edition/web/app_dev.php
Document Length:        12799 bytes

Concurrency Level:      10
Time taken for tests:   54.172 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Total transferred:      6582500 bytes
HTML transferred:       6399500 bytes
Requests per second:    9.23 [#/sec] (mean)
Time per request:       1083.446 [ms] (mean)
Time per request:       108.345 [ms] (mean, across all concurrent requests)
Transfer rate:          118.66 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       6
Processing:   249 1080 661.5   1034    3814
Waiting:      220  905 588.6    889    3796
Total:        249 1080 661.5   1034    3814

Percentage of the requests served within a certain time (ms)
  50%   1034
  66%   1238
  75%   1423
  80%   1612                                                                                                                                                                          
  90%   2031
  95%   2362
  98%   2786                                                                                                                                                                          
  99%   3134
 100%   3814 (longest request)
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /framework-standard-edition/web/app_dev_preloader.php
Document Length:        12839 bytes

Concurrency Level:      10
Time taken for tests:   59.365 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Total transferred:      6602500 bytes
HTML transferred:       6419500 bytes
Requests per second:    8.42 [#/sec] (mean)
Time per request:       1187.309 [ms] (mean)
Time per request:       118.731 [ms] (mean, across all concurrent requests)
Transfer rate:          108.61 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       6
Processing:   301 1185 666.7   1111    4551
Waiting:      237 1036 634.6   1017    4419
Total:        301 1185 666.7   1111    4551

Percentage of the requests served within a certain time (ms)
  50%   1111
  66%   1394
  75%   1654
  80%   1759
  90%   2068
  95%   2360
  98%   2882
  99%   3132
 100%   4551 (longest request)

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ sudo vi /etc/php5/apache2/conf.d/20-apc.ini # Set apc.stat=0

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ sudo service apache2 restart
 * Restarting web server apache2                                                                                                                                                                      ... waiting                                                                                                                                                                                  [ OK ]

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ for file in app.php app_preloader.php; do curl --silent http://localhost/framework-standard-edition/web/$file > /dev/null ; done

ubuntu at ip-10-83-137-206 in /var/www/framework-standard-edition 
$ for file in app.php app_preloader.php; do ab -n 500 -c 10 http://localhost/framework-standard-edition/web/$file ; done
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /framework-standard-edition/web/app.php
Document Length:        553 bytes

Concurrency Level:      10
Time taken for tests:   24.322 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Non-2xx responses:      500
Total transferred:      419000 bytes
HTML transferred:       276500 bytes
Requests per second:    20.56 [#/sec] (mean)
Time per request:       486.439 [ms] (mean)
Time per request:       48.644 [ms] (mean, across all concurrent requests)
Transfer rate:          16.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1  10.8      0     240
Processing:    44  485 454.2    253    2413
Waiting:       34  387 401.4    184    2059
Total:         44  485 454.2    254    2413

Percentage of the requests served within a certain time (ms)
  50%    254
  66%    429
  75%    788
  80%    880
  90%   1141
  95%   1464
  98%   1852
  99%   2023
 100%   2413 (longest request)
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:        Apache/2.2.22
Server Hostname:        localhost
Server Port:            80

Document Path:          /framework-standard-edition/web/app_preloader.php
Document Length:        553 bytes

Concurrency Level:      10
Time taken for tests:   25.167 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Non-2xx responses:      500
Total transferred:      419000 bytes
HTML transferred:       276500 bytes
Requests per second:    19.87 [#/sec] (mean)
Time per request:       503.335 [ms] (mean)
Time per request:       50.333 [ms] (mean, across all concurrent requests)
Transfer rate:          16.26 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1  10.8      0     240
Processing:    56  501 483.2    251    2492
Waiting:       35  415 443.0    173    2381
Total:         56  501 483.3    251    2492

Percentage of the requests served within a certain time (ms)
  50%    251
  66%    450
  75%    807
  80%    876
  90%   1207
  95%   1501
  98%   1930
  99%   2171
 100%   2492 (longest request)

It looks like you're adding every file to your preloader that is present in the Composer classmap. Is that the case? I doubt that you are using each and every class present in the classmap in your test code. The point of the preloader is to preload classes that you know will be autoloaded for a specific use case. Preloading classes that will not be used will likely slow down you application.