Performance test CE 1.7.0.2 vs. CE 2.0.0.0-dev45

README.md

Colins tweet made me curious so I also did a dumb test with CE 1.7.0.2 and CE 2.0.0.0-dev45.

Method

I kept it simple:

1. Set up vanilla 1.7.0.2 install on a local VM.
2. Create sample data using LimeSoda_SampleDataGenerator.
3. Activate and flush caches.
4. Call the homepage, one category and product page 5 times each. Take the arithmetic mean value and standard deviation of the loading time in Firebug.
5. Set up vanilla 2.0.0.0-dev45 install on the same VM. Use the database from 1.7.0.2 for the installation.
6. Repeat steps 3 to 4.
7. Run compiler and classmap generator for 2.0.0.0-dev45.
8. Repeat steps 3 to 4.

Magento 2: Compiling and generating the classmap

I executed the following commands from the Magento root directory:

• php -f dev/tools/Di/compiler.php
• php -f dev/tools/classmap/fs_generator.php ./app > ./var/classmap.ser

Test data

• 2 Websites
• 2 Store Groups
• 7 Store Views
• 66 Categories (spread over 3 levels)
• 3 Attribute Sets
• 10 Additional Attributes
• 1,000 products
• 3 category assignments per product

Results

The values displayed are:

• Loading time (mean value in ms)
• Standard derivation

	CE 1.7.0.2	CE 2.0.0.0-dev45 w/o Compiler / Classmap	CE 2.0.0.0-dev45 w Compiler / Classmap
Homepage	326.60 / 109.28	919.80 / 109.28	393.60 / 15.40
Category page 1st level	584.40 / 85.31	1250.00 / 103.73	734.40 / 13.87
Product page	562.60 / 55.04	1330.00 / 126.81	878.20 / 117.80
Add to cart	262.40 / 20.97	612.60 / 14.98	468.60 / 26.30

Conclusion

• Numbers are only rule of thumb estimates. Take them with a pinch of slat.
• Magento 2 is in the middle of development. Wait for the final product to judge.
• Colin is right. The current Magento 2 version (with Caching, without compilation and classmap) is slower for me too.
• Executing compilation and generating the classmap brings Magento 2 closer to Magento 1.

Hi Matthias- Thanks for sharing your testing. We haven’t focused on making performance improvements to the M2 code base yet. Our expectation is that M2 will perform as well as, or better than the current 1.x platform when it is complete.