When I first wrote Benchmarking JSON Generation in Ruby, the benchmarking setup had an issue which is now fixed.
After the issue got fixed, the blog post got updated with the new results. Below, you can see the original results and analysis:
As you can see below, AMS and presenters have similar performance. It's not worth using presenters over AMS in our API for the tiny performance gain. So let's stop comparing them from now on.
On the other hand, RABL is significantly slower than AMS. It is about 5X slower in the simplest case and about 10X slower in the complex cases. This difference remains the same when we deal with a collection of objects.
The times you see is the total for 10,000 iterations. Refer to "The Setup" section
above to understand the different cases: Ultra Simple, Simple, and Complex.
user system total real
RABL Ultra Simple 1.610000 0.420000 2.030000 ( 2.037046)
AMS Ultra Simple 0.310000 0.000000 0.310000 ( 0.314775)
Presenters Ultra Simple 0.230000 0.000000 0.230000 ( 0.229938)
--------------------------------------------------------------------------------------
RABL Simple 8.690000 2.150000 10.840000 ( 10.834650)
AMS Simple 1.250000 0.000000 1.250000 ( 1.252625)
Presenters Simple 0.840000 0.000000 0.840000 ( 0.837271)
--------------------------------------------------------------------------------------
RABL Complex 18.560000 4.330000 22.890000 ( 22.888495)
AMS Complex 2.410000 0.000000 2.410000 ( 2.416374)
Presenters Complex 1.700000 0.000000 1.700000 ( 1.697204)
Each collection contains 100 objects.
user system total real
RABL Ultra Simple: Collection 1.380000 0.430000 1.810000 ( 1.810402)
AMS Ultra Simple: Collection 0.210000 0.000000 0.210000 ( 0.212134)
Presenters Ultra Simple: Collection 0.200000 0.000000 0.200000 ( 0.195934)
--------------------------------------------------------------------------------------
RABL Simple: Collection 8.260000 2.110000 10.370000 ( 10.368818)
AMS Simple: Collection 1.090000 0.000000 1.090000 ( 1.095966)
Presenters Simple: Collection 0.820000 0.000000 0.820000 ( 0.821661)
--------------------------------------------------------------------------------------
RABL Complex: Collection 18.180000 4.280000 22.460000 ( 22.455245)
AMS Complex: Collection 2.350000 0.010000 2.360000 ( 2.360007)
Presenters Complex: Collection 1.840000 0.010000 1.850000 ( 1.850659)
As per the results of the first benchmark run, RABL is much slower compared to AMS. There is no good reason why it has to be. So we did a bit of investigation, and discovered that RABL spends a lot of time on template lookup. We can configure RABL to cache this lookup (config.cache_sources = true). Below, you can see the results of the benchmark with template lookup caching enabled for RABL.
Although performance doubles for RABL, it is still much slower. It is now about 3X slower in the simplest case, and about 5X slower in the complex cases compared to AMS.
user system total real
RABL Ultra Simple 1.030000 0.000000 1.030000 ( 1.023175)
AMS Ultra Simple 0.310000 0.000000 0.310000 ( 0.313161)
Presenters Ultra Simple 0.230000 0.000000 0.230000 ( 0.231076)
--------------------------------------------------------------------------------------
RABL Simple 5.670000 0.000000 5.670000 ( 5.678033)
AMS Simple 1.260000 0.010000 1.270000 ( 1.253238)
Presenters Simple 0.820000 0.000000 0.820000 ( 0.826598)
--------------------------------------------------------------------------------------
RABL Complex 12.370000 0.010000 12.380000 ( 12.373665)
AMS Complex 2.460000 0.000000 2.460000 ( 2.460624)
Presenters Complex 1.690000 0.000000 1.690000 ( 1.693395)
user system total real
RABL Ultra Simple: Collection 0.820000 0.000000 0.820000 ( 0.825097)
AMS Ultra Simple: Collection 0.220000 0.000000 0.220000 ( 0.213909)
Presenters Ultra Simple: Collection 0.180000 0.000000 0.180000 ( 0.185690)
--------------------------------------------------------------------------------------
RABL Simple: Collection 5.510000 0.000000 5.510000 ( 5.510466)
AMS Simple: Collection 1.110000 0.010000 1.120000 ( 1.116311)
Presenters Simple: Collection 0.840000 0.000000 0.840000 ( 0.844759)
--------------------------------------------------------------------------------------
RABL Complex: Collection 12.710000 0.040000 12.750000 ( 12.750722)
AMS Complex: Collection 2.340000 0.000000 2.340000 ( 2.348584)
Presenters Complex: Collection 1.900000 0.010000 1.910000 ( 1.903831)