Skip to content

Instantly share code, notes, and snippets.

@ahx
Last active March 4, 2024 15:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ahx/e6ffced58bd2e8d5baffb2f4d2c1f823 to your computer and use it in GitHub Desktop.
Save ahx/e6ffced58bd2e8d5baffb2f4d2c1f823 to your computer and use it in GitHub Desktop.
openapi_first benchmark comparison

These are the results running these benchmarks (code).

openapi_first 1.3.2

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin22]
Warming up --------------------------------------
 ./apps/committee.ru     1.000 i/100ms
./apps/committee_with_response_validation.ru
                         1.000 i/100ms
./apps/no-validation.ru
                         2.000 i/100ms
./apps/openapi_first.ru
                         2.000 i/100ms
./apps/openapi_first_with_response_validation.ru
                         1.000 i/100ms
Calculating -------------------------------------
 ./apps/committee.ru     19.097 (± 0.0%) i/s -     96.000 in   5.027948s
./apps/committee_with_response_validation.ru
                         15.910 (± 0.0%) i/s -     80.000 in   5.030133s
./apps/no-validation.ru
                         27.223 (± 0.0%) i/s -    138.000 in   5.069630s
./apps/openapi_first.ru
                         24.172 (± 4.1%) i/s -    122.000 in   5.050524s
./apps/openapi_first_with_response_validation.ru
                         14.822 (± 0.0%) i/s -     75.000 in   5.064667s

Comparison:
./apps/no-validation.ru:       27.2 i/s
./apps/openapi_first.ru:       24.2 i/s - 1.13x  slower
 ./apps/committee.ru:       19.1 i/s - 1.43x  slower
./apps/committee_with_response_validation.ru:       15.9 i/s - 1.71x  slower
./apps/openapi_first_with_response_validation.ru:       14.8 i/s - 1.84x  slower

Calculating -------------------------------------
 ./apps/committee.ru    11.773M memsize (     8.201k retained)
                       158.671k objects (    56.000  retained)
                        50.000  strings (     6.000  retained)
./apps/committee_with_response_validation.ru
                        15.296M memsize (     8.201k retained)
                       215.736k objects (    56.000  retained)
                        50.000  strings (     6.000  retained)
./apps/no-validation.ru
                         6.652M memsize (     1.816k retained)
                        90.294k objects (     7.000  retained)
                        50.000  strings (     3.000  retained)
./apps/openapi_first.ru
                         7.823M memsize (   240.000  retained)
                       101.808k objects (     6.000  retained)
                        41.000  strings (     3.000  retained)
./apps/openapi_first_with_response_validation.ru
                        15.778M memsize (   240.000  retained)
                       182.507k objects (     6.000  retained)
                        47.000  strings (     3.000  retained)

Comparison:
./apps/no-validation.ru:    6652264 allocated
./apps/openapi_first.ru:    7823460 allocated - 1.18x more
 ./apps/committee.ru:   11773065 allocated - 1.77x more
./apps/committee_with_response_validation.ru:   15296443 allocated - 2.30x more
./apps/openapi_first_with_response_validation.ru:   15778220 allocated - 2.37x more

openapi_first 1.1.0

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin22]
Warming up --------------------------------------
 ./apps/committee.ru     1.000 i/100ms
./apps/committee_with_response_validation.ru
                         1.000 i/100ms
./apps/openapi_first.ru
                         1.000 i/100ms
./apps/openapi_first_with_response_validation.ru
                         1.000 i/100ms
Calculating -------------------------------------
 ./apps/committee.ru     10.064 (± 0.0%) i/s -     51.000 in   5.070207s
./apps/committee_with_response_validation.ru
                          8.642 (± 0.0%) i/s -     44.000 in   5.092091s
./apps/openapi_first.ru
                         12.869 (± 0.0%) i/s -     65.000 in   5.058367s
./apps/openapi_first_with_response_validation.ru
                          8.274 (± 0.0%) i/s -     42.000 in   5.086650s

Comparison:
./apps/openapi_first.ru:       12.9 i/s
 ./apps/committee.ru:       10.1 i/s - 1.28x  slower
./apps/committee_with_response_validation.ru:        8.6 i/s - 1.49x  slower
./apps/openapi_first_with_response_validation.ru:        8.3 i/s - 1.56x  slower

Calculating -------------------------------------
 ./apps/committee.ru    12.201M memsize (     8.201k retained)
                       162.408k objects (    56.000  retained)
                        50.000  strings (     6.000  retained)
./apps/committee_with_response_validation.ru
                        15.655M memsize (     8.201k retained)
                       217.756k objects (    56.000  retained)
                        50.000  strings (     6.000  retained)
./apps/openapi_first.ru
                         8.223M memsize (   200.000  retained)
                       105.343k objects (     5.000  retained)
                        41.000  strings (     3.000  retained)
./apps/openapi_first_with_response_validation.ru
                        15.959M memsize (   200.000  retained)
                       182.507k objects (     5.000  retained)
                        47.000  strings (     3.000  retained)

Comparison:
./apps/openapi_first.ru:    8222814 allocated
 ./apps/committee.ru:   12200699 allocated - 1.48x more
./apps/committee_with_response_validation.ru:   15655397 allocated - 1.90x more
./apps/openapi_first_with_response_validation.ru:   15959414 allocated - 1.94x more
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment