prom-client benchmark 1

gistfile1.txt

❯ node index.js

# Benchmark Regression

## Installing baseline packages

http fetch GET 200 https://registry.npmjs.org/prom-client 1000ms
http fetch GET 200 https://registry.npmjs.org/prom-client/-/prom-client-14.0.1.tgz 49ms
+ prom-client@14.0.1
added 3 packages from 3 contributors and audited 3 packages in 0.867s
found 0 vulnerabilities

## Starting benchmark suite

### Progress:

- registry ➭ getMetricsAsJSON#1 with 64 ➭ current x 2,449 ops/sec ±8.26% (72 runs sampled)
- registry ➭ getMetricsAsJSON#1 with 64 ➭ prom-client@latest x 2,547 ops/sec ±7.85% (81 runs sampled)
- registry ➭ getMetricsAsJSON#2 with 8 ➭ current x 2,079 ops/sec ±7.27% (73 runs sampled)
- registry ➭ getMetricsAsJSON#2 with 8 ➭ prom-client@latest x 2,213 ops/sec ±6.06% (77 runs sampled)
- registry ➭ getMetricsAsJSON#2 with 4 and 2 with 2 ➭ current x 1,718 ops/sec ±4.82% (78 runs sampled)
- registry ➭ getMetricsAsJSON#2 with 4 and 2 with 2 ➭ prom-client@latest x 1,828 ops/sec ±4.60% (82 runs sampled)
- registry ➭ getMetricsAsJSON#2 with 2 and 2 with 4 ➭ current x 1,707 ops/sec ±5.72% (76 runs sampled)
- registry ➭ getMetricsAsJSON#2 with 2 and 2 with 4 ➭ prom-client@latest x 1,776 ops/sec ±5.01% (75 runs sampled)
- registry ➭ getMetricsAsJSON#6 with 2 ➭ current x 1,472 ops/sec ±6.20% (75 runs sampled)
- registry ➭ getMetricsAsJSON#6 with 2 ➭ prom-client@latest x 1,593 ops/sec ±5.76% (78 runs sampled)
- registry ➭ metrics#1 with 64 ➭ current x 1,868 ops/sec ±12.99% (62 runs sampled)
- registry ➭ metrics#1 with 64 ➭ prom-client@latest x 2,190 ops/sec ±6.64% (75 runs sampled)
- registry ➭ metrics#2 with 8 ➭ current x 2,127 ops/sec ±5.01% (72 runs sampled)
- registry ➭ metrics#2 with 8 ➭ prom-client@latest x 2,032 ops/sec ±6.52% (72 runs sampled)
- registry ➭ metrics#2 with 4 and 2 with 2 ➭ current x 1,720 ops/sec ±5.85% (77 runs sampled)
- registry ➭ metrics#2 with 4 and 2 with 2 ➭ prom-client@latest x 1,759 ops/sec ±5.58% (80 runs sampled)
- registry ➭ metrics#2 with 2 and 2 with 4 ➭ current x 1,741 ops/sec ±5.04% (69 runs sampled)
- registry ➭ metrics#2 with 2 and 2 with 4 ➭ prom-client@latest x 1,755 ops/sec ±5.09% (77 runs sampled)
- registry ➭ metrics#6 with 2 ➭ current x 1,507 ops/sec ±5.43% (77 runs sampled)
- registry ➭ metrics#6 with 2 ➭ prom-client@latest x 1,584 ops/sec ±4.77% (75 runs sampled)
- histogram ➭ observe#1 with 64 ➭ current x 50,580 ops/sec ±1.18% (91 runs sampled)
- histogram ➭ observe#1 with 64 ➭ prom-client@latest x 47,737 ops/sec ±1.10% (87 runs sampled)
- histogram ➭ observe#2 with 8 ➭ current x 18,870 ops/sec ±0.94% (91 runs sampled)
- histogram ➭ observe#2 with 8 ➭ prom-client@latest x 18,718 ops/sec ±1.13% (89 runs sampled)
- histogram ➭ observe#2 with 4 and 2 with 2 ➭ current x 11,740 ops/sec ±0.98% (90 runs sampled)
- histogram ➭ observe#2 with 4 and 2 with 2 ➭ prom-client@latest x 11,633 ops/sec ±0.98% (87 runs sampled)
- histogram ➭ observe#2 with 2 and 2 with 4 ➭ current x 11,736 ops/sec ±0.94% (92 runs sampled)
- histogram ➭ observe#2 with 2 and 2 with 4 ➭ prom-client@latest x 11,748 ops/sec ±0.88% (88 runs sampled)
- histogram ➭ observe#6 with 2 ➭ current x 8,840 ops/sec ±0.71% (90 runs sampled)
- histogram ➭ observe#6 with 2 ➭ prom-client@latest x 8,661 ops/sec ±0.76% (86 runs sampled)
- gauge ➭ inc ➭ current x 9,352,417 ops/sec ±0.92% (92 runs sampled)
- gauge ➭ inc ➭ prom-client@latest x 9,181,537 ops/sec ±0.92% (92 runs sampled)
- gauge ➭ inc with labels ➭ current x 12,699 ops/sec ±0.68% (89 runs sampled)
- gauge ➭ inc with labels ➭ prom-client@latest x 12,803 ops/sec ±0.96% (89 runs sampled)
- summary ➭ observe#1 with 64 ➭ current x 44,326 ops/sec ±1.60% (94 runs sampled)
- summary ➭ observe#1 with 64 ➭ prom-client@latest x 41,172 ops/sec ±1.50% (84 runs sampled)
- summary ➭ observe#2 with 8 ➭ current x 17,825 ops/sec ±1.16% (88 runs sampled)
- summary ➭ observe#2 with 8 ➭ prom-client@latest x 18,253 ops/sec ±1.02% (91 runs sampled)
- summary ➭ observe#2 with 4 and 2 with 2 ➭ current x 11,421 ops/sec ±1.42% (85 runs sampled)
- summary ➭ observe#2 with 4 and 2 with 2 ➭ prom-client@latest x 11,451 ops/sec ±1.10% (90 runs sampled)
- summary ➭ observe#2 with 2 and 2 with 4 ➭ current x 11,907 ops/sec ±0.84% (88 runs sampled)
- summary ➭ observe#2 with 2 and 2 with 4 ➭ prom-client@latest x 11,817 ops/sec ±0.81% (91 runs sampled)
- summary ➭ observe#6 with 2 ➭ current x 8,839 ops/sec ±0.71% (91 runs sampled)
- summary ➭ observe#6 with 2 ➭ prom-client@latest x 8,797 ops/sec ±0.58% (95 runs sampled)

### Results:

┌────────────────────────────────────────┬────────────────────┬────────────────────┐
│ registry                               │ current            │ prom-client@latest │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ getMetricsAsJSON#1 with 64             │ 2449.110480514575  │ 2547.019469802027  │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ getMetricsAsJSON#2 with 8              │ 2079.0783944050077 │ 2213.3199998143905 │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ getMetricsAsJSON#2 with 4 and 2 with 2 │ 1718.1030916994052 │ 1827.6862257348898 │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ getMetricsAsJSON#2 with 2 and 2 with 4 │ 1707.4358366211598 │ 1776.204881165233  │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ getMetricsAsJSON#6 with 2              │ 1471.6605273400849 │ 1592.8332936742247 │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ metrics#1 with 64                      │ 1867.5688841802805 │ 2189.6009802776084 │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ metrics#2 with 8                       │ 2127.3265690701537 │ 2031.5103552890705 │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ metrics#2 with 4 and 2 with 2          │ 1720.2876361390379 │ 1759.0314930246434 │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ metrics#2 with 2 and 2 with 4          │ 1740.900159001958  │ 1755.0939310027848 │
├────────────────────────────────────────┼────────────────────┼────────────────────┤
│ metrics#6 with 2                       │ 1506.6764464490896 │ 1584.256756799063  │
└────────────────────────────────────────┴────────────────────┴────────────────────┘

┌───────────────────────────────┬────────────────────┬────────────────────┐
│ histogram                     │ current            │ prom-client@latest │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#1 with 64             │ 50580.464596586986 │ 47736.87275244678  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 8              │ 18869.70706754995  │ 18717.612345622474 │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 4 and 2 with 2 │ 11740.00410604403  │ 11632.842237070132 │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 2 and 2 with 4 │ 11736.285616465968 │ 11747.86089991906  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#6 with 2              │ 8839.547803052405  │ 8660.771589151907  │
└───────────────────────────────┴────────────────────┴────────────────────┘

┌─────────────────┬────────────────────┬────────────────────┐
│ gauge           │ current            │ prom-client@latest │
├─────────────────┼────────────────────┼────────────────────┤
│ inc             │ 9352416.791181816  │ 9181537.497898385  │
├─────────────────┼────────────────────┼────────────────────┤
│ inc with labels │ 12698.835676757715 │ 12803.359030057683 │
└─────────────────┴────────────────────┴────────────────────┘

┌───────────────────────────────┬────────────────────┬────────────────────┐
│ summary                       │ current            │ prom-client@latest │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#1 with 64             │ 44326.13979647346  │ 41172.31589113536  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 8              │ 17824.65870572174  │ 18253.05388037391  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 4 and 2 with 2 │ 11421.045939307694 │ 11450.50802767432  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 2 and 2 with 4 │ 11907.291901620381 │ 11816.510273434498 │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#6 with 2              │ 8838.777837406307  │ 8797.061860250538  │
└───────────────────────────────┴────────────────────┴────────────────────┘

### Summary:

⚠ registry ➭ getMetricsAsJSON#1 with 64 is 3.998% acceptably slower.
⚠ registry ➭ getMetricsAsJSON#2 with 8 is 6.457% acceptably slower.
⚠ registry ➭ getMetricsAsJSON#2 with 4 and 2 with 2 is 6.378% acceptably slower.
⚠ registry ➭ getMetricsAsJSON#2 with 2 and 2 with 4 is 4.028% acceptably slower.
⚠ registry ➭ getMetricsAsJSON#6 with 2 is 8.234% acceptably slower.
✗ registry ➭ metrics#1 with 64 is 17.24% slower.
✓ registry ➭ metrics#2 with 8 is 4.717% faster.
⚠ registry ➭ metrics#2 with 4 and 2 with 2 is 2.252% acceptably slower.
⚠ registry ➭ metrics#2 with 2 and 2 with 4 is 0.8153% acceptably slower.
⚠ registry ➭ metrics#6 with 2 is 5.149% acceptably slower.
✓ histogram ➭ observe#1 with 64 is 5.957% faster.
✓ histogram ➭ observe#2 with 8 is 0.8126% faster.
✓ histogram ➭ observe#2 with 4 and 2 with 2 is 0.9212% faster.
⚠ histogram ➭ observe#2 with 2 and 2 with 4 is 0.09863% acceptably slower.
✓ histogram ➭ observe#6 with 2 is 2.064% faster.
✓ gauge ➭ inc is 1.861% faster.
⚠ gauge ➭ inc with labels is 0.8231% acceptably slower.
✓ summary ➭ observe#1 with 64 is 7.660% faster.
⚠ summary ➭ observe#2 with 8 is 2.403% acceptably slower.
⚠ summary ➭ observe#2 with 4 and 2 with 2 is 0.2580% acceptably slower.
✓ summary ➭ observe#2 with 2 and 2 with 4 is 0.7683% faster.
✓ summary ➭ observe#6 with 2 is 0.4742% faster.


⚠ Benchmarks failed to perform better than the currently published version.
- Please determine if the performance changes are expected and acceptable.