Skip to content

Instantly share code, notes, and snippets.

@Lysxia

Lysxia/script.sh

Last active September 28, 2017 13:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Lysxia/e388c13db32c92866918a0e55b46df19 to your computer and use it in GitHub Desktop.
Save Lysxia/e388c13db32c92866918a0e55b46df19 to your computer and use it in GitHub Desktop.
Download ZIP
aeson 2169adca7d (master) vs c857467ecd (#588)
Raw
gistfile1.txt
benchmarking D/toJSON/th benchmarking D/toJSON/th
time 7.576 μs (7.486 μs .. 7.710 μs) time 7.140 μs (7.127 μs .. 7.154 μs)
0.998 R² (0.997 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 7.508 μs (7.479 μs .. 7.571 μs) mean 7.130 μs (7.121 μs .. 7.143 μs)
std dev 186.6 ns (112.6 ns .. 296.5 ns) std dev 46.34 ns (35.27 ns .. 72.45 ns)
benchmarking D/toJSON/generic benchmarking D/toJSON/generic
time 8.946 μs (8.928 μs .. 8.967 μs) time 8.693 μs (8.680 μs .. 8.705 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 8.945 μs (8.927 μs .. 8.969 μs) mean 8.701 μs (8.688 μs .. 8.719 μs)
std dev 94.40 ns (72.42 ns .. 145.6 ns) std dev 68.51 ns (53.92 ns .. 92.66 ns)
benchmarking D/encode/th benchmarking D/encode/th
time 8.692 μs (8.651 μs .. 8.757 μs) time 8.080 μs (8.050 μs .. 8.125 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 8.653 μs (8.639 μs .. 8.674 μs) mean 8.060 μs (8.048 μs .. 8.076 μs)
std dev 75.62 ns (53.59 ns .. 113.4 ns) std dev 61.66 ns (50.08 ns .. 77.56 ns)
benchmarking D/encode/generic benchmarking D/encode/generic
time 18.38 μs (18.34 μs .. 18.43 μs) time 18.34 μs (18.19 μs .. 18.59 μs)
1.000 R² (1.000 R² .. 1.000 R²) 0.998 R² (0.997 R² .. 0.999 R²)
mean 18.48 μs (18.40 μs .. 18.70 μs) mean 18.56 μs (18.36 μs .. 18.98 μs)
std dev 578.7 ns (211.9 ns .. 1.128 μs) std dev 1.247 μs (769.4 ns .. 1.980 μs)
benchmarking D/fromJSON/th benchmarking D/fromJSON/th
time 3.093 μs (3.087 μs .. 3.099 μs) time 3.116 μs (3.112 μs .. 3.120 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 3.094 μs (3.088 μs .. 3.105 μs) mean 3.111 μs (3.106 μs .. 3.116 μs)
std dev 34.19 ns (21.32 ns .. 53.52 ns) std dev 24.53 ns (20.41 ns .. 30.13 ns)
benchmarking D/fromJSON/generic benchmarking D/fromJSON/generic
time 955.8 ns (949.6 ns .. 961.2 ns) time 948.5 ns (940.9 ns .. 963.9 ns)
1.000 R² (1.000 R² .. 1.000 R²) 0.999 R² (0.998 R² .. 1.000 R²)
mean 949.2 ns (947.7 ns .. 951.1 ns) mean 943.2 ns (941.0 ns .. 949.2 ns)
std dev 7.280 ns (5.761 ns .. 9.265 ns) std dev 14.98 ns (8.379 ns .. 27.79 ns)
benchmarking BigRecord/toJSON/th benchmarking BigRecord/toJSON/th
time 2.988 μs (2.984 μs .. 2.993 μs) time 3.092 μs (3.085 μs .. 3.104 μs)
1.000 R² (1.000 R² .. 1.000 R²) 0.999 R² (0.998 R² .. 1.000 R²)
mean 2.985 μs (2.981 μs .. 2.992 μs) mean 3.205 μs (3.140 μs .. 3.327 μs)
std dev 22.56 ns (16.20 ns .. 35.11 ns) std dev 384.6 ns (244.7 ns .. 577.8 ns)
benchmarking BigRecord/toJSON/generic benchmarking BigRecord/toJSON/generic
time 3.601 μs (3.597 μs .. 3.604 μs) time 3.662 μs (3.657 μs .. 3.666 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 3.602 μs (3.595 μs .. 3.611 μs) mean 3.663 μs (3.658 μs .. 3.671 μs)
std dev 34.26 ns (28.02 ns .. 41.76 ns) std dev 30.19 ns (24.11 ns .. 37.92 ns)
benchmarking BigRecord/encode/th benchmarking BigRecord/encode/th
time 4.494 μs (4.448 μs .. 4.557 μs) time 3.826 μs (3.807 μs .. 3.858 μs)
0.999 R² (0.998 R² .. 1.000 R²) 1.000 R² (0.999 R² .. 1.000 R²)
mean 4.527 μs (4.475 μs .. 4.654 μs) mean 3.811 μs (3.804 μs .. 3.821 μs)
std dev 357.2 ns (130.4 ns .. 640.1 ns) std dev 40.73 ns (30.31 ns .. 65.89 ns)
benchmarking BigRecord/encode/generic benchmarking BigRecord/encode/generic
time 3.117 μs (3.114 μs .. 3.121 μs) time 3.092 μs (3.080 μs .. 3.113 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 3.123 μs (3.118 μs .. 3.130 μs) mean 3.091 μs (3.082 μs .. 3.103 μs)
std dev 26.93 ns (22.36 ns .. 33.92 ns) std dev 44.03 ns (31.40 ns .. 58.78 ns)
benchmarking BigRecord/fromJSON/th benchmarking BigRecord/fromJSON/th
time 7.648 μs (7.531 μs .. 7.763 μs) time 6.539 μs (6.530 μs .. 6.550 μs)
0.999 R² (0.998 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 7.543 μs (7.518 μs .. 7.577 μs) mean 6.536 μs (6.526 μs .. 6.554 μs)
std dev 126.6 ns (91.20 ns .. 182.8 ns) std dev 58.20 ns (39.36 ns .. 98.75 ns)
benchmarking BigRecord/fromJSON/generic benchmarking BigRecord/fromJSON/generic
time 8.179 μs (8.158 μs .. 8.211 μs) time 8.875 μs (8.689 μs .. 9.080 μs)
0.999 R² (0.998 R² .. 1.000 R²) 0.997 R² (0.996 R² .. 0.999 R²)
mean 8.377 μs (8.255 μs .. 8.666 μs) mean 8.735 μs (8.663 μs .. 8.852 μs)
std dev 795.1 ns (458.7 ns .. 1.351 μs) std dev 422.1 ns (253.6 ns .. 653.2 ns)
benchmarking BigProduct/toJSON/th benchmarking BigProduct/toJSON/th
time 530.9 ns (530.3 ns .. 531.7 ns) time 477.9 ns (477.3 ns .. 478.7 ns)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 532.6 ns (531.6 ns .. 534.0 ns) mean 477.9 ns (477.5 ns .. 478.5 ns)
std dev 5.257 ns (4.171 ns .. 6.691 ns) std dev 2.026 ns (1.207 ns .. 3.409 ns)
benchmarking BigProduct/toJSON/generic benchmarking BigProduct/toJSON/generic
time 763.7 ns (752.8 ns .. 773.2 ns) time 681.6 ns (678.1 ns .. 690.0 ns)
0.999 R² (0.998 R² .. 1.000 R²) 0.999 R² (0.998 R² .. 1.000 R²)
mean 752.7 ns (750.1 ns .. 757.8 ns) mean 679.9 ns (677.9 ns .. 684.7 ns)
std dev 16.22 ns (10.86 ns .. 26.41 ns) std dev 12.54 ns (790.3 ps .. 23.14 ns)
benchmarking BigProduct/encode/th benchmarking BigProduct/encode/th
time 517.4 ns (516.7 ns .. 518.0 ns) time 503.0 ns (502.8 ns .. 503.1 ns)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 516.7 ns (515.9 ns .. 517.8 ns) mean 503.1 ns (503.0 ns .. 503.3 ns)
std dev 4.303 ns (3.560 ns .. 5.372 ns) std dev 632.2 ps (515.8 ps .. 908.6 ps)
benchmarking BigProduct/encode/generic benchmarking BigProduct/encode/generic
time 3.002 μs (2.997 μs .. 3.008 μs) time 2.982 μs (2.981 μs .. 2.985 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 2.999 μs (2.994 μs .. 3.006 μs) mean 2.980 μs (2.979 μs .. 2.981 μs)
std dev 26.72 ns (20.25 ns .. 37.35 ns) std dev 4.184 ns (3.002 ns .. 6.066 ns)
benchmarking BigProduct/fromJSON/th benchmarking BigProduct/fromJSON/th
time 4.470 μs (4.400 μs .. 4.566 μs) time 4.367 μs (4.325 μs .. 4.474 μs)
0.998 R² (0.998 R² .. 1.000 R²) 0.997 R² (0.992 R² .. 1.000 R²)
mean 4.416 μs (4.396 μs .. 4.453 μs) mean 4.338 μs (4.324 μs .. 4.407 μs)
std dev 117.1 ns (76.71 ns .. 169.4 ns) std dev 123.7 ns (6.140 ns .. 292.8 ns)
benchmarking BigProduct/fromJSON/generic benchmarking BigProduct/fromJSON/generic
time 5.409 μs (5.402 μs .. 5.418 μs) time 5.569 μs (5.568 μs .. 5.570 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 5.413 μs (5.407 μs .. 5.422 μs) mean 5.568 μs (5.567 μs .. 5.570 μs)
std dev 34.61 ns (30.03 ns .. 42.27 ns) std dev 7.768 ns (5.785 ns .. 10.79 ns)
benchmarking BigSum/toJSON/th benchmarking BigSum/toJSON/th
time 10.37 ns (10.36 ns .. 10.39 ns) time 10.59 ns (10.58 ns .. 10.61 ns)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 10.39 ns (10.37 ns .. 10.41 ns) mean 10.61 ns (10.58 ns .. 10.70 ns)
std dev 85.06 ps (71.14 ps .. 114.0 ps) std dev 199.3 ps (34.26 ps .. 427.1 ps)
benchmarking BigSum/toJSON/generic benchmarking BigSum/toJSON/generic
time 49.81 ns (49.62 ns .. 50.15 ns) time 53.82 ns (53.50 ns .. 54.22 ns)
1.000 R² (0.999 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 49.87 ns (49.74 ns .. 50.13 ns) mean 53.51 ns (53.44 ns .. 53.63 ns)
std dev 795.6 ps (496.7 ps .. 1.365 ns) std dev 410.9 ps (287.3 ps .. 588.7 ps)
benchmarking BigSum/encode/th benchmarking BigSum/encode/th
time 201.7 ns (201.4 ns .. 201.9 ns) time 200.2 ns (199.4 ns .. 201.0 ns)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 202.1 ns (201.8 ns .. 202.7 ns) mean 200.3 ns (200.1 ns .. 200.9 ns)
std dev 1.949 ns (1.462 ns .. 2.703 ns) std dev 1.542 ns (940.4 ps .. 2.892 ns)
benchmarking BigSum/encode/generic benchmarking BigSum/encode/generic
time 231.9 ns (231.3 ns .. 232.7 ns) time 202.9 ns (202.4 ns .. 203.5 ns)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 232.2 ns (231.8 ns .. 232.8 ns) mean 202.6 ns (202.5 ns .. 202.9 ns)
std dev 2.360 ns (1.992 ns .. 3.019 ns) std dev 924.8 ps (486.3 ps .. 1.531 ns)
benchmarking BigSum/fromJSON/th benchmarking BigSum/fromJSON/th
time 335.6 ns (333.3 ns .. 338.5 ns) time 345.6 ns (345.0 ns .. 346.2 ns)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 334.0 ns (333.3 ns .. 334.9 ns) mean 345.2 ns (344.9 ns .. 345.7 ns)
std dev 3.520 ns (2.606 ns .. 4.597 ns) std dev 1.644 ns (869.9 ps .. 2.857 ns)
benchmarking BigSum/fromJSON/generic benchmarking BigSum/fromJSON/generic
time 1.652 μs (1.649 μs .. 1.655 μs) time 1.740 μs (1.738 μs .. 1.742 μs)
1.000 R² (1.000 R² .. 1.000 R²) 1.000 R² (1.000 R² .. 1.000 R²)
mean 1.651 μs (1.648 μs .. 1.654 μs) mean 1.739 μs (1.738 μs .. 1.745 μs)
std dev 13.26 ns (10.75 ns .. 17.58 ns) std dev 12.37 ns (2.274 ns .. 26.25 ns)
Raw
script.sh
# script to run in a git branch of aeson
set -e
localdir=local/gen/`git rev-parse --abbrev-ref HEAD`+`stack ghc -- --numeric-version`
logfile=$localdir/report
bb=$localdir/bb+`date %j`
builddir=$localdir/build
mkdir -p $localdir
mkdir -p $builddir
exec > $logfile 2>&1
set -x
date
# for fast in fast -fast ; do
for fast in -fast ; do
echo $fast
stack clean
echo "Build stack"
time stack build --silent --flag aeson:$fast
stack install criterion --silent --flag aeson:$fast --force-dirty
# for optim in -O0 -O2 ; do
for optim in -O2 ; do
echo $optim
rm $builddir -rf
time stack ghc -- \
benchmarks/AesonCompareAutoInstances.hs \
-v0 -i -ibenchmarks \
-hidir $builddir -odir $builddir \
$optim \
-o $bb \
-ddump-simpl \
-ddump-to-file \
-dsuppress-all \
-dumpdir $localdir \
-fforce-recomp
done
echo "Benchmark"
./$bb -L10
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment