A benchmark for JuMP.jl

benchmark.md

Benchmark Report for JuMP

Job Properties

• Time of benchmarks:
  • Target: 8 May 2020 - 16:56
  • Baseline: 8 May 2020 - 16:59
• Package commits:
  • Target: c6609a
  • Baseline: 3b0304
• Julia commits:
  • Target: 381693
  • Baseline: 381693
• Julia command flags:
  • Target: None
  • Baseline: None
• Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID	time ratio	memory ratio
["bt1"]	2.32 (5%) ❌	0.50 (1%) ✅
["hs10"]	2.62 (5%) ❌	0.50 (1%) ✅
["hs100"]	6.70 (5%) ❌	0.09 (1%) ✅
["hs101"]	33.08 (5%) ❌	0.07 (1%) ✅
["hs102"]	24.78 (5%) ❌	0.07 (1%) ✅
["hs103"]	24.81 (5%) ❌	0.07 (1%) ✅
["hs104"]	8.67 (5%) ❌	0.14 (1%) ✅
["hs106"]	3.67 (5%) ❌	0.17 (1%) ✅
["hs107"]	6.41 (5%) ❌	0.24 (1%) ✅
["hs108"]	4.26 (5%) ❌	0.06 (1%) ✅
["hs109"]	8.23 (5%) ❌	0.15 (1%) ✅
["hs11"]	1.58 (5%) ❌	0.50 (1%) ✅
["hs111"]	19.12 (5%) ❌	2.96 (1%) ❌
["hs113"]	6.87 (5%) ❌	0.11 (1%) ✅
["hs114"]	4.58 (5%) ❌	0.21 (1%) ✅
["hs116"]	5.62 (5%) ❌	0.05 (1%) ✅
["hs117"]	9.02 (5%) ❌	0.03 (1%) ✅
["hs12"]	2.70 (5%) ❌	0.50 (1%) ✅
["hs13"]	1.85 (5%) ❌	0.50 (1%) ✅
["hs14"]	2.39 (5%) ❌	0.50 (1%) ✅
["hs15"]	2.35 (5%) ❌	0.47 (1%) ✅
["hs16"]	2.44 (5%) ❌	0.47 (1%) ✅
["hs17"]	2.54 (5%) ❌	0.53 (1%) ✅
["hs18"]	2.22 (5%) ❌	0.47 (1%) ✅
["hs19"]	3.14 (5%) ❌	0.47 (1%) ✅
["hs20"]	2.94 (5%) ❌	0.30 (1%) ✅
["hs22"]	1.79 (5%) ❌	0.56 (1%) ✅
["hs23"]	3.52 (5%) ❌	0.29 (1%) ✅
["hs26"]	2.57 (5%) ❌	0.47 (1%) ✅
["hs27"]	1.93 (5%) ❌	0.50 (1%) ✅
["hs29"]	3.13 (5%) ❌	0.53 (1%) ✅
["hs30"]	2.13 (5%) ❌	0.50 (1%) ✅
["hs31"]	1.98 (5%) ❌	0.50 (1%) ✅
["hs32"]	2.78 (5%) ❌	0.47 (1%) ✅
["hs33"]	3.06 (5%) ❌	0.33 (1%) ✅
["hs34"]	1.65 (5%) ❌	0.59 (1%) ✅
["hs39"]	2.35 (5%) ❌	0.30 (1%) ✅
["hs40"]	3.01 (5%) ❌	0.32 (1%) ✅
["hs42"]	2.32 (5%) ❌	0.50 (1%) ✅
["hs43"]	5.30 (5%) ❌	0.21 (1%) ✅
["hs46"]	3.97 (5%) ❌	0.33 (1%) ✅
["hs47"]	3.04 (5%) ❌	0.29 (1%) ✅
["hs56"]	4.32 (5%) ❌	0.28 (1%) ✅
["hs57"]	88.03 (5%) ❌	17.00 (1%) ❌
["hs59"]	6.01 (5%) ❌	0.33 (1%) ✅
["hs6"]	1.97 (5%) ❌	0.50 (1%) ✅
["hs60"]	3.51 (5%) ❌	0.53 (1%) ✅
["hs63"]	3.02 (5%) ❌	0.47 (1%) ✅
["hs64"]	2.44 (5%) ❌	0.53 (1%) ✅
["hs65"]	2.67 (5%) ❌	0.53 (1%) ✅
["hs66"]	1.74 (5%) ❌	0.59 (1%) ✅
["hs7"]	2.72 (5%) ❌	0.56 (1%) ✅
["hs70"]	336.51 (5%) ❌	7.88 (1%) ❌
["hs71"]	2.57 (5%) ❌	0.29 (1%) ✅
["hs72"]	2.42 (5%) ❌	0.29 (1%) ✅
["hs73"]	5.38 (5%) ❌	0.53 (1%) ✅
["hs74"]	5.58 (5%) ❌	0.57 (1%) ✅
["hs75"]	5.40 (5%) ❌	0.57 (1%) ✅
["hs77"]	4.89 (5%) ❌	0.41 (1%) ✅
["hs78"]	3.51 (5%) ❌	0.17 (1%) ✅
["hs79"]	4.36 (5%) ❌	0.36 (1%) ✅
["hs8"]	2.24 (5%) ❌	0.47 (1%) ✅
["hs80"]	3.56 (5%) ❌	0.19 (1%) ✅
["hs81"]	3.90 (5%) ❌	0.19 (1%) ✅
["hs83"]	3.75 (5%) ❌	0.17 (1%) ✅
["hs84"]	4.36 (5%) ❌	0.18 (1%) ✅
["hs93"]	3.94 (5%) ❌	0.17 (1%) ✅
["hs95"]	6.26 (5%) ❌	0.09 (1%) ✅
["hs96"]	6.24 (5%) ❌	0.09 (1%) ✅
["hs97"]	6.21 (5%) ❌	0.09 (1%) ✅
["hs98"]	6.27 (5%) ❌	0.09 (1%) ✅
["zangwil3"]	1.62 (5%) ❌	0.20 (1%) ✅

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

• []

Julia versioninfo

Target

Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.3.0-51-generic #44~18.04.2-Ubuntu SMP Thu Apr 23 14:27:18 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2718 MHz     199550 s        228 s      60678 s     924147 s          0 s
       #2  2702 MHz     182258 s        211 s      57316 s     943526 s          0 s
       #3  2711 MHz     198584 s        185 s      59911 s     917905 s          0 s
       #4  2700 MHz     248152 s        370 s      59360 s     878424 s          0 s
       
  Memory: 7.654296875 GB (1395.8046875 MB free)
  Uptime: 12005.0 sec
  Load Avg:  1.30615234375  1.28466796875  1.3798828125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

Baseline

Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      Ubuntu 18.04.4 LTS
  uname: Linux 5.3.0-51-generic #44~18.04.2-Ubuntu SMP Thu Apr 23 14:27:18 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz: 
              speed         user         nice          sys         idle          irq
       #1  2707 MHz     202460 s        228 s      61032 s     939231 s          0 s
       #2  2717 MHz     185212 s        211 s      57674 s     958518 s          0 s
       #3  2700 MHz     204903 s        185 s      60222 s     929670 s          0 s
       #4  2763 MHz     256176 s        370 s      59647 s     888520 s          0 s
       
  Memory: 7.654296875 GB (1347.9765625 MB free)
  Uptime: 12190.0 sec
  Load Avg:  1.4443359375  1.32861328125  1.37109375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)