A benchmark for ForwardDiff

benchmark.md

Benchmark Report for ForwardDiff

Job Properties

• Time of benchmarks:
  • Target: 18 Jul 2018 - 18:43
  • Baseline: 18 Jul 2018 - 18:44
• Package commits:
  • Target: 1619fc
  • Baseline: 6d0eb4
• Julia commits:
  • Target: 9d11f6
  • Baseline: 9d11f6
• Julia command flags:
  • Target: -O3
  • Baseline: -O3
• Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID	time ratio	memory ratio
["scalar p=2 width=4 r=0.0"]	1.17 (5%) ❌	1.00 (1%)
["scalar p=2 width=4 r=1.0"]	1.17 (5%) ❌	1.00 (1%)
["scalar p=2 width=8 r=0.0"]	1.17 (5%) ❌	1.00 (1%)
["scalar p=2 width=8 r=1.0"]	1.17 (5%) ❌	1.00 (1%)
["scalar p=3 width=4 r=0.0"]	0.85 (5%) ✅	1.00 (1%)
["scalar p=3 width=4 r=1.0"]	0.60 (5%) ✅	1.00 (1%)
["scalar p=3 width=8 r=0.0"]	0.78 (5%) ✅	1.00 (1%)
["scalar p=3 width=8 r=1.0"]	0.61 (5%) ✅	1.00 (1%)
["scalar p=4 width=4 r=0.0"]	1.15 (5%) ❌	1.00 (1%)
["scalar p=4 width=8 r=0.0"]	0.85 (5%) ✅	1.00 (1%)
["scalar p=4 width=8 r=1.0"]	0.87 (5%) ✅	1.00 (1%)
["scalar p=5 width=4 r=0.0"]	1.27 (5%) ❌	1.00 (1%)
["scalar p=5 width=4 r=1.0"]	1.37 (5%) ❌	1.00 (1%)
["scalar p=5 width=8 r=0.0"]	1.05 (5%) ❌	1.00 (1%)
["scalar p=5 width=8 r=1.0"]	1.18 (5%) ❌	1.00 (1%)
["scalar p=6 width=4 r=0.0"]	1.32 (5%) ❌	1.00 (1%)
["scalar p=6 width=4 r=1.0"]	1.46 (5%) ❌	1.00 (1%)
["scalar p=6 width=8 r=1.0"]	1.27 (5%) ❌	1.00 (1%)
["scalar p=7 width=4 r=0.0"]	1.44 (5%) ❌	1.00 (1%)
["scalar p=7 width=4 r=1.0"]	1.61 (5%) ❌	1.00 (1%)
["scalar p=7 width=8 r=0.0"]	1.22 (5%) ❌	1.00 (1%)
["scalar p=7 width=8 r=1.0"]	1.44 (5%) ❌	1.00 (1%)
["scalar p=8 width=4 r=0.0"]	1.56 (5%) ❌	1.00 (1%)
["scalar p=8 width=4 r=1.0"]	1.68 (5%) ❌	1.00 (1%)
["scalar p=8 width=8 r=0.0"]	1.29 (5%) ❌	1.00 (1%)
["scalar p=8 width=8 r=1.0"]	1.50 (5%) ❌	1.00 (1%)
["vector p=2 width=4"]	0.78 (5%) ✅	1.00 (1%)
["vector p=2 width=8"]	2.09 (5%) ❌	1.00 (1%)
["vector p=3 width=4"]	0.54 (5%) ✅	1.00 (1%)
["vector p=3 width=8"]	0.54 (5%) ✅	1.00 (1%)
["vector p=4 width=4"]	0.66 (5%) ✅	1.00 (1%)
["vector p=4 width=8"]	0.78 (5%) ✅	1.00 (1%)
["vector p=5 width=4"]	1.18 (5%) ❌	1.00 (1%)
["vector p=5 width=8"]	1.14 (5%) ❌	1.00 (1%)
["vector p=6 width=4"]	1.33 (5%) ❌	1.00 (1%)
["vector p=6 width=8"]	1.33 (5%) ❌	1.00 (1%)
["vector p=7 width=4"]	1.46 (5%) ❌	1.00 (1%)
["vector p=7 width=8"]	1.47 (5%) ❌	1.00 (1%)
["vector p=8 width=4"]	1.55 (5%) ❌	1.00 (1%)
["vector p=8 width=8"]	1.55 (5%) ❌	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

• []

Julia versioninfo

Target

Julia Version 0.6.4
Commit 9d11f62bcb (2018-07-09 19:09 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  WORD_SIZE: 64
           Ubuntu 16.04.4 LTS
  uname: Linux 4.4.0-119-generic #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x86_64 x86_64
Memory: 125.78757095336914 GB (56240.15234375 MB free)
Uptime: 3.559675e6 sec
Load Avg:  0.99462890625  0.8955078125  0.76953125
Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz: 
          speed         user         nice          sys         idle          irq
#1-48  1200 MHz  211755009 s      11362 s  184121157 s  16682327038 s          0 s

  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=16)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

Baseline

Julia Version 0.6.4
Commit 9d11f62bcb (2018-07-09 19:09 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  WORD_SIZE: 64
           Ubuntu 16.04.4 LTS
  uname: Linux 4.4.0-119-generic #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x86_64 x86_64
Memory: 125.78757095336914 GB (56215.9453125 MB free)
Uptime: 3.559729e6 sec
Load Avg:  1.35107421875  0.9970703125  0.8125
Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz: 
          speed         user         nice          sys         idle          irq
#1-48  1200 MHz  211762380 s      11362 s  184121463 s  16682577993 s          0 s

  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=16)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

script.jl

using ForwardDiff: Dual
using BenchmarkTools

const SUITE = BenchmarkGroup()

exp_taylor_expr(p::Int, x) =
    reduce((a, b) -> :($a + $b), [:($x^$n * $(1 / factorial(n))) for n in 0:p])

@inline @generated exp_taylor(::Val{p}, x) where p = exp_taylor_expr(p, :x)

function vec_exp!(vp::Val, ys, xs)
    @. ys = exp_taylor(vp, xs)
end

for p in 2:8, r in [0.0, 1.0], w in [4, 8]
    x = Dual(r, (1:1.0:w)...)
    SUITE["scalar p=$p width=$w r=$r"] = @benchmarkable exp_taylor($(Val(p)), $x)
end

for p in 2:8, w in [4, 8]
    xs = Dual.(0:0.01:1, (1:1.0:w)...)
    ys = copy(xs)
    SUITE["vector p=$p width=$w"] = @benchmarkable vec_exp!($(Val(p)), $ys, $xs)
end

let xs = -1:0.1:1
    @assert exp_taylor.(Val(12), xs) ≈ exp.(xs)
end