Instantly share code, notes, and snippets.

# azev77/diffeqflux_differentialequations_vs_torchdiffeq_results.md

Created February 8, 2020 00:09
torchdiffeq (Python) vs DifferentialEquations.jl (Julia) ODE Benchmarks

# Torchdiffeq vs DifferentialEquations.jl (/ DiffEqFlux.jl) Benchmarks

Benchmark: Solve the Lorenz equations from 0 to 100 with abstol=reltol=1e-8

## Absolute Timings

• DifferentialEquations.jl: 1.675 ms
• diffeqpy (DifferentialEquations.jl called from Python): 3.473 ms
• SciPy+Numba: 50.99 ms
• SciPy: 110.6 ms
• torchdiffeq: 48 seconds
• torchscript torchdiffeq: 48 seconds

## Timings Relative to DifferentialEquations.jl

• DifferentialEquations.jl: 1x
• diffeqpy (DifferentialEquations.jl called from Python): 2.07x Slower
• SciPy+Numba: 30x Slower
• SciPy: 66x Slower
• torchdiffeq: 30,000x Slower
• torchscript torchdiffeq: 30,000x Slower

The torchscript versions are kept as separate scripts to allow for the JITing process to occur, and are called before timing to exclude JIT timing, as per the PyTorch documentation suggestions. Python results were scaled by the number of times ran in timeit.

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import numpy as np from julia import Main from diffeqpy import de jul_f = Main.eval(""" function f(du,u,p,t) x, y, z = u, u, u sigma, rho, beta = p, p, p du = sigma * (y - x) du = x * (rho - z) - y du = x * y - beta * z end""") u0 = [1.0,0.0,0.0] tspan = (0., 100.) t = np.linspace(0, 100, 1001) p = [10.0,28.0,8/3] prob = de.ODEProblem(jul_f, u0, tspan, p) sol = de.solve(prob,saveat=t,abstol=1e-8,reltol=1e-8) def time_func(): sol = de.solve(prob,saveat=t,abstol=1e-8,reltol=1e-8) timeit.Timer(time_func).timeit(number=100) # 0.347320199999956 seconds
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 using OrdinaryDiffEq, StaticArrays, BenchmarkTools function lorenz_static(u,p,t) @inbounds begin dx = p*(u-u) dy = u*(p-u) - u dz = u*u - p*u end @SVector [dx,dy,dz] end u0 = @SVector [1.0,0.0,0.0] p = @SVector [10.0,28.0,8/3] tspan = (0.0,100.0) prob = ODEProblem(lorenz_static,u0,tspan,p) @btime solve(prob,Tsit5(),saveat=0.1,reltol=1e-8,abstol=1e-8) # 1.879 ms (56 allocations: 60.19 KiB) @btime solve(prob,DP5(),saveat=0.1,reltol=1e-8,abstol=1e-8) # 1.675 ms (56 allocations: 59.83 KiB)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import numpy as np from scipy.integrate import odeint import timeit import numba def f(u, t, sigma, rho, beta): x, y, z = u return [sigma * (y - x), x * (rho - z) - y, x * y - beta * z] u0 = [1.0,0.0,0.0] tspan = (0., 100.) t = np.linspace(0, 100, 1001) sol = odeint(f, u0, t, args=(10.0,28.0,8/3)) def time_func(): odeint(f, u0, t, args=(10.0,28.0,8/3),rtol = 1e-8, atol=1e-8) timeit.Timer(time_func).timeit(number=100) # 11.0623657 seconds numba_f = numba.jit(f,nopython=True) def time_func(): odeint(numba_f, u0, t, args=(10.0,28.0,8/3),rtol = 1e-8, atol=1e-8) timeit.Timer(time_func).timeit(number=100) # 5.099028400000009 seconds
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import torch from torchdiffeq import odeint import timeit @torch.jit.script def f_script(t, u): x, y, z = u,u,u du1 = 10.0 * (y - x) du2 = x * (28.0 - z) - y du3 = x * y - 2.66 * z return torch.stack([du1, du2, du3]) u0 = torch.tensor([1.0,0.0,0.0]) t = torch.linspace(0, 100, 1001) sol = odeint(f_script, u0, t) def time_func(): odeint(f_script, u0, t, rtol = 1e-8, atol=1e-8) time_func() _t = timeit.Timer(time_func).timeit(number=2) print(_t) # 106.84723600000001 seconds
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import torch from torchdiffeq import odeint import timeit @torch.jit.script class LorenzODE(torch.nn.Module): def __init__(self): super(LorenzODE, self).__init__() def forward(self, t, u): x, y, z = u,u,u du1 = 10.0 * (y - x) du2 = x * (28.0 - z) - y du3 = x * y - 2.66 * z return torch.stack([du1, du2, du3]) u0 = torch.tensor([1.0,0.0,0.0]) t = torch.linspace(0, 100, 1001) sol = odeint(LorenzODE(), u0, t) def time_func(): odeint(LorenzODE(), u0, t, rtol = 1e-8, atol=1e-8) time_func() _t = timeit.Timer(time_func).timeit(number=2) print(_t) # 96.36945809999997 seconds
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 import torch from torchdiffeq import odeint import timeit def f(t, u): x, y, z = u,u,u du1 = 10.0 * (y - x) du2 = x * (28.0 - z) - y du3 = x * y - 2.66 * z return torch.stack([du1, du2, du3]) u0 = torch.tensor([1.0,0.0,0.0]) t = torch.linspace(0, 100, 1001) sol = odeint(f, u0, t) def time_func(): odeint(f, u0, t, rtol = 1e-8, atol=1e-8) time_func() _t = timeit.Timer(time_func).timeit(number=2) print(_t) # 94.1328381000003 seconds class LorenzODE(torch.nn.Module): def __init__(self): super(LorenzODE, self).__init__() def forward(self, t, u): x, y, z = u,u,u du1 = 10.0 * (y - x) du2 = x * (28.0 - z) - y du3 = x * y - 2.66 * z return torch.stack([du1, du2, du3]) sol = odeint(LorenzODE("cpu"), u0, t) def time_func(): odeint(LorenzODE("cpu"), u0, t, rtol = 1e-8, atol=1e-8) time_func() _t = timeit.Timer(time_func).timeit(number=2) print(_t) # 96.36945809999997 seconds import torch from torchdiffeq import odeint import timeit @torch.jit.script def f_script(t, u): x, y, z = u,u,u du1 = 10.0 * (y - x) du2 = x * (28.0 - z) - y du3 = x * y - 2.66 * z return torch.stack([du1, du2, du3]) u0 = torch.tensor([1.0,0.0,0.0]) t = torch.linspace(0, 100, 1001) sol = odeint(f_script, u0, t) def time_func(): odeint(f_script, u0, t, rtol = 1e-8, atol=1e-8) time_func() _t = timeit.Timer(time_func).timeit(number=2) print(_t) # 106.84723600000001 seconds