Skip to content

Instantly share code, notes, and snippets.

@JDWarner
Created July 12, 2015 14:47
Show Gist options
  • Save JDWarner/97902becb7ed682e8513 to your computer and use it in GitHub Desktop.
Save JDWarner/97902becb7ed682e8513 to your computer and use it in GitHub Desktop.
dask.array performance compared to NumPy arrays in color conversion
import numpy as np
import dask.array as da
# Benchmark: NumPy with array construction included
%timeit nparr0 = np.random.random(size=(10000, 1000)); \
nparr1 = np.random.random(size=(10000, 1000)); \
np.sqrt((nparr0 - nparr1) ** 2 + (nparr0 - nparr1) ** 2 + (nparr0 - nparr1) ** 2)
# Benchmark: Dask with dask.array construction included
%timeit x = da.random.random(size=(10000, 1000), chunks=(10000, 1000)); \
y = da.random.random(size=(10000, 1000), chunks=(10000, 1000)); \
(da.sqrt((x - y) ** 2 + (x - y) ** 2 + (x - y) ** 2)).compute()
# Benchmark (unfair): compute using existing NumPy arrays
nparr0 = np.random.random(size=(10000, 1000))
nparr1 = np.random.random(size=(10000, 1000))
%timeit np.sqrt((nparr0 - nparr1) ** 2 + (nparr0 - nparr1) ** 2 + (nparr0 - nparr1) ** 2)
# Benchmark: dask.from_array() from NumPy arrays
%timeit x = da.from_array(nparr0, chunks=(10000, 1000)); \
y = da.from_array(nparr1, chunks=(10000, 1000)); \
(da.sqrt((x - y) ** 2 + (x - y) ** 2 + (x - y) ** 2)).compute()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment