Skip to content

Instantly share code, notes, and snippets.

@nnfuzzy

nnfuzzy/scalable-matrix-multiplication-not-using-spark-2-but-dask

Created January 2, 2018 12:59
Show Gist options
  • Save nnfuzzy/504bdcae10186423ad8de992ce417bfd to your computer and use it in GitHub Desktop.
Save nnfuzzy/504bdcae10186423ad8de992ce417bfd to your computer and use it in GitHub Desktop.
Download ZIP
scalable-matrix-multiplication-using-dask
Raw
scalable-matrix-multiplication-not-using-spark-2-but-dask
import dask.array as da
# Compare with http://bdp.wordpress-staging.uk/2015/11/23/scalable-matrix-multiplication-using-spark-2/
n=1000000000
numberOfChunks=n/100000
a = da.random.random((2,n),chunks=numberOfChunks)
b = da.random.random((n,3),chunks=numberOfChunks)
a.dot(b).compute()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment