Skip to content

Instantly share code, notes, and snippets.

@hanneshapke

hanneshapke/parallelize.py

Created November 1, 2020 01:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hanneshapke/536e842471617af759e6fd3d19a77285 to your computer and use it in GitHub Desktop.
Save hanneshapke/536e842471617af759e6fd3d19a77285 to your computer and use it in GitHub Desktop.
Download ZIP
Parallelize data processing
Raw
parallelize.py
# install job lib
from joblib import Parallel, delayed
import numpy as np
def myfun(p):
return np.linalg.norm(p[0]-p[1])
X = [1,2,3,4]
Y = [6,4,8,1]
element = [(x, y) for x in X for y in Y]
results = Parallel(n_jobs=-1, verbose=1, backend="threading")(map(delayed(myfun), element))
print(results)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment