Skip to content

Instantly share code, notes, and snippets.

@jolks
Last active April 6, 2016 12:39
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save jolks/8648504 to your computer and use it in GitHub Desktop.
Save jolks/8648504 to your computer and use it in GitHub Desktop.
Parallel operation with multiple arguments in Python
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from multiprocessing import Pool
import math
def f(x, y, z):
# (x + 2y)^z
return math.pow(x + 2*y, z)
# In Python 3.3, can use starmap() method instead
def f_star(multiple_args):
# Changes e.g. f([1,2,3]) to f(1, 2, 3)
return f(*multiple_args)
# Set no. of workers in the pool.
# If the argument is not set, it defaults to machine no. of cores.
p = Pool(4) # Initialize 4 parallel processes which can be seen using top command
# Regardless of input type, output will be list type
print p.map(f_star, [[1, 2, 3], [3, 4, 5], [5, 6, 7]]) # list of lists input
print p.map(f_star, ((1, 2, 3), (3, 4, 5), (5, 6, 7))) # tuple of tuples input
# Close the pool. No task is allowed anymore.
p.close()
# Wait for the workers to exit.
p.join()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment