Skip to content

Instantly share code, notes, and snippets.

@cswiercz
Created October 29, 2014 22:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cswiercz/b9ac7287b11371078cf1 to your computer and use it in GitHub Desktop.
Save cswiercz/b9ac7287b11371078cf1 to your computer and use it in GitHub Desktop.
Simple Parallelization in Python with Pools
# Given a function, func(var), the map() function will apply func() to every
# element of a list and return a list of equal size.
def func(var):
return var**2
numbers = [1,2,3,4]
squares = map(func, numbers) # [1,4,9,16]
# The multiprocessing module contains various objects for performing parallel
# tasks in Python. The "Pool" object allows you to perform a map()-like
# operation but over several processors / cores.
import multiprocessing
from multiprocessing import Pool
pool = Pool(processes=4) # set the number of processes you want to spawn
squares_parallel = pool.map(func, numbers) # [1,4,9,16]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment