Skip to content

Instantly share code, notes, and snippets.

@serihiro
Last active May 31, 2019 04:38
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save serihiro/bb9614b816b9c35e95ec4b53d3382f72 to your computer and use it in GitHub Desktop.
Save serihiro/bb9614b816b9c35e95ec4b53d3382f72 to your computer and use it in GitHub Desktop.
python mulitprocessing.Queue test for each Context
import multiprocessing
import argparse
queue = None
def task(queue, value):
queue.put(value)
def task2(value):
queue.put(value)
'''
$ python mp_queue_test.py --start_method fork
1 2 3
$ python mp_queue_test.py --start_method spawn
Process Process-3:
Traceback (most recent call last):
File "/Users/kazuhiroserizawa/.pyenv/versions/3.7.2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/Users/kazuhiroserizawa/.pyenv/versions/3.7.2/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/Users/kazuhiroserizawa/Documents/python-work/experiments/mp_queue_test.py", line 12, in task2
queue.put(value)
AttributeError: 'NoneType' object has no attribute 'put'
$ python mp_queue_test.py --start_method forkserver
Process Process-3:
Traceback (most recent call last):
File "/Users/kazuhiroserizawa/.pyenv/versions/3.7.2/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/Users/kazuhiroserizawa/.pyenv/versions/3.7.2/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/Users/kazuhiroserizawa/Documents/python-work/experiments/mp_queue_test.py", line 12, in task2
queue.put(value)
AttributeError: 'NoneType' object has no attribute 'put'
'''
def main():
global queue
queue = multiprocessing.Queue(100)
p1 = multiprocessing.Process(target=task, args=[queue, 1])
p2 = multiprocessing.Process(target=task, args=[queue, 2])
p3 = multiprocessing.Process(target=task2, args=[3])
p1.start()
p2.start()
p3.start()
p1.join()
p2.join()
p3.join()
print(queue.get(), queue.get(), queue.get())
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--start_method", type=str, default="fork")
args = parser.parse_args()
multiprocessing.set_start_method(args.start_method)
main()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment