Skip to content

Instantly share code, notes, and snippets.

@CSUwangj
Created April 10, 2019 08:40
Show Gist options
  • Save CSUwangj/63c7929a16fbe249c7f6d73a8ed1e8f5 to your computer and use it in GitHub Desktop.
Save CSUwangj/63c7929a16fbe249c7f6d73a8ed1e8f5 to your computer and use it in GitHub Desktop.
[python] multi-threads with a maximum number of simultaneous threads
import queue
import threading
num_worker_threads = 10
source = range(1000)
def do_work(item):
print(item)
def worker():
while True:
item = q.get()
if item is None:
break
do_work(item)
q.task_done()
q = queue.Queue()
threads = []
for i in range(num_worker_threads):
t = threading.Thread(target=worker)
t.start()
threads.append(t)
for item in source:
q.put(item)
# block until all tasks are done
q.join()
# stop workers
for i in range(num_worker_threads):
q.put(None)
for t in threads:
t.join()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment