Skip to content

Instantly share code, notes, and snippets.

@renatocfrancisco
Last active July 19, 2023 15:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save renatocfrancisco/fd4c1aba67fb0c8177d67d843e257478 to your computer and use it in GitHub Desktop.
Save renatocfrancisco/fd4c1aba67fb0c8177d67d843e257478 to your computer and use it in GitHub Desktop.
Multi processing in Python with mp.Process and concurrent
import multiprocessing as mp
import time
import concurrent.futures
def do_some(seconds = 1):
print(f'sleeping {seconds}')
time.sleep(seconds)
return f'done {seconds}'
if __name__ == '__main__':
with concurrent.futures.ProcessPoolExecutor() as executor:
# --------------
print('concurrent with map')
start = time.perf_counter()
secs = [5,4,3,2,1]
results = executor.map(do_some, secs)
for result in results:
print(result)
finish = time.perf_counter()
print(f"Tempo: {round(finish - start,2)} \n")
# --------------
print('concurrent with list comprehension')
start = time.perf_counter()
results = [executor.submit(do_some, sec) for sec in secs]
for f in concurrent.futures.as_completed(results):
print(f.result())
finish = time.perf_counter()
print(f"Tempo: {round(finish - start,2)} \n")
# ------------
print('with concurrent')
start = time.perf_counter()
f1 = executor.submit(do_some, 1)
f2 = executor.submit(do_some, 1)
print(f1.result())
print(f2.result())
finish = time.perf_counter()
print(f"Tempo: {round(finish - start,2)} \n")
# # # -------------
print('with multiprocessing (loop)')
start = time.perf_counter()
processes = []
seconds = [5,4,3,2,1]
for i in range(5):
p = mp.Process(target=do_some, args=[seconds[i],])
p.start()
processes.append(p)
for process in processes:
process.join()
finish = time.perf_counter()
print(f"Tempo: {round(finish - start,2)} \n")
# # -------------
print('with multiprocessing')
start = time.perf_counter()
p1 = mp.Process(target=do_some)
p2 = mp.Process(target=do_some)
p1.start()
p2.start()
p1.join()
p2.join()
finish = time.perf_counter()
print(f"Tempo: {round(finish - start,2)} \n")
@renatocfrancisco
Copy link
Author

concurrent with map
sleeping 5
sleeping 4
sleeping 3
sleeping 2
sleeping 1
done 5
done 4
done 3
done 2
done 1
Tempo: 5.12

concurrent with list comprehension
sleeping 5
sleeping 3
sleeping 4
sleeping 2
sleeping 1
done 1
done 2
done 3
done 4
done 5
Tempo: 5.0     

with concurrent
sleeping 1     
sleeping 1     
done 1
done 1     
Tempo: 1.0 

with multiprocessing (loop)
sleeping 5
sleeping 4
sleeping 2
sleeping 3
sleeping 1
Tempo: 5.14 

with multiprocessing
sleeping 1
sleeping 1
Tempo: 1.14

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment