Skip to content

Instantly share code, notes, and snippets.

@lan496
Created May 19, 2019 07:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lan496/4b8db59d1d6fe020095dbe13bbe6e526 to your computer and use it in GitHub Desktop.
Save lan496/4b8db59d1d6fe020095dbe13bbe6e526 to your computer and use it in GitHub Desktop.
parallel computation with timeout handling
from time import sleep
from joblib import Parallel, delayed
from timeout_decorator import timeout, TimeoutError
@timeout(10, use_signals=False)
def f(n):
if n == 0:
sleep(42)
return n
def wrapped_f(n):
try:
return f(n)
except TimeoutError:
return None
if __name__ == '__main__':
ret = Parallel(n_jobs=-1, backend='threading')([delayed(wrapped_f)(n) for n in range(10)])
print(ret) # -> [None, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment