Skip to content

Instantly share code, notes, and snippets.

@raytroop
Last active September 30, 2018 05:17
Show Gist options
  • Save raytroop/3af5e81ce04ba3e263d3ad1520623255 to your computer and use it in GitHub Desktop.
Save raytroop/3af5e81ce04ba3e263d3ad1520623255 to your computer and use it in GitHub Desktop.
python ThreadPoolExecutor
from concurrent.futures import ThreadPoolExecutor

fastai

with ThreadPoolExecutor(num_cpus()) as e:
  ims = e.map(lambda fname: safely_process(fname), fnames)

dl2

def resize_mask(fn):
    Image.open(fn).resize((128,128)).save((fn.parent.parent)/'train_masks-128'/fn.name)

files = list((PATH/'train_masks_png').iterdir())
with ThreadPoolExecutor(8) as e: 
  e.map(resize_mask, files)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment