Skip to content

Instantly share code, notes, and snippets.

Avatar

DomHudson

View GitHub Profile
View dask.md

I would recommend Dask for this sort of work as it is easy to produce a task graph and minimise computation overhead

import dask


def A(argument):
    print('Running A with', argument)
    return argument
@DomHudson
DomHudson / custom_worker_processes.py
Last active Aug 15, 2018
Multiprocessing in Python with Custom Workers
View custom_worker_processes.py
import random
import multiprocessing
context = multiprocessing.get_context()
class Worker(context.Process):
def __init__(self, *args, **kwargs):