Skip to content

Instantly share code, notes, and snippets.

Avatar

Jacob Tomlinson jacobtomlinson

View GitHub Profile
@jacobtomlinson
jacobtomlinson / notebook.ipynb
Last active Mar 11, 2022
Launch a hybrid CPU and GPU Dask cluster and use annotations to pin work to specific hardware
View notebook.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jacobtomlinson
jacobtomlinson / poke.py
Created Jan 6, 2022
Poke Dask dashboard
View poke.py
# A script that gives a local Dask cluster something to do without stressing hardware.
# Useful for testing the dashboard.
import time
from dask_ctl import get_cluster
from dask.distributed import Client, wait
from dask import delayed
@jacobtomlinson
jacobtomlinson / aws.md
Created Aug 26, 2021
Dask Cloudprovider AWS docs rst-to-myst
View aws.md

Amazon Web Services (AWS)

.. currentmodule:: dask_cloudprovider.aws
.. autosummary::
   EC2Cluster
 ECSCluster
@jacobtomlinson
jacobtomlinson / Introduction_to_Dask_with_RAPIDS.ipynb
Last active Jul 20, 2022
Intro to distributed computing on GPUs with Dask in Python
View Introduction_to_Dask_with_RAPIDS.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jacobtomlinson
jacobtomlinson / docker-compose.yml
Created Mar 25, 2021
Monitoring RAPIDS with Prometheus and Grafana (configs)
View docker-compose.yml
version: "3.9"
services:
rapids:
image: rapidsai/rapidsai:0.18-cuda11.0-runtime-ubuntu16.04-py3.8
ports:
- "8888:8888" # Jupyter
- "8786:8786" # Dask communication
- "8787:8787" # Dask dashboard
@jacobtomlinson
jacobtomlinson / rapids-aws-dask-cloudprovider.ipynb
Created Dec 3, 2020
Launching a RAPIDS cluster on AWS with Dask Cloudprovider
View rapids-aws-dask-cloudprovider.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jacobtomlinson
jacobtomlinson / intro-to-gpu-development-in-python.ipynb
Last active Sep 27, 2021
Tech Exeter Conference 2020: Intro to GPU Development in Python
View intro-to-gpu-development-in-python.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jacobtomlinson
jacobtomlinson / good-timeout.py
Last active Jul 19, 2021
Asyncio wait_for can be foiled by blocking code
View good-timeout.py
import asyncio
async def eternity():
await asyncio.sleep(2) # Sleep asynchronously
print("I should've timed out")
async def main():
# Wait for at most 1 second
try:
await asyncio.wait_for(eternity(), timeout=1.0)
@jacobtomlinson
jacobtomlinson / Dask CuPy cuDF Example.ipynb
Created Feb 11, 2020
Example of using CuPy and cuDF with Dask CUDA
View Dask CuPy cuDF Example.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View 2020-02-03 Azure UCX-Py Testing.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.