Skip to content

Instantly share code, notes, and snippets.

@jakirkham
jakirkham / trace.py
Created January 29, 2021 20:40 — forked from jcrist/trace.py
Dask Execution Tracer
import os
from dask.callbacks import Callback
from dask.dot import dot_graph
class Track(Callback):
def __init__(self, path='dasks', save_every=1):
self.path = path
self.save_every = save_every
@jakirkham
jakirkham / pynvml_query_memory.py
Created January 13, 2020 22:50 — forked from pentschev/pynvml_query_memory.py
Query used GPU memory with pynvml
import datetime
import getopt
import os
import sys
import time
import pynvml
def print_mem(dev_count):
for index in range(dev_count):
@jakirkham
jakirkham / dask_cudf_merge_benchmark.ipynb
Created December 3, 2019 16:19 — forked from rjzamora/dask_cudf_merge_benchmark.ipynb
dask_cudf + UCX merge benchmark
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jakirkham
jakirkham / dask_image_watershed.ipynb
Created November 12, 2019 21:17 — forked from wwarriner/dask_image_watershed.ipynb
First attempt at distributed watershed
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
import asyncio
import time
import numpy as np
import cupy
import numba
import dask.array as da
from dask_cuda import DGX, LocalCUDACluster
from dask.distributed import Client, wait
--- ucx_info_b 2019-08-08 11:01:32.820637475 -0700
+++ ucx_from_source_info_b 2019-08-08 11:02:15.976537113 -0700
@@ -8,6 +8,7 @@
#define HAVE_CUDA 1
#define HAVE_CUDA_H 1
#define HAVE_CUDA_RUNTIME_H 1
+#define HAVE_DC_EXP 1
#define HAVE_DECL_ASPRINTF 1
#define HAVE_DECL_BASENAME 1
#define HAVE_DECL_CPU_ISSET 1
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.