Skip to content

Instantly share code, notes, and snippets.

View pentschev's full-sized avatar

Peter Andreas Entschev pentschev

47 followers · 0 following
  • NVIDIA
View GitHub Profile
@pentschev
pentschev / dask_cupy_custom_kernel.py
Created August 9, 2019 20:15
Blog Post - Parallelizing Custom CuPy Kernels with Dask - Complete
from dask.distributed import Client
from dask_cuda import LocalCUDACluster
from dask.array.utils import assert_eq
import dask.array as da
import cupy
add_broadcast_kernel = cupy.RawKernel(
r'''
extern "C" __global__
@pentschev
pentschev / pynvml_query_memory.py
Last active April 28, 2020 08:22
Query used GPU memory with pynvml
import datetime
import getopt
import os
import sys
import time
import pynvml
def get_printable_util_mem(dev_count, peak_mem):
@pentschev
pentschev / occupancy_hang.txt
Created September 17, 2019 18:30
Thread 9 (Thread 0x7f2da4fe8700 (LWP 39329)):
Undefined command: "btr". Try "help".
Thread 9 (Thread 0x7f2da4fe8700 (LWP 39329)):
#0 0x00007f2dccf7b709 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x000055787b3902d8 in PyCOND_TIMEDWAIT (cond=0x55787b5c2a38 <_PyRuntime+1208>, mut=0x55787b5c2a68 <_PyRuntime+1256>,
us=5000) at /home/conda/feedstock_root/build_artifacts/python_1562015400360/work/Python/condvar.h:90
#2 take_gil (tstate=0x55787f257a40)
at /home/conda/feedstock_root/build_artifacts/python_1562015400360/work/Python/ceval_gil.h:208
@pentschev
pentschev / dask_cudf_cuml.ipynb
Created September 3, 2019 19:55
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@pentschev
pentschev / ucx_map_overlap_smooth.py
Last active August 16, 2019 10:06
UCX map overlap sample
import asyncio
import time
import numpy as np
import cupy
import numba
import dask.array as da
from dask_cuda import DGX, LocalCUDACluster
from dask.distributed import Client, wait
@pentschev
pentschev / ucx_setup.sh
Last active August 15, 2019 13:25
Script for setup of a UCX-enabled conda environment
#!/bin/bash
SOURCE_PATH=${HOME}/ucx-src
CONDA_ENV_NAME=ucx
RAPIDS_VERSION="0.9"
CUDA_VERSION="9.2"
# CUDA_VERSION="10.0"
if [ ! -d ${SOURCE_PATH} ]; then
mkdir ${SOURCE_PATH}
fi
@pentschev
pentschev / ucx_info_-b.diff
Last active August 14, 2019 17:06
--- ucx_info_b 2019-08-08 11:01:32.820637475 -0700
+++ ucx_from_source_info_b 2019-08-08 11:02:15.976537113 -0700
@@ -8,6 +8,7 @@
#define HAVE_CUDA 1
#define HAVE_CUDA_H 1
#define HAVE_CUDA_RUNTIME_H 1
+#define HAVE_DC_EXP 1
#define HAVE_DECL_ASPRINTF 1
#define HAVE_DECL_BASENAME 1
#define HAVE_DECL_CPU_ISSET 1
@pentschev
pentschev / map_overlap_simple_10k_tcp.html
Created August 12, 2019 22:50
Task Streams for map_overlap(lambda x: x) with UCX and TCP
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@pentschev
pentschev / map_overlap_10k_tcp.html
Created August 12, 2019 22:10
Task Streams for map_overlap with UCX and TCP
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@pentschev
pentschev / map_overlap_10k_tcp.html
Created August 12, 2019 22:10
Task Streams for map_overlap with UCX and TCP
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">