Skip to content

Instantly share code, notes, and snippets.

View quasiben's full-sized avatar

Benjamin Zaitlen quasiben

146 followers · 4 following
View GitHub Profile
@quasiben
quasiben / big-shuffle-optimized.html
Created January 8, 2021 15:19
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@quasiben
quasiben / profiled-main-not-optimized.html
Created January 8, 2021 14:16
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@quasiben
quasiben / profiled-main-not-optimized.html
Created January 8, 2021 14:15
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@quasiben
quasiben / profiled-main-optimized.html
Created January 8, 2021 14:12
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@quasiben
quasiben / RL-w-fft-w-dask-3D.ipynb
Last active November 23, 2020 11:43
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@quasiben
quasiben / Napari-RL-w-fft-w-dask-3D.ipynb
Last active November 13, 2020 18:02
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@quasiben
quasiben / RL-w-fft-w-dask.ipynb
Created October 21, 2020 20:20
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@quasiben
quasiben / napari-dask-imaging.ipynb
Created October 21, 2020 20:20
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@quasiben
quasiben / cudf-merge-stats.txt
Created October 16, 2020 20:58
NVTX Push-Pop Range Statistics:
Time(%) Total Time (ns) Instances Average Minimum Maximum Range
------- --------------- --------- ------------- ---------- ----------- ------------------------------------
17.2 64,995,321,276 394,497 164,754.9 625 901,797,038 ucxpy:UCXPY_PROGRESS
14.5 54,752,809,329 486 112,660,101.5 891,800 252,248,846 libcudf:inner_join
10.9 41,264,369,784 243 169,812,221.3 1,515,855 193,479,633 cudf_python:SORT_INDEX
10.6 40,038,984,207 648 61,788,555.9 59,485,360 66,858,388 cudf_python:SCATTER_BY_MAP
10.3 39,047,108,085 243 160,687,687.6 341,675 181,847,452 libcudf:sorted_order
10.0 37,966,864,680 648 58,590,840.6 53,434,145 65,389,102 libcudf:partition
@quasiben
quasiben / prom-dask-cudf-merge-test.sh
Created October 9, 2020 14:31
#!/bin/bash -eu
# Environment variables to enable GPUs, InfiniBand, NVLink
# These are read by the scheduler and client script
module load cuda/11.0.3
export PYTHON=/gpfs/fs1/bzaitlen/miniconda3/envs/20201008/bin/python
export DASK_UCX__CUDA_COPY=True
export DASK_UCX__TCP=True
export DASK_UCX__NVLINK=True
export DASK_UCX__INFINIBAND=True