Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@jakirkham
jakirkham / dask_xgboost_higgs.ipynb
Created June 16, 2022 04:44
Notebook loading Higgs dataset for use with Dask + XGBoost
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file has been truncated, but you can view the full file.
/* Generated by Cython 0.29.21 */
#define PY_SSIZE_T_CLEAN
#include "Python.h"
#ifndef Py_PYTHON_H
#error Python headers needed to compile C extensions, please install development version of Python.
#elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000)
#error Cython requires Python 2.6+ or Python 3.3+.
#else
#define CYTHON_ABI "0_29_21"
@jakirkham
jakirkham / trace.py
Created January 29, 2021 20:40 — forked from jcrist/trace.py
Dask Execution Tracer
import os
from dask.callbacks import Callback
from dask.dot import dot_graph
class Track(Callback):
def __init__(self, path='dasks', save_every=1):
self.path = path
self.save_every = save_every
@jakirkham
jakirkham / Miniconda_CPython_RAPIDS_install_exp.txt
Created March 19, 2020 05:27
Some experiments installing RAPIDS using Conda with CPython and PyPy respectively
(base) jkirkham@dgx15:~$ clear
@jakirkham
jakirkham / dask-report-rmm_suballoc-tcp.html
Created March 10, 2020 04:17
Dask Reports using RMM Suballocator
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@jakirkham
jakirkham / meta.yaml
Last active March 5, 2020 03:02
Hacky workaround for CPython 3.6 ABI issue
package:
name: blah
version: 1
requirements:
host:
- python
- python_abi * *_cp36m # [py36]
run:
- python
@jakirkham
jakirkham / dask-report.html
Created March 3, 2020 04:47
Dask/Distributed GH# 932 (Bandwidth during shuffles)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@jakirkham
jakirkham / zarr_quick_demo.ipynb
Last active February 26, 2020 18:14
Zarr quick demo for lightning talk
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jakirkham
jakirkham / multi_arr_repro.py
Last active February 24, 2020 23:52
Attempt at repro for multi-array return
import sys
import dask
import dask.array
from dask.delayed import delayed
import distributed
try:
from dask_cuda import LocalCUDACluster as DaskCluster
@jakirkham
jakirkham / dask-cudf_multi_partition_merge.ipynb
Last active February 21, 2020 00:07
dask-cudf_multi_partition_merge
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.