Skip to content

Instantly share code, notes, and snippets.

@jakirkham
jakirkham / dask_xgboost_higgs.ipynb
Created Jun 16, 2022
Notebook loading Higgs dataset for use with Dask + XGBoost
View dask_xgboost_higgs.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View scheduler.c
This file has been truncated, but you can view the full file.
/* Generated by Cython 0.29.21 */
#define PY_SSIZE_T_CLEAN
#include "Python.h"
#ifndef Py_PYTHON_H
#error Python headers needed to compile C extensions, please install development version of Python.
#elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000)
#error Cython requires Python 2.6+ or Python 3.3+.
#else
#define CYTHON_ABI "0_29_21"
@jakirkham
jakirkham / trace.py
Created Jan 29, 2021 — forked from jcrist/trace.py
Dask Execution Tracer
View trace.py
import os
from dask.callbacks import Callback
from dask.dot import dot_graph
class Track(Callback):
def __init__(self, path='dasks', save_every=1):
self.path = path
self.save_every = save_every
@jakirkham
jakirkham / Miniconda_CPython_RAPIDS_install_exp.txt
Created Mar 19, 2020
Some experiments installing RAPIDS using Conda with CPython and PyPy respectively
@jakirkham
jakirkham / dask-report-rmm_suballoc-tcp.html
Created Mar 10, 2020
Dask Reports using RMM Suballocator
View dask-report-rmm_suballoc-tcp.html
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@jakirkham
jakirkham / meta.yaml
Last active Mar 5, 2020
Hacky workaround for CPython 3.6 ABI issue
View meta.yaml
package:
name: blah
version: 1
requirements:
host:
- python
- python_abi * *_cp36m # [py36]
run:
- python
@jakirkham
jakirkham / dask-report.html
Created Mar 3, 2020
Dask/Distributed GH# 932 (Bandwidth during shuffles)
View dask-report.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@jakirkham
jakirkham / zarr_quick_demo.ipynb
Last active Feb 26, 2020
Zarr quick demo for lightning talk
View zarr_quick_demo.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jakirkham
jakirkham / multi_arr_repro.py
Last active Feb 24, 2020
Attempt at repro for multi-array return
View multi_arr_repro.py
import sys
import dask
import dask.array
from dask.delayed import delayed
import distributed
try:
from dask_cuda import LocalCUDACluster as DaskCluster
@jakirkham
jakirkham / dask-cudf_multi_partition_merge.ipynb
Last active Feb 21, 2020
dask-cudf_multi_partition_merge
View dask-cudf_multi_partition_merge.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.