Skip to content

Instantly share code, notes, and snippets.

View fjetter's full-sized avatar

Florian Jetter fjetter

29 followers · 7 following
View GitHub Profile
@fjetter
fjetter / distributed_spill_interface_draft.py
Last active April 30, 2021 09:56
A draft for a possible new multi-layered spilling interface for dask.distributed
import io
from collections import defaultdict
from typing import Callable, Dict, List, MutableMapping, Set
from dask.sizeof import sizeof
class Data:
async def open(self) -> io.IOBase:
return io.StringIO()
@fjetter
fjetter / gh4471.html
Created June 15, 2021 14:26
https://github.com/dask/distributed/pull/4920
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@fjetter
fjetter / gh4471_dataframe_shuffle_main.html
Created June 23, 2021 15:52
work-stealing-shuffle
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@fjetter
fjetter / gist:f695527168f845ce49b83ae925f3a8b7
Created June 24, 2021 17:26
GH 4962
We couldn’t find that file to show.
@fjetter
fjetter / gh4471_dataframe_shuffle_fixed_sizeof.html
Created June 24, 2021 17:27
GH 4962
This file has been truncated, but you can view the full file.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
@fjetter
fjetter / example_parquet_dask.ipynb
Created October 12, 2021 17:34
Parquet read+write example dask
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@fjetter
fjetter / conda list --export
Created March 31, 2022 12:12
https://github.com/dask/distributed/pull/6033
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-arm64
aiobotocore=2.1.2=pyhd8ed1ab_0
aiohttp=3.8.1=py310he2143c4_0
aioitertools=0.10.0=pyhd8ed1ab_0
aiosignal=1.2.0=pyhd8ed1ab_0
anyio=3.5.0=py310hbe9552e_0
appnope=0.1.2=py310hbe9552e_2
argon2-cffi=21.3.0=pyhd8ed1ab_0
@fjetter
fjetter / a8eb3b23b8fe91f52758db155e7151e3d516cbdc.html
Last active August 11, 2022 09:10
Stress test distributed CI rest report
<!DOCTYPE html>
<html>
<head>
<script src="https://cdn.jsdelivr.net/npm/vega@5"></script>
<script src="https://cdn.jsdelivr.net/npm/vega-lite@4.17.0"></script>
<script src="https://cdn.jsdelivr.net/npm/vega-embed@6"></script>
</head>
<body>
<div class="vega-visualization" id="vega-visualization-24ce1165c3754a298229aae3dac86213"></div>
@fjetter
fjetter / NestedCollectionsSerialization.ipynb
Last active October 26, 2023 15:30
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.