Skip to content

Instantly share code, notes, and snippets.

View GenevieveBuckley's full-sized avatar

Genevieve Buckley GenevieveBuckley

  • Monash University
  • Melbourne
View GitHub Profile
@GenevieveBuckley
GenevieveBuckley / 2021.ipynb
Created September 6, 2021 07:21
Dask survey 2021 analysis
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@GenevieveBuckley
GenevieveBuckley / correct_tensordot_auto_rechunkiing.py
Last active July 27, 2021 09:47
Tensordot auto-rechunking experiments
import dask
import dask.array as da
import numpy as np
def _inner_axes(a_ndim, b_ndim, axes):
"""Given tensordot axes argument, return list of axes to sum over."""
if isinstance(axes, (int, float)):
if axes == 0:
inner_axes_a = []
def _tensordot_shape_output(a, b, axes):
if isinstance(axes, (int, float)):
if axes == 0:
shape_out = a.shape + b.shape
chunks_out = a.chunks + b.chunks
elif axes > 0:
shape_out = a.shape[:axes-1] + b.shape[axes-1:]
chunks_out = a.chunks[:axes-1] + b.chunks[axes-1:]
else:
axes_a, axes_b = axes
@GenevieveBuckley
GenevieveBuckley / alternative_to_scipy_ndi_findobjects.py
Last active July 23, 2021 06:51
Slightly simpler way to get the array location for any dask block
import numpy as np
import pandas as pd
def _find_slices(x):
"""An alternative to scipy.ndi.find_objects"""
unique_vals = np.unique(x)
unique_vals = unique_vals[unique_vals != 0]
result = {}
for val in unique_vals:
@GenevieveBuckley
GenevieveBuckley / map-overlap-length-without-materializing.ipynb
Created June 28, 2021 13:27
map-overlap-length-without-materializing-the-task-graph
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@GenevieveBuckley
GenevieveBuckley / combine_slicing.py
Last active June 28, 2021 12:58
combine_slicing.py
import math
import numpy as np
import pytest
def combine_slices(slices):
starts = [s.start for s in slices if s.start is not None]
stops = [s.stop for s in slices if s.stop is not None]
steps = [s.step for s in slices if s.step is not None]
@GenevieveBuckley
GenevieveBuckley / plot.py
Created June 23, 2021 08:15
Simple matplotlib line plot with error bars
import matplotlib.pyplot as plt
%matplotlib inline
plt.errorbar(
[1,2,3],
[1,2,3],
yerr=[.1, .2, .3],
color='blue', marker='o', linestyle='dashed',
)
plt.title("Example")
@GenevieveBuckley
GenevieveBuckley / highlevelgraph-html.py
Last active June 4, 2021 09:19
HighLevelGraph HTML
from html import escape
class HighLevelGraphHTML():
def __init__(self, highlevelgraph):
self.highlevelgraph = highlevelgraph
pass
def _repr_html_(self):
highlevelgraph = self.highlevelgraph
@GenevieveBuckley
GenevieveBuckley / Dask-task-graph-handling-costs-on-the-client.ipynb
Last active June 1, 2021 06:49
Dask task graph handling costs on the client
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@GenevieveBuckley
GenevieveBuckley / distributed-skeleton-analysis.ipynb
Last active May 6, 2021 10:04
distributed skeleton analysis
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.