Skip to content

Instantly share code, notes, and snippets.

@ayushdg
ayushdg / Numba_cuda.ipynb
Created July 29, 2019 04:42
Small Numba examples
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ayushdg
ayushdg / gby-sort-perf.ipynb
Last active September 4, 2019 03:33
Groupby sort performance
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ayushdg
ayushdg / dask-git-bisect.md
Last active October 10, 2019 22:24
Steps to git-bisect

Git bisect to identify bad commit (here dask is used as an example)

Prerequisites to git bisect:

  • Know how to build dask from source
  • Known good commit eg: Dask version 2.4
  • Known bad commit eg: Dask version 2.5
  • Have a script/test that fails with bad commit but works with good commit
  • Ensure that the test indeed fails with dask built from source with bad commit and passes for dask built from source with good commit. (In the existing environment)
import dask_xgboost as dxgb_gpu
import dask
import dask_cudf
from dask_cuda import LocalCUDACluster
from dask.delayed import delayed
from dask.distributed import Client, wait
import cudf
import numpy as np
import xgboost as xgb
@ayushdg
ayushdg / Groupby_sort_test.ipynb
Created February 11, 2020 01:51
Groupby sort test
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ayushdg
ayushdg / coalesce.py
Created April 30, 2020 19:46
Concat partitions within worker
def concat_dfs(df_list):
"""
Concat a list of cudf dataframes
"""
return cudf.concat(df_list)
def get_delayed_dict(ddf):
"""
Returns a dicitionary with the dataframe tasks as keys and
the dataframe delayed objects as values
@ayushdg
ayushdg / Orc_predicate_filters.ipynb
Last active January 13, 2021 20:33
Predicate-Pushdown
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ayushdg
ayushdg / Rapids_Prefect.ipynb
Last active January 21, 2021 21:24
Prefect Flow example with cuDF
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ayushdg
ayushdg / cudf_predicate_filtering.ipynb
Last active February 1, 2021 22:13
Predicate Filtering on Apache Parquet & ORC with cuDF, dask-cuDF
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.