Skip to content

Instantly share code, notes, and snippets.

View grantmwilliams's full-sized avatar

Grant Williams grantmwilliams

View GitHub Profile
@grantmwilliams
grantmwilliams / df_plot_example.ipynb
Created March 25, 2022 18:41
Example showing common plotting problems with bar charts in pandas data frames
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@grantmwilliams
grantmwilliams / iter_parquet.py
Created June 27, 2021 12:45
Pyarrow iter_batches as python native iterable
import s3fs
import pyarrow as pa
import pyarrow.parquet as pq
from itertools import chain
from typing import Tuple, Any
def iter_parquet(s3_uri: str, columns = None, batch_size=1_000) -> Tuple[Any]: