Skip to content

Instantly share code, notes, and snippets.

@martindurant

martindurant/big.ipynb

Created May 14, 2023 16:45
Show Gist options
  • Save martindurant/513e38ea951ef96affc7927516322545 to your computer and use it in GitHub Desktop.
Save martindurant/513e38ea951ef96affc7927516322545 to your computer and use it in GitHub Desktop.
Download ZIP
parquet merger
Display the source blob
Display the rendered blob
Raw
big.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@martindurant
Copy link
Author

@rsignell-usgs - 40 years of references merged in 19min and about 1.5gb of memory.

@martindurant
Copy link
Author

(this will require some code pushed, merged and released for others to use)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment