Skip to content

Instantly share code, notes, and snippets.

@samklr

samklr/parquet_tools.md

Last active January 22, 2021 19:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save samklr/218a73404c5c383adf02595db9b8ae7a to your computer and use it in GitHub Desktop.
Save samklr/218a73404c5c383adf02595db9b8ae7a to your computer and use it in GitHub Desktop.
Download ZIP
Parquet Tools
Raw
parquet_tools.md

Setup Parquet-tools brew install parquet-tools

Help parquet-tools -h

parquet-tools rowcount part-00000-fc34f237-c985-4ebc-822b-87fa446f6f70.c000.snappy.parquet

parquet-tools head -n 1 part-00000-fc34f237-c985-4ebc-822b-87fa446f6f70.c000.snappy.parquet

parquet-tools meta part-00000-fc34f237-c985-4ebc-822b-87fa446f6f70.c000.snappy.parquet

parquet-tools dump -m -c make part-00000-fc34f237-c985-4ebc-822b-87fa446f6f70.c000.snappy.parquet | head -20

parquet-tools cat --json hdfs://localhost/tmp/save/part-r-00000-6a3ccfae-5eb9-4a88-8ce8-b11b2644d5de.gz.parquet

Checkout https://github.com/chhantyal/parquet-cli

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment