Skip to content

Instantly share code, notes, and snippets.

@niger-prequel
Last active January 12, 2024 18:19
Show Gist options
  • Save niger-prequel/040d7ccd53ddc840d3bc28de6b3fe011 to your computer and use it in GitHub Desktop.
Save niger-prequel/040d7ccd53ddc840d3bc28de6b3fe011 to your computer and use it in GitHub Desktop.
The Duck(DB) Feather in Your Parquet Cap
-- Read Parquet File:
SELECT * FROM read_parquet('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet') LIMIT 100;
-- Read Parquet Metadata
SELECT * FROM parquet_metadata('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet');
-- Read Parquet Schema
SELECT * FROM parquet_schema('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet');
-- Identify columns with decimals that have too much scale for our destination
SELECT
name
FROM
parquet_schema('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet')
WHERE
converted_type = 'DECIMAL'
AND
scale > 1;
-- Normalize decimal columns via rounding strategy
SELECT
test_decimal,
CAST(ROUND(test_decimal, 1) AS DECIMAL(38, 1)) as normalized_test_decimal
FROM
read_parquet('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet')
LIMIT 100;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment