Last active
January 12, 2024 18:19
-
-
Save niger-prequel/040d7ccd53ddc840d3bc28de6b3fe011 to your computer and use it in GitHub Desktop.
The Duck(DB) Feather in Your Parquet Cap
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-- Read Parquet File: | |
SELECT * FROM read_parquet('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet') LIMIT 100; | |
-- Read Parquet Metadata | |
SELECT * FROM parquet_metadata('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet'); | |
-- Read Parquet Schema | |
SELECT * FROM parquet_schema('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet'); | |
-- Identify columns with decimals that have too much scale for our destination | |
SELECT | |
name | |
FROM | |
parquet_schema('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet') | |
WHERE | |
converted_type = 'DECIMAL' | |
AND | |
scale > 1; | |
-- Normalize decimal columns via rounding strategy | |
SELECT | |
test_decimal, | |
CAST(ROUND(test_decimal, 1) AS DECIMAL(38, 1)) as normalized_test_decimal | |
FROM | |
read_parquet('https://raw.githubusercontent.com/prequel-co/Scratch/main/mock.parquet') | |
LIMIT 100; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment