Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
database=smart_hub_data_catalog
tables=(smart_hub_locations_parquet sensor_mappings_parquet smart_hub_data_parquet etl_output_parquet)
for table in ${tables}; do
fixed_table=$(aws glue get-table \
--database-name "${database}" \
--name "${table}" \
| jq '.Table.Parameters.classification = "parquet" | del(.Table.DatabaseName) | del(.Table.CreateTime) | del(.Table.UpdateTime) | del(.Table.CreatedBy) | del(.Table.IsRegisteredWithLakeFormation)')
fixed_table=$(echo ${fixed_table} | jq .Table)
aws glue update-table \
--database-name "${database}" \
--table-input "${fixed_table}"
echo "table '${table}' classification changed to 'parquet'"
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment