Skip to content

Instantly share code, notes, and snippets.

@archongum
Created August 27, 2019 07:29
Show Gist options
  • Save archongum/9a8843f3e867c34f354693cca73d20fa to your computer and use it in GitHub Desktop.
Save archongum/9a8843f3e867c34f354693cca73d20fa to your computer and use it in GitHub Desktop.
use presto to automatically sync hive partitions
#!/bin/bash
COMMAND_PREFIX="presto-cli --server host:8080 --catalog hive --user hive --execute "
SQL="select table_name from information_schema.tables where table_schema = 'dw' and table_type = 'BASE TABLE' and table_name like 'ods_%'"
GET_TABLES_COMMAND=$COMMAND_PREFIX"\""$SQL"\""
echo $GET_TABLES_COMMAND
for TABLE in `eval $GET_TABLES_COMMAND`
do
TABLE=`echo $TABLE | cut -d '"' -f 2`
SYNC_QUERY="call system.sync_partition_metadata('dw', '$TABLE', 'full')"
SYNC_QUERIES=$SYNC_QUERY";"$SYNC_QUERIES
done
SYNC_COMMAND=$COMMAND_PREFIX"\""$SYNC_QUERIES"\""
eval $SYNC_COMMAND
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment