Skip to content

Instantly share code, notes, and snippets.

@disciplezero
Last active August 29, 2015 14:19
Show Gist options
  • Save disciplezero/141f34dbe687559c16c4 to your computer and use it in GitHub Desktop.
Save disciplezero/141f34dbe687559c16c4 to your computer and use it in GitHub Desktop.
convertTable from mysql to redshift
#!/bin/bash
RDSHOST=$1
RDSUSER=$2
RDSPASS=$3
RDSDB=$4
PGHOST=$5
PGUSER=$6
PGPASS=$8
PGDB=$9
OUTPREFIX=$10
TABLE=$11
# https://gist.github.com/robertobarreda/ede01dd9bfb5badec8af#file-mysql_to_redshift-py
CONVERTER=helpers/mysql_to_redshift.py
TMP=/tmp
mkdir -p $TMP
MySQLFile=$TMP/my-$TABLE-$(date +%m%d%H%M%S).psql
echo "Mysql input: $MySQLFile"
mysqldump -h $RDSHOST -u $RDSUSER --password=$RDSPASS --compatible=postgresql --default-character-set=utf8 --skip-triggers -n -d $RDSDB $1 > $MySQLFile
RedFile=$TMP/red-$TABLE-$(date +%m%d%H%M%S).psql
echo "Redfile output: $RedFile"
python $CONVERTER --input_file=$MySQLFile --output_file=$RedFile --table_name=$OUTPREFIX$TABLE --insert_mode=OVERWRITE_EXISTING
export PGPASSWORD=$PGPASS
psql -h $PGDB -U $PGUSER -d $PGDB -p 5439 -f $RedFile
rm -f $MySQLFile
rm -f $RedFile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment