Skip to content

Instantly share code, notes, and snippets.

@staticor
Created December 28, 2018 08:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save staticor/6664ac843daf910df0e0a3bcbc9d4647 to your computer and use it in GitHub Desktop.
Save staticor/6664ac843daf910df0e0a3bcbc9d4647 to your computer and use it in GitHub Desktop.
sqoop example for import and export data
function sqoop_import_data() {
sqoop import -D mapreduce.job.queuename=mapreduce.normal \
--connect 'jdbc:mysql://123.123.123.123/optimus?useUnicode=true&characterEncoding=GBK' \
--username data \
--password 'password' \
--query " SELECT '' as id,
day,
agent,
register_name,
adv_id,
......
......
.....
.....
AND \$CONDITIONS
ORDER BY p.name,
adv_id,
platform) hl
" \
-m 1 --target-dir ${hdfs_input} \
--fields-terminated-by '\t'
}
sqoop_import_data;
# #############################################################################
table_export=checklist_advertiser_audit
function sqoop_export_data() {
sqoop export -D mapreduce.job.queuename=mapreduce.normal \
--connect 'jdbc:mysql://123.123.123.123/monitor?useUnicode=true&characterEncoding=GBK' \
--username data --password 'password' \
-m 1 --table ${table_export} --export-dir ${hdfs_input} \
--input-fields-terminated-by '\t'
}
sqoop_export_data;
echo 'data transfered, boom.'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment