Skip to content

Instantly share code, notes, and snippets.

@seunggabi

seunggabi/copy_hdfs.sh

Last active May 17, 2020 16:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save seunggabi/6b8b985b5a076e1b90a1eaf1ac8059c8 to your computer and use it in GitHub Desktop.
Save seunggabi/6b8b985b5a076e1b90a1eaf1ac8059c8 to your computer and use it in GitHub Desktop.
Download ZIP
copy_hdfs.sh
Raw
copy_hdfs.sh
#!/bin/sh
target=$1
source /home1/irteam/apps/${target}/source.me
user=$2
from=$3
to=$4
path="hdfs://${target}/user/${user}/${to}"
HADOOP_USER_NAME=${user} hdfs --config ${C3_CONF_DIR} dfs -rmr ${path}
HADOOP_USER_NAME=${user} hdfs --config ${C3_CONF_DIR} dfs -mkdir ${path}
HADOOP_USER_NAME=${user} hdfs --config ${C3_CONF_DIR} dfs -copyFromLocal ${from}/* ${path}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment