Skip to content

Instantly share code, notes, and snippets.

@dgadiraju
Created January 23, 2019 23:48
Show Gist options
  • Save dgadiraju/085ee66867a891108e91cd092dc0f90b to your computer and use it in GitHub Desktop.
Save dgadiraju/085ee66867a891108e91cd092dc0f90b to your computer and use it in GitHub Desktop.
# This command fails on largedeck.txt as the block size of that file is 64 MB
hadoop distcp \
hdfs://nameservice1/user/itversity/cards \
hdfs://nameservice1/user/itversity/cards1
hadoop fs -ls /user/itversity/cards1
hadoop distcp \
-update \
-skipcrccheck \
hdfs://nameservice1/user/itversity/cards \
hdfs://nameservice1/user/itversity/cards1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment