robcowie/hdfs_usage_notes.md

## hdfs_usage_notes.md

      
    Raw
  

              hdfs_usage_notes.md
            
          
    HDFS Notes

Moving files to and from HDFS

hdfs dfs -copyFromLocal file hdfs://path/to/dir/file
hdfs dfs -copyToLocal hdfs://path/to/dir/file file

Moving files within the cluster


Single file

hdfs dfs -mv hdfs://path/to/dirA/file hdfs://path/to/dirB/file


With wildcard

hdfs dfs -mv "hdfs://path/to/dirA/*" hdfs://path/to/dirB/
hdfs dfs -mv "hdfs://path/to/dirA/*.0" hdfs://path/to/dirB/

Listing

hdfs dfs -ls hdfs://path/to/dir/
hdfs dfs -ls hdfs://path/to/dir/*.0
hdfs dfs -ls -R hdfs://path/to/dir/

Get file contents

hdfs dfs -cat hdfs://path/to/dir/file
hdfs dfs -cat hdfs://path/to/dir/file | pbcopy (macos)