Skip to content

Instantly share code, notes, and snippets.

@Shaunakde

Shaunakde/fast-rsync.md

Created November 30, 2020 21:21
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Shaunakde/ef615b58ebf9acdd7e8652f83dae3174 to your computer and use it in GitHub Desktop.
Save Shaunakde/ef615b58ebf9acdd7e8652f83dae3174 to your computer and use it in GitHub Desktop.
Download ZIP
Fast rsync from mac to linux
Raw
fast-rsync.md

This set of parameters experimentally seems to work the best for macos to linux file transfer. This is ofcourse not secure and is intended for transferring things like machine learning datasets to a workstation from a laptop etc.

Command

rsync -rltv --progress --human-readable --delete -e 'ssh -T -c aes128-gcm@openssh.com -o Compression=no -x'

Explaination

  • r: Recursive. Allow directories to be traversed and copied.
  • l: Links. Copy symlinks as symlinks and not what they point to.
  • t: Times. Preserve timestamps associated with the files.
  • v: Verbose. Print everything.
  • progress: Show the progress of the transfer
  • human-readable: Print file sizes with M, G etc. so its much easier to read and understand.
  • delete: Delete extraneous files from destination
  • The SSH command uses aes and turns off compression. Using arcfour would be faster, but many newer systems don't support it. There might be more optimization to be done here.
@Shaunakde
Copy link
Author

Inspired by: https://gist.github.com/KartikTalwar/4393116

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment