Skip to content

Instantly share code, notes, and snippets.

@sideshowcoder
Last active July 14, 2016 12:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sideshowcoder/1cab4589e0ac0abbf1536d66ae7e3b54 to your computer and use it in GitHub Desktop.
Save sideshowcoder/1cab4589e0ac0abbf1536d66ae7e3b54 to your computer and use it in GitHub Desktop.
Transfer a huge file over a flaky SCP connection
# On the server side split the core file into 50M chunks naming them core.0000000000 etc.
# $ split -a 10 -b 50M -d core core.
# reassamble on the other side via
# $ cat core.* > core
chunk_count = 538
root_name = "core." # name on server is 'core.00001 etc'
host = "myhost.com"
path = "/tmp"
target = "put-this-here"
(0..chunk_count).map do |n|
fname = sprintf "#{root_name}%010d", n # in my infinite wisdom I used 10 element long numeric chunks
begin
puts "downloading #{fname}"
`scp #{host}:#{path}/#{fname} ./#{target}/`
fail "download failed" unless $?.success? # SCP why are you dying on me?!
rescue
puts "retrying #{fname}"
retry
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment