Skip to content

Instantly share code, notes, and snippets.

@bradland
Last active August 8, 2018 02:33
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bradland/5568217 to your computer and use it in GitHub Desktop.
Save bradland/5568217 to your computer and use it in GitHub Desktop.
Net::SFTP test script. Fails when downloading files larger than 4 GB
#!/usr/bin/env ruby
require 'net/sftp'
@host = 'localhost'
@user = 'username'
@password = 'password'
@remote_dir = '/source/path/goes/here'
@remote_file_name = 'test-data.txt'
@tmp_dir = '/destination/path/goes/here'
# @read_size = 64000
@read_size = 1048576 # SSH won't send more than 64K (65536 bytes) anyway
Net::SFTP.start(@host, @user, password: @password) do |sftp|
remote_path = File.join(@remote_dir, @remote_file_name)
extract_path = File.join(@tmp_dir, @remote_file_name)
sftp.download(remote_path, extract_path, read_size: @read_size) do |event, downloader, *args|
case event
when :open then
# args[0] : file metadata
puts "Starting download: #{remote_path}"
when :get then
# args[0] : file metadata
# args[1] : byte offset in remote file
# args[2] : data that was received
puts "writing #{args[2].length} bytes to #{args[0].local} starting at #{args[1]}"
when :finish then
puts "Completed download of #{remote_path}"
end
end
end
puts extract_path
writing 65536 bytes to /Users/bradland/Desktop/test-data.tar starting at 4294049792
^C/Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-ssh-2.6.7/lib/net/ssh/ruby_compat.rb:30:in `select': Interrupt
from /Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-ssh-2.6.7/lib/net/ssh/ruby_compat.rb:30:in `io_select'
from /Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:204:in `process'
from /Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `block in loop'
from /Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `loop'
from /Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `loop'
from /Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:802:in `loop'
from /Users/bradland/.rvm/gems/ruby-1.9.3-p327@testing/gems/net-sftp-2.1.2/lib/net/sftp.rb:35:in `start'
from ./sftp.rb:15:in `<main>'
(gdb) where
#0 0x00007fff8e50c322 in select$DARWIN_EXTSN ()
#1 0x000000010f49f5f7 in do_select [inlined] () at /Users/bradland/.rvm/src/ruby-1.9.3-p392/thread.c:2433
#2 0x000000010f49f5f7 in rb_thread_fd_select (max=8, read=0x7fff508f4840, write=0x7fff508f4850, except=0x0, timeout=0x0) at thread.c:2775
#3 0x000000010f378f3f in select_internal [inlined] () at /Users/bradland/.rvm/src/ruby-1.9.3-p392/io.c:7658
#4 0x000000010f378f3f in select_call (arg=<value temporarily unavailable, due to optimizations>) at io.c:7728
#5 0x000000010f34f5df in rb_ensure (b_proc=0x10f378b30 <select_call>, data1=140734544955424, e_proc=0x10f371890 <select_end>, data2=140734544955424) at eval.c:744
#6 0x000000010f371ade in rb_f_select (argc=<value temporarily unavailable, due to optimizations>, argv=<value temporarily unavailable, due to optimizations>, obj=<value temporarily unavailable, due to optimizations>) at io.c:7982
#7 0x000000010f493127 in vm_call_cfunc [inlined] () at /Users/bradland/.rvm/src/ruby-1.9.3-p392/vm_insnhelper.c:404
#8 0x000000010f493127 in vm_call_method (th=0x7f85eb403f10, cfp=0x10f751c38, num=4, blockptr=0x1, flag=2, id=<value temporarily unavailable, due to optimizations>, me=0x7f85eb459dd0, recv=140213158871120) at vm_insnhelper.c:530
#9 0x000000010f480545 in vm_exec_core (th=0x7f85eb403f10, initial=<value temporarily unavailable, due to optimizations>) at insns.def:1018
#10 0x000000010f48517a in vm_exec (th=0x7f85eb403f10) at vm.c:1236
#11 0x000000010f495331 in invoke_block_from_c [inlined] () at /Users/bradland/.rvm/src/ruby-1.9.3-p392/vm.c:640
#12 0x000000010f495331 in vm_yield [inlined] () at vm.c:670
#13 0x000000010f495331 in rb_yield_0 [inlined] () at /Users/bradland/.rvm/src/ruby-1.9.3-p392/vm_eval.c:760
#14 0x000000010f495331 in loop_i () at vm.c:818
#15 0x000000010f34f9f7 in rb_rescue2 (b_proc=0x10f495100 <loop_i>, data1=0, r_proc=0, data2=0) at eval.c:647
#16 0x000000010f478556 in rb_f_loop (self=140213190176280) at vm_eval.c:846
#17 0x000000010f493127 in vm_call_cfunc [inlined] () at /Users/bradland/.rvm/src/ruby-1.9.3-p392/vm_insnhelper.c:404
#18 0x000000010f493127 in vm_call_method (th=0x7f85eb403f10, cfp=0x10f751df0, num=0, blockptr=0x10f751e19, flag=8, id=<value temporarily unavailable, due to optimizations>, me=0x7f85ecb956c0, recv=140213190176280) at vm_insnhelper.c:530
#19 0x000000010f480545 in vm_exec_core (th=0x7f85eb403f10, initial=<value temporarily unavailable, due to optimizations>) at insns.def:1018
#20 0x000000010f48517a in vm_exec (th=0x7f85eb403f10) at vm.c:1236
#21 0x000000010f48545c in rb_iseq_eval_main (iseqval=140213158601680) at vm.c:1478
#22 0x000000010f34fc62 in ruby_exec_internal (n=0x7f85eb82e3d0) at eval.c:204
#23 0x000000010f3527cc in ruby_exec_node [inlined] () at /Users/bradland/.rvm/src/ruby-1.9.3-p392/eval.c:251
#24 0x000000010f3527cc in ruby_run_node (n=<value temporarily unavailable, due to optimizations>) at eval.c:244
#25 0x000000010f30aedf in main (argc=3, argv=0x7fff508f57c8) at main.c:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment