Skip to content

Instantly share code, notes, and snippets.

@ricardochimal
Created August 7, 2009 01:39
Show Gist options
  • Save ricardochimal/163663 to your computer and use it in GitHub Desktop.
Save ricardochimal/163663 to your computer and use it in GitHub Desktop.
def cmd_receive_data
puts "Receiving data"
tables_with_counts, record_count = fetch_remote_tables_info
puts "#{tables_with_counts.size} tables, #{format_number(record_count)} records"
tables_with_counts.each do |table_name, count|
table = db[table_name.to_sym]
chunksize = default_chunksize
progress = ProgressBar.new(table_name.to_s, count)
offset = 0
loop do
begin
chunksize, rows = fetch_table_rows(table_name, chunksize, offset)
rescue CorruptedData
next
end
break if rows == { }
# debug stuff
File.open("/tmp/taps.debug", "w") { |f| f.write(Marshal.dump(rows)) }
table.import(rows[:header], rows[:data])
progress.inc(rows[:data].size)
offset += rows[:data].size
end
progress.finish
end
end
irb
>> data = Marshal.load(File.read("/tmp/taps.debug"))
>> # look over the data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment