Skip to content

Instantly share code, notes, and snippets.

@AeroCross
Created June 6, 2018 06:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AeroCross/c51c620c41cdf7252c6c198c4fff4b10 to your computer and use it in GitHub Desktop.
Save AeroCross/c51c620c41cdf7252c6c198c4fff4b10 to your computer and use it in GitHub Desktop.
Splitting files into multiple, smaller files
#!/usr/bin/env ruby
require 'pathname'
require 'csv'
if ARGV.empty? || ARGV[0].empty? || ARGV[1].empty?
puts 'Usage: split <file> <number-of-files-to-create>'
end
file = begin
Pathname.new(ARGV[0]).realpath
rescue Errno::ENOENT
raise "Path doesn't exist"
end
working_dir = file.dirname
rows = File.readlines(file)
number_of_files_to_create = ARGV[1].to_i
number_of_rows_per_file = (rows.length / number_of_files_to_create).floor
counter = 1;
puts "------------------------"
puts "Number of rows per file: #{number_of_rows_per_file}"
puts "Number of files to create: #{number_of_files_to_create}"
puts "------------------------"
# This will create an additional file if
# number_of_rows_per_file has a modulo of != 0
while rows.length > 0 do
output_file = CSV.open("#{working_dir}/#{counter}.csv", "w") do |csv|
input = rows.pop(number_of_rows_per_file)
input.each do |row|
csv << [row.to_s.strip!]
end
counter += 1
puts "Remaining: #{rows.length}"
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment