Skip to content

Instantly share code, notes, and snippets.

@tessro
Created May 8, 2012 21:31
Show Gist options
  • Save tessro/2639501 to your computer and use it in GitHub Desktop.
Save tessro/2639501 to your computer and use it in GitHub Desktop.
#!/usr/bin/env ruby
#
# gzsplit - split a file into gzipped pieces
filename = ARGV[0]
prefix = ARGV[1] || "part"
line_number = 0
chunk_size = 500_000
next_part_number = 0
gzip = nil
IO.popen(filename) do |line|
if line_number % chunk_size == 0
gzip = IO.popen("gzip > #{prefix}-#{'%04i' % next_part_number}.gz", "w")
next_part_number += 1
end
gzip.puts line
line_number += 1
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment