Skip to content

Instantly share code, notes, and snippets.

@ohnishiakira
Last active November 22, 2016 14:57
Show Gist options
  • Save ohnishiakira/f69e76d22078ce2a83fbfdab0a004c0e to your computer and use it in GitHub Desktop.
Save ohnishiakira/f69e76d22078ce2a83fbfdab0a004c0e to your computer and use it in GitHub Desktop.
#!/usr/bin/env ruby
require "fileutils"
class ScatteredFile
attr_writer :filelist
def initialize(file_template)
@file_template = file_template
@filelist = []
end
def write(file_key, log)
filename = @file_template % file_key
f = @filelist.find {|f| f.path == filename }
@filelist.reject! {|f| f.path == filename }
if f.nil?
begin
f = File.new(filename, "a+")
rescue Errno::EMFILE => e
@filelist.delete_at(0)&.close
retry
end
end
f.puts log
@filelist << f
end
def close
@filelist.map{|f| f.close }
end
end
FileUtils.mkdir_p "./tmp"
scatterd_file = ScatteredFile.new "tmp/tmp_%05d.txt"
(1...10000).each {|i|
file_key = i
scatterd_file.write file_key, i
}
scatterd_file.close
$ ruby scattered.rb
$ ls -1 tmp/
tmp_00000.txt
tmp_00001.txt
tmp_00002.txt
tmp_00003.txt
tmp_00004.txt
...
tmp_09995.txt
tmp_09996.txt
tmp_09997.txt
tmp_09998.txt
tmp_09999.txt
$ cat tmp/tmp_09999.txt
9999
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment