Skip to content

Instantly share code, notes, and snippets.

@jordan-brough
Last active August 29, 2015 14:06
Show Gist options
  • Save jordan-brough/5ac0363722cafee81fc0 to your computer and use it in GitHub Desktop.
Save jordan-brough/5ac0363722cafee81fc0 to your computer and use it in GitHub Desktop.
Ruby parellel processing and inter-process messaging with fork
class ParallelProcessor
attr_accessor :pid_params
def initialize
@pid_params = {}
end
def spawn_worker(start, finish, writer)
pid = fork do
puts "worker working on #{start} to #{finish}"
(start..finish).each do |i|
sleep rand(0.5) # do some work
puts "finished #{i}"
writer.puts
end
puts "#{start}-#{finish} finished!"
end
puts "started worker for #{start} to #{finish}"
pid_params[pid] = [start, finish]
end
def run
reader, writer = IO.pipe
stuff_to_do = [ [1,10], [11,20], [21,30], [31,40], [41,50], [51,60], [61,70], [71,80], [81,90], [91,100] ]
fork do
completed = 0
puts "progress bar process started"
writer.close
while !reader.eof?
reader.readline
completed += 1
puts "finished #{completed} of 100 parts"
end
end
reader.close
running_workers = 0
while !stuff_to_do.empty?
start, finish = stuff_to_do.shift
spawn_worker(start, finish, writer)
running_workers += 1
if running_workers >= 2
pid = Process.wait
puts "child #{pid} finished doing #{pid_params[pid]}"
running_workers -= 1
end
end
end
end
ParallelProcessor.new.run
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment