Skip to content

Instantly share code, notes, and snippets.

@noelwarr
Last active December 17, 2015 09:58
Show Gist options
  • Save noelwarr/5590840 to your computer and use it in GitHub Desktop.
Save noelwarr/5590840 to your computer and use it in GitHub Desktop.
Parrallel processing for ruby
# Use needle by telling it how many cpus you want to use
# and what you want it to do.
# Then just feed it data and watch all your cores go crazy.
# You can feed it standard ruby objects and it will return all
# results in the same order you gave them to it.
# n = Needle.new(cpus) {|input| input.do_something }
# result_of_all = n.sew[a,b,c,d,e,f,g,h,i...]
# uncomment the lines at the bottom to run the test
class Needle
def initialize(max_forks, &block)
@block = block
@max_forks = 3
@map = Hash.new
@output = Array.new
end
def sew(input)
processes = 0
input.each_with_index {|arg, i|
puts (i - @output.length)
wait if (i - @output.length) >= @max_forks
read, write = IO.pipe
pid = Process.fork {
write.puts Marshal.dump(@block.call(arg),write)
exit!(0)
}
processes += 1
@map[pid] = {read: read, write: write, index: i}
}
wait until (@map.length == 0)
@output
end
def wait
pid = Process.wait
process = @map[pid]
process[:write].close
@output[process[:index]] = Marshal.load(process[:read].read)
process[:read].close
@map.delete(pid)
return process
end
end
n = Needle.new(4) {|input|
klass = input.class.to_s
permutation = String.new
100000.times {
permutation = klass.split(//).permutation.to_a
}
i = (rand * permutation.length).to_i
"a #{klass} was recieved which has #{permutation.length} permutations of which number #{i} is #{permutation[i]}"
}
arguments = [Array.new, Hash.new, :symbol, 100, 0.1, ]
puts n.sew(arguments)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment