Last active
December 17, 2015 09:58
-
-
Save noelwarr/5590840 to your computer and use it in GitHub Desktop.
Parrallel processing for ruby
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Use needle by telling it how many cpus you want to use | |
# and what you want it to do. | |
# Then just feed it data and watch all your cores go crazy. | |
# You can feed it standard ruby objects and it will return all | |
# results in the same order you gave them to it. | |
# n = Needle.new(cpus) {|input| input.do_something } | |
# result_of_all = n.sew[a,b,c,d,e,f,g,h,i...] | |
# uncomment the lines at the bottom to run the test | |
class Needle | |
def initialize(max_forks, &block) | |
@block = block | |
@max_forks = 3 | |
@map = Hash.new | |
@output = Array.new | |
end | |
def sew(input) | |
processes = 0 | |
input.each_with_index {|arg, i| | |
puts (i - @output.length) | |
wait if (i - @output.length) >= @max_forks | |
read, write = IO.pipe | |
pid = Process.fork { | |
write.puts Marshal.dump(@block.call(arg),write) | |
exit!(0) | |
} | |
processes += 1 | |
@map[pid] = {read: read, write: write, index: i} | |
} | |
wait until (@map.length == 0) | |
@output | |
end | |
def wait | |
pid = Process.wait | |
process = @map[pid] | |
process[:write].close | |
@output[process[:index]] = Marshal.load(process[:read].read) | |
process[:read].close | |
@map.delete(pid) | |
return process | |
end | |
end | |
n = Needle.new(4) {|input| | |
klass = input.class.to_s | |
permutation = String.new | |
100000.times { | |
permutation = klass.split(//).permutation.to_a | |
} | |
i = (rand * permutation.length).to_i | |
"a #{klass} was recieved which has #{permutation.length} permutations of which number #{i} is #{permutation[i]}" | |
} | |
arguments = [Array.new, Hash.new, :symbol, 100, 0.1, ] | |
puts n.sew(arguments) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment