Skip to content

Instantly share code, notes, and snippets.

Created November 22, 2010 20:24
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Simple way to do multi-threaded chunking in Ruby :
require 'thread'
# Stupid simple "multi-threading" - it doesn't use mutex or queues but
# it does have access to local variables, which is convenient. This will
# break a data set into equal slices and process them, but it is not
# perfect in that it will not start the next set until the first is
# completely processed -- so, if you have 1 slow item it loses benefit
# NOTE: this is not thread-safe!
class ThreadPool
def self.process!(data, size = 2, &block)
Array(data).each_slice(size) do |slice| do |item|{ }{ |t| t.join }
def initialize(size)
@size = size
def process!(data, &block)
self.class.process!(data, @size, &block)
# Playing around with it on the alphabet
# adjust the +size+ to adjust how many threads are
# being used at once
if $0 == __FILE__
require 'pp'
require 'benchmark'
size = 10
words = ('a'..'z').to_a
list = []
pool =
puts "Starting (P: %d W: %d)" % [size, words.size]
b = Benchmark.realtime{
pool.process!(words) do |word|
b = Benchmark.realtime{
list << word
sleep rand*2
puts "\t\tFinished: #{word} -- %0.2f seconds" % b
# Output some times
puts "Finished all: %0.2f seconds" % b
puts "\nList: %s\n\n" % list.join(', ')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment