Skip to content

Instantly share code, notes, and snippets.

@arbovm
Created August 13, 2010 12:25
Show Gist options
  • Save arbovm/522781 to your computer and use it in GitHub Desktop.
Save arbovm/522781 to your computer and use it in GitHub Desktop.
#!/usr/bin/ruby
# to count the words of "The Complete Works of William Shakespeare", type
# $ curl http://www.gutenberg.org/cache/epub/100/pg100.txt | ./wordcount_map_reduce.rb
def map line
line.split(' ').each do |word|
emit(word, 1)
end
end
def reduce key, values
values.inject(0){|sum, value| sum += value}
end
def finalize key_value_rows
key_value_rows.sort_by{|key, value| value}.reverse
end
def format_output key, value
"#{value}\t#{key}"
end
KEY_TO_VALUES = {}
def emit key, value
KEY_TO_VALUES[key] ||= []
KEY_TO_VALUES[key] << value
end
ARGF.each do |line|
map(line)
end
key_value_rows = KEY_TO_VALUES.map do |key, values|
[key, reduce(key, values)]
end
key_value_rows = finalize(key_value_rows)
key_value_rows.each do |key, value|
puts format_output(key, value)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment