Skip to content

Instantly share code, notes, and snippets.

@GusGA
Created December 8, 2013 23:04
Show Gist options
  • Save GusGA/7864946 to your computer and use it in GitHub Desktop.
Save GusGA/7864946 to your computer and use it in GitHub Desktop.
Mapper & Reducer in ruby
# Ruby code for map.rb
ARGF.each do |line|
# remove any newline
line = line.chomp
# do nothing will lines shorter than 2 characters
next if ! line || line.length < 2
# grab our key as the two-character prefix (lower-cased)
key = line[0,2].downcase
# value is a count of 1 occurence
value = 1
# output to STDOUT
# <key><tab><value><newline>
puts key + "\t" + value.to_s
end
# Ruby code for reduce.rb
prev_key = nil
key_total = 0
ARGF.each do |line|
# remove any newline
line = line.chomp
# split key and value on tab character
(key, value) = line.split(/\t/)
# check for new key
if prev_key && key != prev_key && key_total > 0
# output total for previous key
# <key><tab><value><newline>
puts prev_key + "\t" + key_total.to_s
# reset key total for new key
prev_key = key
key_total = 0
elsif ! prev_key
prev_key = key
end
# add to count for this current key
key_total += value.to_i
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment