Skip to content

Instantly share code, notes, and snippets.

@BitOfUniverse
Created October 23, 2019 12:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save BitOfUniverse/d5207a28adee262a4ec531c03d61ca58 to your computer and use it in GitHub Desktop.
Save BitOfUniverse/d5207a28adee262a4ec531c03d61ca58 to your computer and use it in GitHub Desktop.
Consistent splitting using hashing and modulo
require 'zlib'
six_items = ["item_1", "item_2", "item_3", "item_4", "item_5", "item_6"]
ten_items = ["item_1", "item_2", "item_3", "item_4", "item_5", "item_6", "item_7", "item_8", "item_9", "item_10"]
chunks = 3 # should be set once and stay constant to preserve item<=>feed stickiness
six_items.each { |item_id| puts "#{item_id} goes to feed ##{Zlib.crc32(item_id) % chunks}" }
ten_items.each { |item_id| puts "#{item_id} goes to feed ##{Zlib.crc32(item_id) % chunks}" }
ten_items.reverse.each { |item_id| puts "#{item_id} goes to feed ##{Zlib.crc32(item_id) % chunks}" }
# -----------------------------------
irb(main):008:0> six_items.each { |item_id| puts "#{item_id} goes to feed ##{Zlib.crc32(item_id) % chunks}" }
item_1 goes to feed #0
item_2 goes to feed #1
item_3 goes to feed #0
item_4 goes to feed #1
item_5 goes to feed #2
item_6 goes to feed #0
=> ["item_1", "item_2", "item_3", "item_4", "item_5", "item_6"]
# order stays the same for larger number of items
irb(main):009:0> ten_items.each { |item_id| puts "#{item_id} goes to feed ##{Zlib.crc32(item_id) % chunks}" }
item_1 goes to feed #0
item_2 goes to feed #1
item_3 goes to feed #0
item_4 goes to feed #1
item_5 goes to feed #2
item_6 goes to feed #0
item_7 goes to feed #0
item_8 goes to feed #1
item_9 goes to feed #0
item_10 goes to feed #2
=> ["item_1", "item_2", "item_3", "item_4", "item_5", "item_6", "item_7", "item_8", "item_9", "item_10"]
# order stays the same for reversed list of items
irb(main):010:0> ten_items.reverse.each { |item_id| puts "#{item_id} goes to feed ##{Zlib.crc32(item_id) % chunks}" }
item_10 goes to feed #2
item_9 goes to feed #0
item_8 goes to feed #1
item_7 goes to feed #0
item_6 goes to feed #0
item_5 goes to feed #2
item_4 goes to feed #1
item_3 goes to feed #0
item_2 goes to feed #1
item_1 goes to feed #0
=> ["item_10", "item_9", "item_8", "item_7", "item_6", "item_5", "item_4", "item_3", "item_2", "item_1"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment