Skip to content

Instantly share code, notes, and snippets.

/explain.md

Created Dec 19, 2012
Embed
What would you like to do?

so, for the above script, we store the values in a text file that we read into mysql.

we then have 1000 unique URLs in a table. We read one row at a time into a redis set, then pop values off the set whenever we need a new url. When the set is empty, we read the next row from mysql into the set.

The only thing to look out for is your collation in the db. A lookup is utf8_ci (case-insnsitive) for queries unless you set your collation to utf8_bin

collation-server = utf8_bin
init-connect='SET NAMES utf8'
init_connect='SET collation_connection = utf8_bin'
skip-character-set-client-handshake
character-set-server = utf8, so you have to set that up in your db.
##This script creates hashes
# it must be run from `rails console` because of
# the Time.now.to_s(:db), which is a rails thing.
#
# After this script is run, make the following sql call:
# load data infile '/tmp/perm_block.out' into table codepen.hash_seeds FIELDS TERMINATED BY ',' (hash_value,created_at,updated_at) SET id = NULL;
#
# The output of this call produces 60,233,040 unique hashes. At 100k growth per month, this should last us 55 years.
puts 'gather and shuffle.'
perms = (('a'..'z').to_a + ('A'..'L').to_a).permutation(5).to_a
perms.shuffle!
puts 'shuffled'
open('/tmp/perm_block.out', 'w') do |f|
x = 0
s = []
count = 1
perms.each do |p|
s << p.to_a.join
x = x + 1
if x == 1000
puts "writing #{count} of #{(perms.count / 1000).to_i}"
f.puts [s.join(':'), Time.now.to_s(:db), Time.now.to_s(:db)].join(',')
x = 0
count = count + 1
s.clear
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment