Skip to content

Instantly share code, notes, and snippets.



Forked from anonymous/
Created Dec 19, 2012
What would you like to do?

so, for the below script, we store the values in a text file that we read into mysql.

we then have 1000 unique URLs per row in a table, separated by a colon. We read and split one row at a time into a redis set, then pop values off the set whenever we need a new url. When the set is empty, we read the next row from mysql into the set.

The only thing to look out for is your collation in the db. A lookup is utf8_ci (case-insnsitive) for queries unless you set your collation to utf8_bin

collation-server = utf8_bin
init-connect='SET NAMES utf8'
init_connect='SET collation_connection = utf8_bin'
character-set-server = utf8
##This script creates hashes
# it must be run from `rails console` because of
# the, which is a rails thing.
# After this script is run, make the following sql call:
# load data infile '/tmp/perm_block.out' into table codepen.hash_seeds FIELDS TERMINATED BY ',' (hash_value,created_at,updated_at) SET id = NULL;
# The output of this call produces 60,233,040 unique hashes. At 100k growth per month, this should last us 55 years.
puts 'gather and shuffle.'
perms = (('a'..'z').to_a + ('A'..'L').to_a).permutation(5).to_a
puts 'shuffled'
open('/tmp/perm_block.out', 'w') do |f|
x = 0
s = []
count = 1
perms.each do |p|
s << p.to_a.join
x = x + 1
if x == 1000
puts "writing #{count} of #{(perms.count / 1000).to_i}"
f.puts [s.join(':'),,].join(',')
x = 0
count = count + 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment