Packing a bunch of ids into an encrypted string for use in a URL -- just some experiments. Driven by https://bibwild.wordpress.com/2013/01/07/crazy-use-of-encryption-to-protect-refworks-callback-urls/

gistfile1.rb

# Packing a bunch of ids into an encrypted string for use in a URL -- just some experiments. 
# Driven by https://bibwild.wordpress.com/2013/01/07/crazy-use-of-encryption-to-protect-refworks-callback-urls/

# A quick experiment to look at how many ids of various types we can crust into a 
# encrypted string. Obviously we could be more particular about how we pack them in
# if we know the characteristics of the IDs ahead of time.


require 'stringio'
require 'zlib'
require 'random'
require 'encryptor' # not part of stdlib; will have to install the gem
require 'base64'

# Generate a random list of IDs of the given length
# so we have something to mess with. This is just a simple,
# comma-delimited list of ids; we're not doing anything smart at all.

def idlist(size = 100, idlength=9, allow_alpha=false)
  a = {}
  r = Random.new # re-seed
  if allow_alpha
    anum = [(0..9).to_a, ('A'..'Z').to_a, ('a'..'z').to_a].flatten
  else
    anum = [0,1,2,3,4,5,6,7,8,9]
  end

  while a.size < size
    x = []
    idlength.times do
      x << anum[r.rand(anum.size)]
    end
    a[x.join('')] = 1
  end  
  
  return a.keys.join(',')
end  
 

# Simple helpers to do gzipping
def gz(str)
  s = StringIO.new  
  gz = Zlib::GzipWriter.new(s)  
  gz.write str
  gz.close  
  return s.string
end  

# ...and encryption
@secret_key = Digest::SHA256.hexdigest('generating the secret key with this text')
def ec(str)
  el = Encryptor.encrypt(str, :key=>@secret_key)  
  return el
end   


# Pull it all together: gzip, then encrypt, then base64 encode
def bstring(str)
  Base64.encode64(ec(gz(str)))
end


# Wrap it up nicely
def composed_list(size, idlength, allow_alpha=false)
  return bstring(idlist(size, idlength, allow_alpha))
end

# Reverse the process to get the original list back again
def decomposed_list(str)
  unb64 = Base64.decode64(str)
  uncrypt = Encryptor.decrypt(unb64, :key=>@secret_key)
  return Zlib::GzipReader.new(StringIO.new(uncrypt)).read
end


# How big are they? We need to know how many bytes these things take up so we know
# roughly how many ids we can stuff into a URL. The size is affected most significantly
# by how efficiently gzip can compress the list; you see this in the differences between
# the straight-numeric and alpha-numeric IDs.
def compare_sizes(sizes_to_compare = [10, 25, 50, 75, 100, 125, 150, 175,200, 225, 250])
  puts "\nSizes (in bytes) for a list of ids of various lengths, numeric-only or alpha-numeric"
  puts "after gzip/encrypt/base64 encoding (i.e., ready to include in a URL)"
  puts '\n%3s %5s %5s %5s %7s %7s %7s' % ['IDs', '8num', '9num', '10num', '8alpha', '9alpha', '10alpha']
  sizes_to_compare.each do |ids|
    a = [ids]
    a << composed_list(ids, 8).size
    a << composed_list(ids, 9).size
    a << composed_list(ids, 10).size
    a << composed_list(ids, 8, true).size
    a << composed_list(ids, 9, true).size
    a << composed_list(ids, 10, true).size
    puts '%3d %5d %5d %5d %7d %7d %7d' % a
  end
  puts "\n"
end  




 

gistfile2.txt

compare_sizes()

Sizes (in bytes) for a list of ids of various lengths, numeric-only or alpha-numeric
after gzip/encrypt/base64 encoding (i.e., ready to include in a URL)

IDs  8num  9num 10num  8alpha  9alpha 10alpha
 10   110   131   131     155     175     196
 25   196   220   240     305     326     350
 50   350   391   415     521     566     631
 75   501   566   610     741     826     891
100   651   716   806     956    1066    1172
125   826   911  1001    1196    1326    1432
150   956  1066  1172    1412    1562    1717
175  1131  1261  1367    1627    1822    1997
200  1281  1432  1586    1867    2062    2278
225  1456  1607  1782    2083    2323    2562
250  1607  1782  1952    2302    2562    2823