Ruby Redis benchmarks for JSON vs Marshal vs String and code examples

gistfile1.txt

Redis Marshaling

If you have time consuming method in your app that you call frequently, chances are that you’d like to cache it. Redis is great key-value store that can be used for this purpose. However, redis only store string values. In order to store other kinds of objects, you’ll have to use marshaling.

There is a built-in ruby class called Marshal that does the job well. The only thing you have to do is to call Marshal.dump method and pass it an object. The exact opposite is the Marshal.load method.

I’ve created helper method for easier integration with Redis. Here is how it looks:

module RedisHelper

  # Returns the cached value for given key.
  # If it doesn't exist, it evaluates the block and caches what it returns.
  # 
  # @param [String] key the cache key
  # @param [Hash] options
  # @option options [Integer, nil] :ttl time to live in seconds
  # @option options [Boolean, false] :marshal whether to perform marshaling
  # @yield [] to be evaluated if the key does not exist
  def self.fetch(key, options={}, &block)
    raise ArgumentError.new("Must Supply Block") unless block_given?
    return yield unless Rails.application.config.action_controller.perform_caching

    val = nil

    if REDIS.exists(key)
      val = REDIS.get(key)
      begin
        val = Marshal.load(val) if options[:marshal]
      rescue TypeError
        val = nil
      end
    end

    if !val
      val = yield

      if options[:marshal]
        REDIS.set key, Marshal.dump(val)
      else
        REDIS.set key, val
      end

      REDIS.expire key, options[:ttl] if options[:ttl]
    end

    val
  end

end

There is not much to describe here since its already documented using yard. Instead, I’ll show how I used in in one of my apps:

module Geocoder

  def self.geocode(location)
    key = "geolocations/#{location}"
    RedisHelper.fetch key, ttl: 24.hours, marshal: true do
      Geokit::Geocoders::GoogleGeocoder.geocode(location)
    end
  end

end

I’m using geokit gem for discovering geocoordinate of an address. It uses Google geocoding API which is limited to 2500 queries per day. This piece of code will cache the result of GoogleGeocoder.geocode method so all successive requests will be returned from cache.

redis_json_marshal_eval_benchmarks.rb

# http://forrst.com/posts/JSON_vs_Marshal_vs_eval_Which_is_the_fastest_fo-6Qy

require 'benchmark'
require 'json'
require 'redis'

# -----------------

puts "Initialize variables.."

# Initialize connection to Redis via unix socket
redis = Redis.new(path: '/tmp/redis.sock')

# Generate array containing nested hashes with random integers, ex: { 1931 => { 9159 => 9366 }, 'stringkey' => 'stringvalue' }
hashes = []
100000.times do |i|
  hashes[i] = { Random.rand(1..100) => { Random.rand(1..100) => Random.rand(1..100) }, 'stringkey' => 'stringvalue' }
end

puts '-----------------'

puts "Benchmark converting and inserting to Redis\n\n"

# Benchmark converting all the generated hashes into Redis
Benchmark.bm(20) do |x|
  # Convert to json
  x.report('to_json:') {  hashes.each { |h| redis.set 'hash:j', h.to_json }  }
  # "{\"1931\":{\"9159\":9366},\"stringkey\":\"stringvalue\"}"

  # Convert to bytestream
  x.report('Marshal.dump:') {  hashes.each { |h| redis.set 'hash:m', Marshal.dump(h) }  }
  # "\x04\b{\ai\x02\x8B\a{\x06i\x02\xC7#i\x02\x96$I\"\x0Estringkey\x06:\x06EFI\"\x10stringvalue\x06;\x00F"

  # Convert to string
  x.report('hash.to_s:') {  hashes.each { |h| redis.set 'hash:s', h } }
  # "{1931=>{9159=>9366}, \"stringkey\"=>\"stringvalue\"}"
end

puts '-----------------'

puts "Retrieve from Redis to compare\n\n"

# Retrieve from Redis
rawjson = redis.get 'hash:j'
rawbytestream = redis.get 'hash:m'
rawstring = redis.get 'hash:s'

# Convert back to a hash object
json = JSON.parse(rawjson)
bytestream = Marshal.restore(rawbytestream)
string = eval(rawstring)

puts "json == bytestream: #{json == bytestream}" # FALSE
puts "string == bytestream: #{string == bytestream}" # TRUE

puts '-----------------'

puts "Benchmark retrieving from Redis and converting back to a hash\n\n"

# Benchmark converting the hash 100000 times
Benchmark.bm(20) do |x|
  x.report('JSON.parse') { 100000.times { JSON.parse(redis.get 'hash:j') } }
  x.report('Marshal.restore') { 100000.times { Marshal.restore(redis.get 'hash:m') } }
  x.report('eval(string)') { 100000.times { eval(redis.get 'hash:s') } }
end
# 
# 
# 
# Initialize variables..
# -----------------
# Benchmark converting and inserting to Redis
# 
#                            user     system      total        real
# to_json:               4.380000   1.060000   5.440000 (  6.570592)
# Marshal.dump:          2.870000   1.030000   3.900000 (  5.021693)
# hash.to_s:             2.890000   1.010000   3.900000 (  4.997409)
# -----------------
# Retrieve from Redis to compare
# 
# json == bytestream: false
# string == bytestream: true
# -----------------
# Benchmark retrieving from Redis and converting back to a hash
# 
#                            user     system      total        real
# JSON.parse             3.440000   1.070000   4.510000 (  5.497154)
# Marshal.restore        3.130000   1.050000   4.180000 (  5.163987)
# eval(string)           5.130000   1.140000   6.270000 (  7.270163)