Skip to content

Instantly share code, notes, and snippets.

@bouk
Last active November 4, 2016 01:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bouk/6408a4ccec4ce20118e6ad27c9233d28 to your computer and use it in GitHub Desktop.
Save bouk/6408a4ccec4ce20118e6ad27c9233d28 to your computer and use it in GitHub Desktop.
path = Rails.root.join("tmp/cache/sprockets-lmdb-#{Rails.env}"); path.mkpath; env.cache = LMDBCacheStore.new(path.to_s)
require 'lmdb'
require 'snappy'
class LMDBCacheStore
attr_reader :max_size, :env, :db, :lru
delegate :size, to: :db
# The LMDB Gem has a bug where the Environment garbage collection handler will crash sometimes
# if the environment wasn't closed explicitely before the reference was lost.
# As a shitty workaround, we can make sure that we never lose a reference to the Environment by
# putting it into a class variable.
def self.save_env(env)
@saved_envs ||= []
@saved_envs << env
end
def initialize(path, max_size: 25_000)
@max_size = max_size
# mapsize is the absolute maximum size of the database before it will error out, it won't
# allocate space on disk until it's needed.
# According to the LMDB documentation, disabling sync means we lose the D in ACID (durability)
# We don't care about that, because a missing transaction will
# just lead to slightly more work being done the next time around.
@env = LMDB.new(path, mapsize: 2.gigabyte, nometasync: true, nosync: true)
self.class.save_env(@env)
@db = env.database('cache', create: true)
@lru = env.database('lru', create: true)
end
def get(key)
value = nil
env.transaction do
value = db.get(key)
update_lru(key) unless value.nil?
end
value && Marshal.load(Snappy.inflate(value))
end
def set(key, value)
value = Snappy.deflate(Marshal.dump(value))
env.transaction do
db.put(key, value)
update_lru(key)
gc if lru.size > max_size * 1.5
end
end
def inspect
"#<#{self.class} size=#{size}/#{max_size}>"
end
def gc
env.transaction do
keys = []
lru.each do |(hash, time)|
keys << [time.unpack('q').first, hash]
end
return if keys.size <= max_size
keys.sort!
keys = keys.slice(0, keys.size - max_size)
keys.each do |(_, key)|
delete(key)
end
end
end
private
def update_lru(key)
lru.put(key, [Time.now.to_i].pack('q'))
end
def delete(key)
db.delete(key)
rescue LMDB::Error::NOTFOUND
nil
ensure
lru.delete(key)
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment