Skip to content

Instantly share code, notes, and snippets.

Forked from ddemaree/
Created January 28, 2012 00:18
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save agnellvj/1691748 to your computer and use it in GitHub Desktop.
How Sunspot implements its wonderful search/index DSL

This code is extracted/adapted from Mat Brown's Sunspot gem. One of Sunspot's nicest features is an expressive DSL for defining search indexes and performing queries. It works by instance_eval-ing a block you pass into it in the context of its own search builder object. In this code, the pig thing1 statement is roughly equivalent to zoo =; zoo.pig(thing1).

Sunspot's DSL has to resort to trickery: the instance_eval_with_context method uses eval to get the block to give up the object it considers to be self, then sets up an elaborate system of delegates and method_missing calls so any methods not handled by the DSL are forwarded to the surrounding object. But as a result, this syntax is minimal and beautiful, and it works the way you expect whether or not you prefer blocks to yield an object.

Without this trick the block would be restricted to either the original, "calling" context (as a closure) or the DSL's "receiving" context (using instance_eval), but not both.

Using instance_eval for cleaner syntax do
  # This will raise NoMethodError; this block is executing in the context of 
  # a Carburetor search builder and has no access to data from the controller.
  keywords params[:query]

Simply calling the blocks, no eval do |search|
  # `instance_eval` breaks the block/closure's normal scoping behavior
  # This works because we're allowing the block to behave normally, as
  # a closure.
  search.keywords params[:query]

To be very clear, while I like Sunspot's syntax and prefer code that behaves in one (roughly) consistent way, there is nothing wrong with the latter, more blocky form of this code. I do think it's uglier, in the objective sense that it has a lot of repetitive information that makes this code harder to read, especially as I add statements or nest multiple levels of blocks. I also think homemade muffins are uglier than the ones they have at Starbucks, but in that case I think it's clear that uglier is better.

require 'set'
module Util
def self.instance_eval_or_call(object, &block)
# Here we assume that if the block takes one or more arguments,
# you want to execute the block in its own scope, yielding the
# DSL constructor as an object. In this case the DSL would function
# like this:
# Zoo.setup do |zoo|
# zoo.pig likes: "slop"
# end
if block.arity > 0
# But that's boring. The real trick is this: we want to use this DSL
# without yielding an argument, like this:
# Zoo.setup do
# pig likes: "slop"
# end
# `instance_eval` is the typical way to accomplish that, eval'ing the
# block so that `pig` and `cow` are local methods. But doing that, you
# lose the surrounding context. If this were in a Rails controller,
# you couldn't use the `params` hash or any of your instance variables
# because the `Zoo` instance doesn't know about them.
# Ruby has no real concept of closures, or anything like JavaScript's
# `Function#apply()` function. So here Mat has created a proxy/delegate
# class, `ContextBoundDelegate`. Inside your block it looks as if you
# have access to both the DSL object _and_ its surrounding scope, but this
# is an illusion. You're actually calling methods on `ContextBoundDelegate`,
# which in turn forwards them to either the DSL object (if it responds to
# a particular message) or the surrounding scope (if it's not). I will try
# to explain how this works below.
ContextBoundDelegate.instance_eval_with_context(object, &block)
class ContextBoundDelegate
class <<self
# Designated initializer for `ContextBoundDelegate` -- in fact, the only
# public initializer, as the `#new` method has been made private below.
# It's possible you could do this same job inside of `#initialize`, but
# I like that this method is very, _very_ clear about the fact that it's
# eval'ing something. You're not supposed to know about `ContextBoundDelegate`;
# it's an implementation detail of your _real_ API.
def instance_eval_with_context(receiver, &block)
# Extract the block's context by eval'ing `self`, so we can store it in a variable.
calling_context = eval('self', block.binding)
# It's possible that one ContextBoundDelegate may be nested inside another.
# In fact, Sunspot uses this feature in a few places such as boolean queries.
# If the calling context has a calling context of its own (i.e., if it's another
# ContextBoundDelegate) then we need to forward messages to _that_ object instead.
# Consequently, syntax like this will Just Work:
# Zoo.setup do
# monkey_house do
# chimp eats: params[:chimp_food]
# end
# end
# In this example, `Zoo#monkey_house` is a DSL instance method that
# opens a new ContextBoundDelegate, nested inside the first one. In order
# to access the `params` object we need to still forward that message
# to the original surrounding context. So we do.
if parent_calling_context = calling_context.instance_eval{@__calling_context__}
calling_context = parent_calling_context
# One reason why you never initialize this object directly is that it's
# intended use is as a proxy. Here we construct a new ContextBoundDelegate,
# then immediately `instance_eval` the block using _the delegate_ as context.
# It is now aware of both our DSL and the controller around it, and can
# serve as a kind of internal message bus, ensuring things are called on the
# correct object.
new(receiver, calling_context).instance_eval(&block)
private :new
BASIC_METHODS = Set[:==, :equal?, :"!", :"!=", :instance_eval,
:object_id, :__send__, :__id__]
# Like all Ruby objects, ContextBoundDelegate inherits from Object,
# and so carries a lot of baggage in the form of standard instance
# methods. But our intent is for ContextBoundDelegate to be more or
# less invisible, to behave as if it _is_ one of our two contexts.
# So here we're un-defining (i.e. deleting) *all* instance methods
# except for a few specific ones we need, listed in BASIC_METHODS
# above. Those are limited to methods responsible for telling whether
# any two delegate proxies are the same, plus `instance_eval`.
instance_methods.each do |method|
unless BASIC_METHODS.include?(method.to_sym)
def initialize(receiver, calling_context)
@__receiver__, @__calling_context__ = receiver, calling_context
# In the case of #id, we want to short-circuit the normal proxying
# behavior (which favors our DSL receiver over the calling context)
# and send any #id messages directly to the caller. I didn't understand
# why at first, but then I did: it's for ActiveRecord and other ORMs
# where you might call `` or just `id` and expect it to return
# a record/object identifier. Consequently, one constraint on your DSL
# syntax is that it can't use the `#id` method for anything.
def id
# Special case due to `Kernel#sub`'s existence. Kernel methods (such as
# `rand()`) are forwarded directly to Kernel, bypassing `method_missing`
# and skipping this object completely unless it implements this method
# itself. Here we're just forcing #sub to go through our cascading proxy
# like any other method, just in case either context cares to implement it.
def sub(*args, &block)
__proxy_method__(:sub, *args, &block)
# Receives any methods that aren't explicitly implemented by this proxy
# thingy, and forwards them to `__proxy_method__`. Why is `__proxy_method__`
# separate? So it can be reused for special cases like `#sub` above.
def method_missing(method, *args, &block)
__proxy_method__(method, *args, &block)
# Where the magic happens. This method is actually really simple: _every_
# method or variable you call inside your block is run through here. The
# proxy attempts to call it on the receiver (i.e. the DSL). If it's not
# implemented there, it tries the calling context. If _that_ doesn't work,
# NoMethodError is raised. It actually sends the message to the objects
# (rather than check for them using `respond_to?()`) because you may want
# to implement your DSL syntax using `method_missing`, or through some other
# metaprogramming trick that might short-circuit Ruby's ways of detecting
# the presence of a method. Hence the cascading rescue statements. With the
# exception of #id (handled as a special case above), the receiver always
# has precedence over the caller.
def __proxy_method__(method, *args, &block)
@__receiver__.__send__(method.to_sym, *args, &block)
rescue ::NoMethodError => e
@__calling_context__.__send__(method.to_sym, *args, &block)
rescue ::NoMethodError
# This is a DSL for describing a zoo, with animals and buildings.
class Zoo
# Constructor for our zoo. It takes a block, which is passed to
# `instance_eval_or_call`, which does all the magical things discussed
# above. Returns our constructed Zoo object.
def self.setup(&block) do |zoo|
Util.instance_eval_or_call(zoo, &block)
def initialize
@data = {zoo:{}}
# The __current_target__ variable is used to track where we are in
# the zoo. If we enter a building, like the monkey house or a barn,
# that will need to be represented here until we come out of it.
@__current_target__ = @data[:zoo]
# You describe animals using these declarative methods:
# pig goes: "oink", eats: "slop"
def pig(thing)
add_thing_to_animal(:pig, thing)
def cow(thing)
add_thing_to_animal(:cow, thing)
def chimp(thing)
add_thing_to_animal(:chimp, thing)
def elephant(thing)
add_thing_to_animal(:elephant, thing)
# To enter a building, you use a handy block syntax:
# monkey_house do
# orangutan is:"orange"
# end
# Buildings can be nested inside one another:
# african_pavilion do
# elephant size:"extra-large"
# reptile_house do
# end
# end
def monkey_house(&block)
add_building(:monkey_house, &block)
def barn(&block)
add_building(:barn, &block)
def african_pavilion(&block)
add_building(:african_pavilion, &block)
def reptile_house(&block)
add_building(:reptile_house, &block)
def add_thing_to_animal(animal_name, thing)
(@__current_target__[animal_name] ||= []) << thing
def add_building(building_name, &block)
Util.instance_eval_or_call(self, &block)
def enter_building(building_name)
@__previous_target__ = @__current_target__
@__current_target__[building_name] ||= {}
@__current_target__ = @__current_target__[building_name]
def exit_building
raise "You're not currently in a building" if @__previous_target__.nil?
@__current_target__ = @__previous_target__
what_the_pig_says = "oink"
the_zoo = Zoo.setup do
pig says: what_the_pig_says # Variable from outside the block
cow says:"moo", provides: "milk"
monkey_house do
chimp eats: "bananas"
african_pavilion do
reptile_house do
cow "Why is a cow in the reptile house???"
require 'pp'
pp the_zoo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment