Skip to content

Instantly share code, notes, and snippets.



Last active Aug 29, 2015
What would you like to do?
Payload Builder

Suggested Plan to migrate from AR to ES in our Rails app

Clean up Queries

  • Change the specs and the queries to move filters from filter to query.filtered.filter
  • Move {term: {budget_lea_id: budget_lea_id} from query to query.filtered.filter

Remove JBuilder

  • Create PayloadBuilder library to build payloads
  • Convert JBuilder code to PayloadBuilder library

Remove AR

  • Move authorization from AR to ES.*
  • Move ordering from PG to ES
  • Remove results.records**
  • Move pagination from PG to ES


*Originally, I thought we could just use a Ruby function which rejects items in the results set. For pagination (as a filter could change the pagination), better to do in ES

** Due to the fact we call results.records and then use Postgres to filter, we can't use ElasticSearch for pagination until we remove results.records.

# ============================
# ============================
# ==============================
# On the organization of queries
# ==============================
# All of our queries have the same basic form (below). The methods assume we're passing a payload around that starts like this.
"query": {
"filtered": {
"filter": {
"and": []
"query": {
# ==========
# Philosophy
# ==========
# ES queries can be nested (queries in filters in queries in filters, etc). Thus, we shouldn't assume that when we add a filter, we're operating on the root of our payload.
# However, we also want to make it easy to add a query to the right place without having to repeat ourselves.
# Thus, I propose each function has a param that let's the location of the operation be changed
# Method Signature:
PB::Module::type.method(payload, hash_or_array, location)
# =====
# =====
# Add filter to query. Will be added by default to "query.filtered.filter"
PB::Filter.and.add(payload, filter)
# Add filter to an aggregation
PB::Filter.and.add(payload, filter, [:aggs, :segment_filter_agg, :filter])
# Add filter to post_filter at root of hash
PB::Filter.and.add(payload, filter, [:post_filter])
# Add filter to a arbitrary hash. For instance, if you're building up a filter in a complex aggregation, you want to use this function to add the hash
PB::Filter.and.add(some_hash, filter, [:filters])
# =====
# =====
# Add query to query
PB::Query.merge(payload, and_query)
# Add fuzzy_like_this
PB::Query.fuzzy_like_this.add(payload, hash)
# Add boolean#must
PB::Query::Boolean.must.add(payload, must_array)
# Add boolean#should
PB::Query::Boolean.should.add(payload, should_array)

This comment has been minimized.

Copy link

@khristian khristian commented Jul 21, 2015

I like it. Looks good. Didn't see any use of the boolean OR operator on filters. Also, may want to use 'multi-match' and 'query' instead of 'fuzzy_like_this' (

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.