Skip to content

Instantly share code, notes, and snippets.

@mottaquikarim
Created December 29, 2020 17:15
Show Gist options
  • Save mottaquikarim/8d2b3acb3d4e158da2077dccb482042d to your computer and use it in GitHub Desktop.
Save mottaquikarim/8d2b3acb3d4e158da2077dccb482042d to your computer and use it in GitHub Desktop.

TL; DR: ya boy wrote a golang elasticsearch query dsl utility. Find it here!


The Why

If you've used elasticsearch with golang, then you've probably used the official elasticsearch go client.

The es go client is exhaustive and generally, pretty great. However, it can be a bit...scary when having to deal with constructing search queries using the elasticsearch query dsl

Take for instance the following (from here in the docs):

https://gist.github.com/c3b3b43363bae46de96fc8298f3dc416

Using strings

In my experience, the simplest/fastest way to construct this json string is with...well, a string:

https://gist.github.com/22bcf88b08e76df82c94a3dcfbbd9546

If we need to inject variable values, we just use fmt.Sprintf and move on with our lives. The primary issue here is that validating/formatting these json strings require additional work and can be error prone (ie: fat-fingering an additional comma somewhere, etc).

Using (use-case specific) structs

The other approach would be to make everything hyper specific and create structs / custom marshal-ers that would generate the query DSL json format when json.Marshal is called (on said custom struct(s)).

This approach requires creating custom structs and code for the sole purpose of building these queries. (This may work for certain usecases! But, it also means more code and therefore additional maintenance and more trouble translating to other projects).

What does Google say?

The best "documentation"/support I could find through google-fu was this article that also just suggested building a json string and crossing your fingers.

(excerpt from the blog post above):

https://gist.github.com/dea851bbcc1767d8d100c4a6041a1847

Finally, there was this issue on the go elasticsearch client from 2019:

This package is intentionally a low-level client, while olivere/elastic is a high-level client with extensions for building the requests and deserializing the responses. We are aiming for offering a more high-level API in the future, but — as I've indicated in other tickets — no sooner than a machine-readable, formal specification of the request and response bodies is available.

Moreover (and awesomely, that comment thread has a gem of a code snippet):

https://gist.github.com/1cd5fd5a8fa08e6646a486c7872bb607

that looks very similar to what I've ended up with (wish I had seen this first, heh) as I tackled this problem.

Regardless the main point is this:

Currently there isn't an easy way to define query DSL json strings for use with the elasticsearch go client.

And - for good reason perhaps - the official client looks like it will not support such a feature anytime soon.

package esquerydsl

For all these reasons, I decided to build a simple, dependency less (aside from go stdlib deps) utility that generically defines structs to build queryDSL json strings.

Here's an example (playground)

https://gist.github.com/e3abacd8e091efd60c82fca77add1b68

The output:

https://gist.github.com/c9b9217fb2e248041997b37103c46725

(Find more examples in tests, including the initial queryDSL example referenced at the top of this post here)

PRs welcome! Especially re: unittests such that documentation coverage is increased. If you use this lib and it is useful, do let me know please!

Happy querying, fam 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment