Skip to content

Instantly share code, notes, and snippets.

@karussell
Last active May 7, 2020 10:25
  • Star 24 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save karussell/1639766 to your computer and use it in GitHub Desktop.
Bird's Eye View on ElasticSearch its Query DSL
Several times in a month there pop up questions regarding query structure on the ElasticSearch user group.
Although there are good docs explaining this in depth probably the bird view of the Query DSL is necessary to
understand what is written there. There is even already some good external documentation available:
http://www.elasticsearch.org/tutorials/2011/08/28/query-dsl-explained.html
And there were attempts to define a schema:
http://groups.google.com/group/json-schema/browse_thread/thread/ae498ee818155d50
https://gist.github.com/8887766ca0e7052814b0
... but nevertheless I'll add my 2 cents here. I assume you set up your ElasticSearch
instance correctly and on the local machine:
http://www.elasticsearch.org/guide/reference/setup/installation.html
filled with only 3 articles:
http://www.elasticsearch.org/guide/reference/api/search/facets/index.html
Now we can query it as it is done there (Keep in mind to use the keyword analyzer for tags!):
curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '
{
"query" : { "query_string" : {"query" : "T*"} },
"facets" : {
"tags" : { "terms" : {"field" : "tags"} }
}
}'
But when you now look into the query DSL docs you'll only find the query part:
http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query.html
{
"query_string" : {
"default_field" : "content",
"query" : "this AND that OR thus"
}
}
And this query part can be replaced by your favourite query. Be it a filtered, term, a boolean or whatever query:
http://www.elasticsearch.org/guide/reference/query-dsl/filtered-query.html
http://www.elasticsearch.org/guide/reference/query-dsl/term-query.html
http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html
So what is the main structure of a query? It is:
curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '
{
"from": 0,
"size": 10,
"query" : QUERY_JSON,
FILTER_JSON,
FACET_JSON,
SORT_JSON
}'
Keep in mind that the FILTER_JSON only applies to the query not to facets. Read on how to do this.
And now a short example how this nicely maps to the Java API:
SearchRequestBuilder srb = client.prepareSearch("your_index");
srb.setQuery(QueryBuilders.queryString("title:test"));
srb.addSort("tags", SortOrder.ASC);
srb.addFacet(FacetBuilders.termsFacet("tags"));
If you install my hack:
https://github.com/mobz/elasticsearch-head/pull/3
you can formulate the above query separation directly in your browser. E.g.:
q={ match_all:{} };
{ query:q }
A more detailed query structure is as follows - you could obtain it via Java API easily or via
the right navigational elements of this site:
http://www.elasticsearch.org/guide/reference/api/search/
or directly from the source:
https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/search/builder/SearchSourceBuilder.java#L576
curl -X POST "http://localhost:9200/articles/_search?pretty=true" -d '
{
"query" : QUERY_JSON,
"filter" : FILTER_JSON,
"from": 0,
"size": 10,
"sort" : SORT_ARRAY,
"highlight" : HIGHLIGHT_JSON,
"fields" : ["tags", "title"],
"script_fields": SCRIPT_FIELDS_JSON,
"preference": "_local",
"facets" : FACET_JSON,
"search_type": "query_then_fetch",
"timeout": -1,
"version": true,
"explain": true,
"min_score": 0.5,
"partial_fields": PARTIAL_FIELDS_JSON,
"stats" : ["group1", "group2"]
}'
Let us dig into a simple query with some filters and facets:
curl -XGET 'http://localhost:9200/articles/_search?pretty=true' -d '
{"query": {
"filtered" : {
"query" : { "match_all" : {} },
"filter" : {
"term" : { "tags" : "bar" }
}
}
},
"facets" : {
"tags" : { "terms" : {"field" : "tags"} }
}}'
You will get 2 out of 3 documents and the filter directly applies on the facets as well.
If you don't want that put the filter part under the query:
curl -XGET 'http://localhost:9200/articles/_search?pretty=true' -d '
{"query" : { "match_all" : {} },
"filter" : {
"term" : { "tags" : "bar" }
},
"facets" : {
"tags" : { "terms" : {"field" : "tags"} }
}}'
And how can I only filter on the facets? You'll need facet_filter:
curl -XGET 'http://localhost:9200/articles/_search?pretty=true' -d '
{"query" : { "match_all" : {} },
"facets" : {
"mytags" : {
"terms" : {"field" : "tags"},
"facet_filter" : {
"term" : { "tags" : "bar"}
}
}
}}'
You'll get 3 documents with filtered facets
@lucianspec
Copy link

es reference about query DSL is full of examples, no syntax specs. THANKS for your help & i found this offical rest-api-spec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment