Skip to content

Instantly share code, notes, and snippets.

@neilmiddleton
Last active August 29, 2015 14:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save neilmiddleton/1c38fcbfaf8d9955ffbf to your computer and use it in GitHub Desktop.
Save neilmiddleton/1c38fcbfaf8d9955ffbf to your computer and use it in GitHub Desktop.
Searching within Morpheus

Simple search

broken

This simple search returns all tickets which contain the word "broken" across any of the ticket fields and comments.

Not so simple searches

The next query searches for tickets:

  • where the subject field contains "broken" or "API"
  • and where the created_at date is greater than 2014-09-10
  • and which contain either of the words "aggregations" or "geo" in any field
+name:(broken API) +date:>2014-09-10 +(aggregations geo)

Fields

The following fields are searchable:

  • state
  • app or app_name
  • category
  • priorty
  • subject
  • meta_data
  • comments.body
  • comments.created_at
  • comments.user_cache (contains a cache of user information at the time of the comment)
  • created_at
  • updated_at
  • views
  • premium_support

Syntax

where the status field contains pending

status:pending

where the subject field contains quick or brown. If you omit the OR operator the default operator will be used

subject:(quick OR brown)
subject:(quick brown)

where the subject field contains the exact phrase "john smith"

subject:"John Smith"

where any of the fields comment.body, comment.created_at or comment.user_cache contains neil@heroku.com or jd@heroku.com (note how we need to escape the * with a backslash):

book.\*:(neil@heroku.com jd@heroku.com)

where the field subject has no value (or is missing):

_missing_:subject

where the field subject has any non-null value:

_exists_:subject

Wildcards

Wildcard searches can be run on individual terms, using ? to replace a single character, and * to replace zero or more characters:

qu?ck bro*

Regular expressions

Regular expression patterns can be embedded in the query string by wrapping them in forward-slashes ("/"):

subject:/joh?n(ath[oa]n)/

Fuzziness

We can search for terms that are similar to, but not exactly like our search terms, using the “fuzzy” operator:

quikc~ brwn~ foks~

This uses the Damerau-Levenshtein distance to find all terms with a maximum of two changes, where a change is the insertion, deletion or substitution of a single character, or transposition of two adjacent characters.

The default edit distance is 2, but an edit distance of 1 should be sufficient to catch 80% of all human misspellings. It can be specified as:

quikc~1

Proximity searches

While a phrase query (eg "john smith") expects all of the terms in exactly the same order, a proximity query allows the specified words to be further apart or in a different order. In the same way that fuzzy queries can specify a maximum edit distance for characters in a word, a proximity search allows us to specify a maximum edit distance of words in a phrase:

"fox quick"~5

The closer the text in a field is to the original order specified in the query string, the more relevant that ticket is considered to be. When compared to the above example query, the phrase "quick fox" would be considered more relevant than "quick brown fox".

Ranges

Ranges can be specified for date, numeric or string fields. Inclusive ranges are specified with square brackets [min TO max] and exclusive ranges with curly brackets {min TO max}. Currently ranges on states are not supported.

All days in 2012:

date:[2012-01-01 TO 2012-12-31]

Dates before 2012

date:{* TO 2012-01-01}

Curly and square brackets can be combined:

Numbers from 1 up to but not including 5

count:[1..5}

Ranges with one side unbounded can use the following syntax:

age:>10
age:>=10
age:<10
age:<=10

Boosting

Use the boost operator ^ to make one term more relevant than another. For instance, if we want to find all tickets about foxes, but we are especially interested in quick foxes:

quick^2 fox

The default boost value is 1, but can be any positive floating point number. Boosts between 0 and 1 reduce relevance.

Boosts can also be applied to phrases or to groups:

"john smith"^2   (foo bar)^4

Boolean operators

By default, all terms are optional, as long as one term matches. A search for foo bar baz will find any ticket that contains one or more of foo or bar or baz. We have already discussed the default_operator above which allows you to force all terms to be required, but there are also boolean operators which can be used in the query string itself to provide more control.

The preferred operators are + (this term must be present) and - (this term must not be present). All other terms are optional. For example, this query:

quick brown +fox -news

states that:

  • fox must be present
  • news must not be present
  • quick and brown are optional -- their presence increases the relevance

The familiar operators AND, OR and NOT (also written &&, || and !) are also supported. However, the effects of these operators can be more complicated than is obvious at first glance. NOT takes precedence over AND, which takes precedence over OR. While the + and - only affect the term to the right of the operator, AND and OR can affect the terms to the left and right.

Reserved characters

If you need to use any of the characters which function as operators in your query itself (and not as operators), then you should escape them with a leading backslash. For instance, to search for (1+1)=2, you would need to write your query as (1+1)=2.

The reserved characters are: + - && || ! ( ) { } [ ] ^ " ~ * ? : \ /

Failing to escape these special characters correctly could lead to a syntax error which prevents your query from running.

Date math

Date fields support using date math expression when using it in a query/filter (mainly makes sense in range query/filter).

The expression starts with an "anchor" date, which can be either now or a date string (in the applicable format) ending with ||. It can then follow by a math expression, supporting +, - and / (rounding). The units supported are

  • y (year)
  • M (month)
  • w (week)
  • h (hour)
  • m (minute)
  • s (second).

Here are some samples: now+1h, now+1h+1m, now+1h/d, 2012-01-01||+1M/d.

Note, when doing range type searches, and the upper value is inclusive, the rounding will properly be rounded to the ceiling instead of flooring it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment