NGram Analyzer in ElasticSearch

elastic_search_ngram_analyzer_for_urls.sh

# ========================================
# Testing n-gram analysis in ElasticSearch
# ========================================

curl -X DELETE localhost:9200/ngram_test
curl -X PUT    localhost:9200/ngram_test -d '
{
  "settings" : {
    "index" : {
      "analysis" : {
        "analyzer" : {
          "url_analyzer" : {
            "type" : "custom",
            "tokenizer" : "lowercase",
            "filter"    : ["stop", "url_stop", "url_ngram"]
          }
        },
        "filter" : {
          "url_stop" : {
            "type" : "stop",
            "stopwords" : ["http", "https"]
          },
          "url_ngram" : {
            "type" : "nGram",
            "min_gram" : 3,
            "max_gram" : 5
          }
        }
      }
    }
  },

  "mappings": {
    "url": {
      "properties": {
        "url": {
          "type": "string",
          "analyzer": "url_analyzer",
          "boost": 10
        }
      }
    }
  }
}
'

curl -X POST "http://localhost:9200/ngram_test/url" -d '{ "url" : "http://heise.de" }'
curl -X POST "http://localhost:9200/ngram_test/url" -d '{ "url" : "http://heisewetter.de" }'
curl -X POST "http://localhost:9200/ngram_test/url" -d '{ "url" : "http://eisenwerken.de" }'
curl -X POST "http://localhost:9200/ngram_test/url" -d '{ "url" : "http://eisenwerkenberlin.de" }'
curl -X POST "http://localhost:9200/ngram_test/url" -d '{ "url" : "http://urlaubinkroatien.de" }'
curl -X POST "http://localhost:9200/ngram_test/url" -d '{ "url" : "http://besteurlaubinkroatien.de" }'
curl -X POST "http://localhost:9200/ngram_test/url" -d '{ "url" : "http://kroatien.de" }'
curl -X POST "http://localhost:9200/ngram_test/_refresh"

# curl "http://localhost:9200/ngram_test/_analyze?text=http://heise.de&analyzer=url_analyzer"

URLS='
http://localhost:9200/ngram_test/_search?q=url:heise
http://localhost:9200/ngram_test/_search?q=url:eis
http://localhost:9200/ngram_test/_search?q=url:berlin
http://localhost:9200/ngram_test/_search?q=url:wetter
http://localhost:9200/ngram_test/_search?q=url:kroatien
http://localhost:9200/ngram_test/_search?q=url:(urlaub%20kroatien)
'

for url in ${URLS}
do
  echo; echo; echo ">>> ${url}"
  if which open &> /dev/null; then
    open "${url}&pretty=true"
  fi
    curl "${url}&pretty=true"
done

@karmi, are you aware here that if I search for eisen, I get back http://heise.de as a result, how do I stop this from happening.
In other words, I want the results to contains all of the letters in the search term.

http://localhost:9200/ngram_test/_search?q=url:eisen&pretty=true

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 4,
    "max_score" : 29.876406,
    "hits" : [ {
      "_index" : "ngram_test",
      "_type" : "url",
      "_id" : "AVMOHbCJAFXKSyJelryL",
      "_score" : 29.876406,
      "_source" : {
        "url" : "http://eisenwerken.de"
      }
    }, {
      "_index" : "ngram_test",
      "_type" : "url",
      "_id" : "AVMOHbCUAFXKSyJelryM",
      "_score" : 20.15047,
      "_source" : {
        "url" : "http://eisenwerkenberlin.de"
      }
    }, {
      "_index" : "ngram_test",
      "_type" : "url",
      "_id" : "AVMOHbBqAFXKSyJelryJ",
      "_score" : 10.0413685,
      "_source" : {
        "url" : "http://heise.de"
      }
    }, {
      "_index" : "ngram_test",
      "_type" : "url",
      "_id" : "AVMOHbB8AFXKSyJelryK",
      "_score" : 5.262483,
      "_source" : {
        "url" : "http://heisewetter.de"
      }
    } ]
  }
}

I also would like to know the answer to the question @stevebissett asked.

Apparently the answer is in the documentation https://www.elastic.co/guide/en/elasticsearch/guide/current/_index_time_search_as_you_type.html.

We want to ensure that our inverted index contains edge n-grams of every word, but we want to match only the full words that the user has entered (brown and fo). We can do this by using the autocomplete analyzer at index time and the standard analyzer at search time.

Hope this helps someone

You could add a condition to make all the NGrams match,

{
          "match": {
            "url": {
              "query": "eisen",
              "analyzer": "trigrams", <-- run the same analyzer on the query
              "minimum_should_match": "100%"
            }
          }
        }