Skip to content

Instantly share code, notes, and snippets.

@ianribas
Last active January 8, 2021 13:23
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ianribas/f76d20c21bb9f5c0df2f to your computer and use it in GitHub Desktop.
Save ianribas/f76d20c21bb9f5c0df2f to your computer and use it in GitHub Desktop.
Elasticsearch problem with multi word query time synonyms and match queries with AND operator
#!/bin/bash
# delete old index if exists
curl -XDELETE 'http://localhost:9200/multiwordsyns?pretty'
# create index with synonym analyzer and mapping
curl -XPUT 'http://localhost:9200/multiwordsyns?pretty' -d '{
"settings" : {
"number_of_replicas": 0,
"number_of_shards": 1,
"index": {
"analysis": {
"analyzer": {
"index_time": {
"tokenizer": "standard",
"filter": ["standard", "lowercase"]
},
"synonym": {
"tokenizer": "standard",
"filter": ["standard", "lowercase", "stop", "synonym"]
}
},
"filter": {
"synonym": {
"type": "synonym",
"synonyms": [
"spider man, spiderman"
]
}
}
}
}
},
"mappings": {
"test": {
"properties": {
"text": {"type": "string", "index_analyzer": "index_time", "search_analyzer": "synonym"}
}
}
}
}'
# index the test documents
curl -XPUT 'http://localhost:9200/multiwordsyns/test/1?pretty' -d '{"text": "the adventures of spiderman"}'
curl -XPUT 'http://localhost:9200/multiwordsyns/test/2?pretty' -d '{"text": "what hath man wrought?"}'
curl -XPUT 'http://localhost:9200/multiwordsyns/test/3?pretty' -d '{"text": "that spider is the size of a man"}'
curl -XPUT 'http://localhost:9200/multiwordsyns/test/4?pretty&refresh=true' -d '{"text": "spiders eat insects"}'
# WRONG! finds only #1, should find #1 & #3
curl -XPOST 'http://localhost:9200/multiwordsyns/test/_search?pretty' -d '{"query": {"match": {"text": {"query": "spiderman", "operator": "and"}}}}'
# Also WRONG! finds only #1, should find #1 & #3
curl -XPOST 'http://localhost:9200/multiwordsyns/test/_search?pretty' -d '{"query": {"match": {"text": {"query": "spider man", "operator": "and"}}}}'
@jzbahrai
Copy link

jzbahrai commented Dec 8, 2015

Is there a solution to this?

@rathko
Copy link

rathko commented Jan 11, 2016

I was wondering the same. Did you find a solution to it?

@ricardocabral
Copy link

same problem here :(

@MichelHalmes
Copy link

From what I understand "spider man, spiderman" corresponds to the simple contraction "spider man, spiderman => spiderman".

If you use the following it works (on 6.1):

"synonyms": [
      "spider man, spiderman => spider man, spiderman"
]

The documentation is quite unclear and references some SOLR parameter expand that is not available in Elasticsearch where one has to expand explicitly:
https://www.elastic.co/guide/en/elasticsearch/reference/6.1/analysis-synonym-tokenfilter.html

Hope this helps?!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment