Skip to content

Instantly share code, notes, and snippets.

@klausbrunner
Last active December 11, 2015 04:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save klausbrunner/4547457 to your computer and use it in GitHub Desktop.
Save klausbrunner/4547457 to your computer and use it in GitHub Desktop.
Elasticsearch issue with phonetic plugin and index templates (stored in config). Tested with elasticsearch-analysis-phonetic 1.1.0 and 1.2.0 on both Linux and Windows on ES 0.19.12 and 0.20.2. Summary: it works without using a template, it works with a template added through the template API, but consistently fails with a template that is stored…
{
"asset_template": {
"template": "*",
"settings": {
"analysis": {
"filter": {
"metaphone_filter": {
"replace": false,
"encoder": "doublemetaphone",
"type": "phonetic"
}
},
"analyzer": {
"phonetic_name": {
"filter": ["metaphone_filter"],
"type": "custom",
"tokenizer": "standard"
}
}
}
},
"mappings": {
"asset": {
"_source": {
"compress": true
},
"dynamic_templates": [{
"template_searchable": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"type": "multi_field",
"fields": {
"metaphone": {
"type": "string",
"analyzer": "phonetic_name"
},
"untouched": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}]
}
}
}
}
# NB: make sure the template config JSON is in /etc/elasticsearch/templates, then restart elasticsearch
curl -XPUT http://localhost:9200/savedassets/asset/1234 -d '
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]}
'
# FAILURE:
# {"error":"MapperParsingException[Analyzer [phonetic_name] not found for field [metaphone]]","status":400}
# verify settings:
curl http://localhost:9200/_settings
# {"savedassets":{"settings":{"analysis.filter.metaphone_filter.replace":"false","analysis.analyzer.phonetic_name.filter.0":"metaphone_filter","analysis.filter.metaphone_filter.encoder":"doublemetaphone","analysis.analyzer.phonetic_name.tokenizer":"standard","analysis.filter.metaphone_filter.type":"phonetic","analysis.analyzer.phonetic_name.type":"custom","index.number_of_shards":"1","index.number_of_replicas":"0","index.version.created":"190799"}}}
# verify mappings
curl http://localhost:9200/savedassets/_mapping?pretty
# clean up:
curl -XDELETE http://localhost:9200/savedassets
# this is what ES logs on the above failure:
[2013-01-16 15:26:33,590][DEBUG][action.index ] [Hardnose] [savedassets][0], node[hQvApltZS864HsUevbJqEw], [P], s[STARTED]: Failed to execute [index {[savedassets][asset][1234], source[
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]}
]}]
org.elasticsearch.index.mapper.MapperParsingException: Analyzer [phonetic_name] not found for field [metaphone]
at org.elasticsearch.index.mapper.core.TypeParsers.parseField(TypeParsers.java:74)
at org.elasticsearch.index.mapper.core.StringFieldMapper$TypeParser.parse(StringFieldMapper.java:116)
at org.elasticsearch.index.mapper.multifield.MultiFieldMapper$TypeParser.parse(MultiFieldMapper.java:129)
at org.elasticsearch.index.mapper.object.RootObjectMapper.findTemplateBuilder(RootObjectMapper.java:218)
at org.elasticsearch.index.mapper.object.RootObjectMapper.findTemplateBuilder(RootObjectMapper.java:204)
at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:689)
at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:449)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:437)
at org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:311)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:202)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:532)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:430)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
# NB: make sure the template config JSON is deleted from /etc/elasticsearch/templates, then restart elasticsearch
curl -XPUT localhost:9200/_template/template_1 -d '
{
"template": "*",
"settings": {
"analysis": {
"filter": {
"metaphone_filter": {
"replace": false,
"encoder": "doublemetaphone",
"type": "phonetic"
}
},
"analyzer": {
"phonetic_name": {
"filter": ["metaphone_filter"],
"type": "custom",
"tokenizer": "standard"
}
}
}
},
"mappings": {
"asset": {
"_source": {
"compress": true
},
"dynamic_templates": [{
"template_searchable": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"type": "multi_field",
"fields": {
"metaphone": {
"type": "string",
"analyzer": "phonetic_name"
},
"untouched": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}]
}
}
}'
curl -XPUT http://localhost:9200/savedassets/asset/1234 -d '
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]}
'
# WORKS!
# {"ok":true,"_index":"savedassets","_type":"asset","_id":"1234","_version":1}
# verify mappings
curl http://localhost:9200/savedassets/_mapping?pretty
# clean up:
curl -XDELETE http://localhost:9200/savedassets
# NB: make sure the template config JSON is deleted from /etc/elasticsearch/templates, then restart elasticsearch
curl -XPOST http://localhost:9200/savedassets -d '
{
"settings": {
"analysis": {
"filter": {
"metaphone_filter": {
"replace": false,
"encoder": "doublemetaphone",
"type": "phonetic"
}
},
"analyzer": {
"phonetic_name": {
"filter": ["metaphone_filter"],
"type": "custom",
"tokenizer": "standard"
}
}
}
},
"mappings": {
"asset": {
"_source": {
"compress": true
},
"dynamic_templates": [{
"template_searchable": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"type": "multi_field",
"fields": {
"metaphone": {
"type": "string",
"analyzer": "phonetic_name"
},
"untouched": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}]
}
}
}'
# {"ok":true,"acknowledged":true}
curl -XPUT http://localhost:9200/savedassets/asset/1234 -d '
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]}
'
# WORKS!
# {"ok":true,"_index":"savedassets","_type":"asset","_id":"1234","_version":1}
# verify mappings
curl http://localhost:9200/savedassets/_mapping?pretty
# clean up:
curl -XDELETE http://localhost:9200/savedassets
@klausbrunner
Copy link
Author

The problem in this case was a malformed JSON tempate file (thanks to Igor Motov for spotting it: https://groups.google.com/d/msg/elasticsearch/X_YRiaV6zLg/9ZEQ8rHAkzQJ ).

Basically, it appears that a settings JSON in a separate file must explicitly add "index" for index-related settings, while those settings that are specified as part of an index template are implicitly under "index" anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment