Last active
December 11, 2015 04:48
-
-
Save klausbrunner/4547457 to your computer and use it in GitHub Desktop.
Elasticsearch issue with phonetic plugin and index templates (stored in config). Tested with elasticsearch-analysis-phonetic 1.1.0 and 1.2.0 on both Linux and Windows on ES 0.19.12 and 0.20.2. Summary: it works without using a template, it works with a template added through the template API, but consistently fails with a template that is stored…
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"asset_template": { | |
"template": "*", | |
"settings": { | |
"analysis": { | |
"filter": { | |
"metaphone_filter": { | |
"replace": false, | |
"encoder": "doublemetaphone", | |
"type": "phonetic" | |
} | |
}, | |
"analyzer": { | |
"phonetic_name": { | |
"filter": ["metaphone_filter"], | |
"type": "custom", | |
"tokenizer": "standard" | |
} | |
} | |
} | |
}, | |
"mappings": { | |
"asset": { | |
"_source": { | |
"compress": true | |
}, | |
"dynamic_templates": [{ | |
"template_searchable": { | |
"match": "*", | |
"match_mapping_type": "string", | |
"mapping": { | |
"type": "multi_field", | |
"fields": { | |
"metaphone": { | |
"type": "string", | |
"analyzer": "phonetic_name" | |
}, | |
"untouched": { | |
"type": "string", | |
"index": "not_analyzed" | |
} | |
} | |
} | |
} | |
}] | |
} | |
} | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# NB: make sure the template config JSON is in /etc/elasticsearch/templates, then restart elasticsearch | |
curl -XPUT http://localhost:9200/savedassets/asset/1234 -d ' | |
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]} | |
' | |
# FAILURE: | |
# {"error":"MapperParsingException[Analyzer [phonetic_name] not found for field [metaphone]]","status":400} | |
# verify settings: | |
curl http://localhost:9200/_settings | |
# {"savedassets":{"settings":{"analysis.filter.metaphone_filter.replace":"false","analysis.analyzer.phonetic_name.filter.0":"metaphone_filter","analysis.filter.metaphone_filter.encoder":"doublemetaphone","analysis.analyzer.phonetic_name.tokenizer":"standard","analysis.filter.metaphone_filter.type":"phonetic","analysis.analyzer.phonetic_name.type":"custom","index.number_of_shards":"1","index.number_of_replicas":"0","index.version.created":"190799"}}} | |
# verify mappings | |
curl http://localhost:9200/savedassets/_mapping?pretty | |
# clean up: | |
curl -XDELETE http://localhost:9200/savedassets | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# this is what ES logs on the above failure: | |
[2013-01-16 15:26:33,590][DEBUG][action.index ] [Hardnose] [savedassets][0], node[hQvApltZS864HsUevbJqEw], [P], s[STARTED]: Failed to execute [index {[savedassets][asset][1234], source[ | |
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]} | |
]}] | |
org.elasticsearch.index.mapper.MapperParsingException: Analyzer [phonetic_name] not found for field [metaphone] | |
at org.elasticsearch.index.mapper.core.TypeParsers.parseField(TypeParsers.java:74) | |
at org.elasticsearch.index.mapper.core.StringFieldMapper$TypeParser.parse(StringFieldMapper.java:116) | |
at org.elasticsearch.index.mapper.multifield.MultiFieldMapper$TypeParser.parse(MultiFieldMapper.java:129) | |
at org.elasticsearch.index.mapper.object.RootObjectMapper.findTemplateBuilder(RootObjectMapper.java:218) | |
at org.elasticsearch.index.mapper.object.RootObjectMapper.findTemplateBuilder(RootObjectMapper.java:204) | |
at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:689) | |
at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:449) | |
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493) | |
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:437) | |
at org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:311) | |
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:202) | |
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:532) | |
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:430) | |
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) | |
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) | |
at java.lang.Thread.run(Thread.java:722) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# NB: make sure the template config JSON is deleted from /etc/elasticsearch/templates, then restart elasticsearch | |
curl -XPUT localhost:9200/_template/template_1 -d ' | |
{ | |
"template": "*", | |
"settings": { | |
"analysis": { | |
"filter": { | |
"metaphone_filter": { | |
"replace": false, | |
"encoder": "doublemetaphone", | |
"type": "phonetic" | |
} | |
}, | |
"analyzer": { | |
"phonetic_name": { | |
"filter": ["metaphone_filter"], | |
"type": "custom", | |
"tokenizer": "standard" | |
} | |
} | |
} | |
}, | |
"mappings": { | |
"asset": { | |
"_source": { | |
"compress": true | |
}, | |
"dynamic_templates": [{ | |
"template_searchable": { | |
"match": "*", | |
"match_mapping_type": "string", | |
"mapping": { | |
"type": "multi_field", | |
"fields": { | |
"metaphone": { | |
"type": "string", | |
"analyzer": "phonetic_name" | |
}, | |
"untouched": { | |
"type": "string", | |
"index": "not_analyzed" | |
} | |
} | |
} | |
} | |
}] | |
} | |
} | |
}' | |
curl -XPUT http://localhost:9200/savedassets/asset/1234 -d ' | |
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]} | |
' | |
# WORKS! | |
# {"ok":true,"_index":"savedassets","_type":"asset","_id":"1234","_version":1} | |
# verify mappings | |
curl http://localhost:9200/savedassets/_mapping?pretty | |
# clean up: | |
curl -XDELETE http://localhost:9200/savedassets |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# NB: make sure the template config JSON is deleted from /etc/elasticsearch/templates, then restart elasticsearch | |
curl -XPOST http://localhost:9200/savedassets -d ' | |
{ | |
"settings": { | |
"analysis": { | |
"filter": { | |
"metaphone_filter": { | |
"replace": false, | |
"encoder": "doublemetaphone", | |
"type": "phonetic" | |
} | |
}, | |
"analyzer": { | |
"phonetic_name": { | |
"filter": ["metaphone_filter"], | |
"type": "custom", | |
"tokenizer": "standard" | |
} | |
} | |
} | |
}, | |
"mappings": { | |
"asset": { | |
"_source": { | |
"compress": true | |
}, | |
"dynamic_templates": [{ | |
"template_searchable": { | |
"match": "*", | |
"match_mapping_type": "string", | |
"mapping": { | |
"type": "multi_field", | |
"fields": { | |
"metaphone": { | |
"type": "string", | |
"analyzer": "phonetic_name" | |
}, | |
"untouched": { | |
"type": "string", | |
"index": "not_analyzed" | |
} | |
} | |
} | |
} | |
}] | |
} | |
} | |
}' | |
# {"ok":true,"acknowledged":true} | |
curl -XPUT http://localhost:9200/savedassets/asset/1234 -d ' | |
{"assetId" : "testBLABLA", "flatTitles": ["title1", "title2"]} | |
' | |
# WORKS! | |
# {"ok":true,"_index":"savedassets","_type":"asset","_id":"1234","_version":1} | |
# verify mappings | |
curl http://localhost:9200/savedassets/_mapping?pretty | |
# clean up: | |
curl -XDELETE http://localhost:9200/savedassets |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The problem in this case was a malformed JSON tempate file (thanks to Igor Motov for spotting it: https://groups.google.com/d/msg/elasticsearch/X_YRiaV6zLg/9ZEQ8rHAkzQJ ).
Basically, it appears that a settings JSON in a separate file must explicitly add "index" for index-related settings, while those settings that are specified as part of an index template are implicitly under "index" anyway.