Skip to content

Instantly share code, notes, and snippets.

@lukas-vlcek
Last active February 7, 2023 21:50
Show Gist options
  • Save lukas-vlcek/5143799 to your computer and use it in GitHub Desktop.
Save lukas-vlcek/5143799 to your computer and use it in GitHub Desktop.
Adding a new analyzer into existing index in Elasticsearch (requires close/open the index). Tested with Elasticsearch 0.19.12.
// create an index with an analyzer "myindex"
curl -X PUT localhost:9200/myindex -d '
{
"settings" : {`
"index":{
"number_of_replicas":0,
"number_of_shards":1,
"analysis":{
"analyzer":{
"first":{
"type":"whitespace"
}
}
}
}
}
}'
// verify analyzers for "myindex"
curl -XGET 'http://localhost:9200/_cluster/state?pretty&filter_nodes=true&filter_routing_table=true&filter_indices=myindex'
# {
# "cluster_name" : "elasticsearch",
# "blocks" : { },
# "metadata" : {
# "templates" : { },
# "indices" : {
# "myindex" : {
# "state" : "open",
# "settings" : {
# "index.number_of_replicas" : "0",
# "index.number_of_shards" : "1",
# "index.analysis.analyzer.first.type" : "whitespace",
# "index.version.created" : "191299"
# },
# "mappings" : { },
# "aliases" : [ ]
# }
# }
# }
# }
// try to add a new analyzer
curl -XPUT 'localhost:9200/myindex/_settings' -d '{
"analysis" : {
"analyzer":{
"second":{
"type":"custom",
"tokenizer":"whitespace",
"filter":["lowercase"]
}
}
}
}'
# {"ok":true}
// but in fact index setting is not modified - the following is found in the log
[WARN ][cluster.metadata ] [Captain Omen] [myindex] ignoring non dynamic index level settings for open indices: [index.analysis.analyzer.second.type, index.analysis.analyzer.second.tokenizer, index.analysis.analyzer.second.filter.0]
// close the index
curl -XPOST 'localhost:9200/myindex/_close'
# {"ok":true,"acknowledged":true}
// we can verify index is closed
curl -XGET 'http://localhost:9200/_cluster/state?pretty&filter_nodes=true&filter_routing_table=true&filter_indices=myindex'
# {
# "cluster_name" : "elasticsearch",
# "blocks" : {
# "indices" : {
# "myindex" : {
# "4" : {
# "description" : "index closed", // <--- myindex is closed
# "retryable" : false,
# "levels" : [ "read", "write" ]
# }
# }
# }
# },
# "metadata" : {
# "templates" : { },
# "indices" : {
# "myindex" : {
# "state" : "close", // <--- state: close
# "settings" : {
# "index.number_of_replicas" : "0",
# "index.number_of_shards" : "1",
# "index.analysis.analyzer.first.type" : "whitespace",
# "index.version.created" : "191299"
# },
# "mappings" : { },
# "aliases" : [ ]
# }
# }
# }
# }
// try to add a new analyzer again
curl -XPUT 'localhost:9200/myindex/_settings' -d '{
"analysis" : {
"analyzer":{
"second":{
"type":"custom",
"tokenizer":"whitespace",
"filter":["lowercase"]
}
}
}
}'
# {"ok":true}
// we can add a new analyzer now
curl -XGET 'http://localhost:9200/_cluster/state?pretty&filter_nodes=true&filter_routing_table=true&filter_indices=myindex'
# {
# "cluster_name" : "elasticsearch",
# "blocks" : {
# "indices" : {
# "myindex" : {
# "4" : {
# "description" : "index closed",
# "retryable" : false,
# "levels" : [ "read", "write" ]
# }
# }
# }
# },
# "metadata" : {
# "templates" : { },
# "indices" : {
# "myindex" : {
# "state" : "close",
# "settings" : {
# "index.number_of_replicas" : "0",
# "index.number_of_shards" : "1",
# "index.analysis.analyzer.first.type" : "whitespace",
# "index.version.created" : "191299",
# "index.analysis.analyzer.second.tokenizer" : "whitespace",
# "index.analysis.analyzer.second.type" : "custom",
# "index.analysis.analyzer.second.filter.0" : "lowercase"
# },
# "mappings" : { },
# "aliases" : [ ]
# }
# }
# }
# }
// open the index now
curl -XPOST 'localhost:9200/myindex/_open'
# {"ok":true,"acknowledged":true}
// now everything seems to be ok
curl -XGET 'http://localhost:9200/_cluster/state?pretty&filter_nodes=true&filter_routing_table=true&filter_indices=myindex'
# {
# "cluster_name" : "elasticsearch",
# "blocks" : { },
# "metadata" : {
# "templates" : { },
# "indices" : {
# "myindex" : {
# "state" : "open",
# "settings" : {
# "index.number_of_replicas" : "0",
# "index.number_of_shards" : "1",
# "index.analysis.analyzer.first.type" : "whitespace",
# "index.version.created" : "191299",
# "index.analysis.analyzer.second.tokenizer" : "whitespace",
# "index.analysis.analyzer.second.type" : "custom",
# "index.analysis.analyzer.second.filter.0" : "lowercase"
# },
# "mappings" : { },
# "aliases" : [ ]
# }
# }
# }
# }
@maruf89
Copy link

maruf89 commented Jun 18, 2014

Useful, thank you!

@samsullivan
Copy link

Sweet, thanks! Just a heads up, as of ES 1.2.2, adding an analyzer to an open index will not appear to work. The error is returned as it should:

{
    "error": "ElasticsearchIllegalArgumentException[Can't update non dynamic settings[[index.analysis.filter.shingle_filter.type, index.analysis.filter.shingle_filter.min_shingle_size, index.analysis.analyzer.shingle_analyzer.filter.0, index.analysis.analyzer.shingle_analyzer.filter.1, index.analysis.analyzer.shingle_analyzer.tokenizer, index.analysis.filter.shingle_filter.max_shingle_size, index.analysis.analyzer.shingle_analyzer.type]] for open indices[[index]]]",
    "status": 400
}

@azhar04
Copy link

azhar04 commented Sep 20, 2016

@samsullivan! it seems your index still open, try closing it first.

@Raghav2211
Copy link

backtick is not used properly in first curl command it must be

curl -X PUT localhost:9200/myindex -d ' { "settings" : { "index":{ "number_of_replicas":0, "number_of_shards":1, "analysis":{ "analyzer":{ "first":{ "type":"whitespace" } } } } } }'

@ythalorossy
Copy link

@r-martins
Copy link

r-martins commented Jan 30, 2018

Does not work on AWS (_open & _close)..
You might need to recreate the index with the filter/analyzer.

{
"Message": "Your request: '/dev_product/_close' is not allowed by Amazon Elasticsearch Service."
}

@nitingadekar
Copy link

Hi r-martins,
Please share your experience while recreating the analyzer in AWS elasticsearch, and more details about why open and close index does not work for AWS ES. I have my prod infra running and need to verify if recreating the index would help.
Below is the query raised by my development team.

@nitingadekar
Copy link

We are trying to define few analyzers to elastic index. Service throwing following error.

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't update non dynamic settings [[index.analysis.analyzer.whitespace_analyzer.filter, index.analysis.analyzer.whitespace_analyzer.tokenizer, index.analysis.filter.autocomplete_filter.token_chars, index.analysis.filter.autocomplete_filter.min_gram, index.analysis.analyzer.whitespace_analyzer.type, index.analysis.analyzer.autocomplete_analyzer.type, index.analysis.analyzer.autocomplete_analyzer.tokenizer, index.analysis.filter.autocomplete_filter.type, index.analysis.analyzer.autocomplete_analyzer.filter, index.analysis.filter.autocomplete_filter.max_gram]] for open indices [[nse/aZsP4c90QwWylVhv2eA84g]]"}],"type":"illegal_argument_exception","reason":"Can't update non dynamic settings [[index.analysis.analyzer.whitespace_analyzer.filter, index.analysis.analyzer.whitespace_analyzer.tokenizer, index.analysis.filter.autocomplete_filter.token_chars, index.analysis.filter.autocomplete_filter.min_gram, index.analysis.analyzer.whitespace_analyzer.type, index.analysis.analyzer.autocomplete_analyzer.type, index.analysis.analyzer.autocomplete_analyzer.tokenizer, index.analysis.filter.autocomplete_filter.type, index.analysis.analyzer.autocomplete_analyzer.filter, index.analysis.filter.autocomplete_filter.max_gram]] for open indices [[nse/aZsP4c90QwWylVhv2eA84g]]"},"status":400}

@TactiC
Copy link

TactiC commented Oct 25, 2019

He nitingadekar
Opening and closing indices on AWS elasticsearch is not supported.
https://docs.aws.amazon.com/en_pv/elasticsearch-service/latest/developerguide/aes-supported-es-operations.html

@stefan-cross
Copy link

As an update on opening and closing indicies on AWS ES/OpenSearch, this is supported as of V7.4

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/supported-operations.html#version_7_4

@lx-se
Copy link

lx-se commented Nov 2, 2022

But we still need to reindex. such settings doesnt apply to existing data :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment