-
-
Save markwalkom/8a7201e3f6ea4354ae06 to your computer and use it in GitHub Desktop.
| input { | |
| elasticsearch { | |
| hosts => [ "HOSTNAME_HERE" ] | |
| port => "9200" | |
| index => "INDEXNAME_HERE" | |
| size => 1000 | |
| scroll => "5m" | |
| docinfo => true | |
| scan => true | |
| } | |
| } | |
| output { | |
| elasticsearch { | |
| hosts => [ "HOSTNAME_HERE" ] | |
| index => "%{[@metadata][_index]}" | |
| document_type => "%{[@metadata][_type]}" | |
| document_id => "%{[@metadata][_id]}" | |
| } | |
| stdout { | |
| codec => "dots" | |
| } | |
| } |
Hi, I use the default index naming "logstash-" for a daily index. I have altered the number of shards from the default 5 to 1. I need to re-index my indices. I don't want to re-index into a new index eg "logstash-new-" but instead I want the existing indices to end up being spread across their single shard (instead of the current 5 shards per index).
How can I use this logstash script to do this?
Is there a better way to do this - eg re-index into new indices eg "logstash-new-", delete the original "logstash-" indices, then re-index back into "logstash-" from the new "logstash-new-" indices?
Many thanks.
Reindex API is a nice option:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#_reindex_daily_indices
Also look into automatic scroll slicing that allows scrolls to be processed by multiple threads in parallel giving a nice speed boost.
Can anybody please explain that scroll option? I do reindex with logstash and it loops endlessly - the data from source index is randomly duplicated to output
Thanks for the comments @blavioe!