Skip to content

Instantly share code, notes, and snippets.

@josue
Created October 3, 2014 00:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save josue/08e5cace2e0196d8e6e0 to your computer and use it in GitHub Desktop.
Save josue/08e5cace2e0196d8e6e0 to your computer and use it in GitHub Desktop.
ElasticSearch - backup production index from host (1) to host (2) with new index
#!/bin/sh
# Required: npm install elasticdump
# Usage: ./{script} {host-1} {index-to-backup} {host-2} {new-index}
ES_OLD_HOST=$1
ES_OLD_INDEX=$2
ES_NEW_HOST=$3
ES_NEW_INDEX=$4
MAP_FILE="`echo $ES_OLD_INDEX`_mapping.json"
DATA_FILE="`echo $ES_OLD_INDEX`_data.json"
MAP_FILE_CLEAN="`echo $MAP_FILE`_clean"
DATA_FILE_CLEAN="`echo $DATA_FILE`_clean"
ES_NEW_INDEX_PRODUCTION="`echo $ES_NEW_INDEX`_`date +"%Y_%m_%d"`_production_backup"
echo "# backup indices to json files"
./node_modules/.bin/elasticdump --input=$ES_OLD_HOST/$ES_OLD_INDEX --output=$MAP_FILE --type=mapping
./node_modules/.bin/elasticdump --input=$ES_OLD_HOST/$ES_OLD_INDEX --output=$DATA_FILE --type=data
echo "# rename old index to new index for importing mapping/data"
ES_OLD_R1="`echo $ES_OLD_INDEX`_\([0-9\_]\+\)"
cat $MAP_FILE | sed -e "s/\\\"$ES_OLD_R1\\\"/\\\"$ES_NEW_INDEX_PRODUCTION\\\\\"/g" > $MAP_FILE_CLEAN
cat $DATA_FILE | sed -e "s/\"_index\":\"$ES_OLD_R1\"/\"_index\":\"$ES_NEW_INDEX_PRODUCTION\"/g" > $DATA_FILE_CLEAN
echo "# import entities mapping"
./node_modules/.bin/elasticdump --bulk=true --input=$MAP_FILE_CLEAN --output=$ES_NEW_HOST
echo "# import entities data"
./node_modules/.bin/elasticdump --bulk=true --input=$DATA_FILE_CLEAN --output=$ES_NEW_HOST
echo "# create alias: $ES_NEW_INDEX_PRODUCTION -> $ES_NEW_INDEX"
POST_JSON="{\"actions\":[{\"add\":{\"index\":\"$ES_NEW_INDEX_PRODUCTION\",\"alias\":\"$ES_NEW_INDEX\"}}]}"
curl -XPOST "$ES_NEW_HOST/_aliases" -d "$POST_JSON"
echo "# removing json files"
rm *.json*
@josue
Copy link
Author

josue commented Oct 3, 2014

From any box inside the vpn, how to run script (example):

mkdir /tmp/es_dump && cd $_
npm install elasticdump
time ./es_backup_index_to_new_host.sh {host-1} {index-to-backup} {host-2} {new-index}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment