elasticsearch bible

elasticsearch-bible.md

Elasticsearch Bible

For ease of use, use Kibana for running API requests

Backup

1. Create snapshot dir

mkdir -p /snapshots

2. Set owner of snapshot dir

chown -R elasticsearch. /snapshots

3. Set snapshot dir in config

cat >> /etc/elasticsearch/elasticsearch.yml << EOF
path.repo: ["/snapshots"]
EOF

4. Restart elasticsearch service

systemctl restart elasticsearch

5. Set up snapshot repo

PUT /_snapshot/logs
{
  "type": "fs",
  "settings": {
    "location": "/snapshots",
    "compress": true
  }
}

4. Create script to backup

#!/bin/bash
SNAPSHOT=`date +%Y%m%d-%H%M%S`
curl -XPUT "localhost:9200/_snapshot/logs/$SNAPSHOT?wait_for_completion=true"

5. Execute backup script

Restore

1. View all snapshots

GET /_snapshot/logs/_all?pretty

2. Close index

POST /logs/_close

3. Restore index

POST /_snapshot/<repo_name>/<snapshot_name>/_restore?wait_for_completion=true
{
	"indices": "logs"
}

4. Open index

POST /logs/_open

Clone index

1. Disable writes on old index

PUT /logs/_settings
{
  "settings": {
    "index.blocks.write": true
  }
}

2. Clone index

POST /logs/_clone/logs-new
{
  "settings": {
    "index.number_of_shards": 5
  }
}

3. Re-enable writes on old index

PUT /logs/_settings
{
  "settings": {
    "index.blocks.write": false
  }
}

Delete Index

DELETE /audit-logs

Update Mappings

First create temp index

curl -X POST http://localhost:9200/_reindex?wait_for_completion=true -H "Content-Type: application/json" \
  -d '{"source":{"index":"logs"},"dest":{"index":"logs-temp"}}'

curl -X PUT http://localhost:9200/logs-temp/_settings -H "Content-Type: application/json" \
  -d '{"settings":{"index.blocks.write":true}}'

curl -X DELETE http://localhost:9200/logs

curl -X POST http://localhost:9200/logs-temp/_clone/logs

curl -X DELETE http://localhost:9200/logs-temp

curl -X PUT http://localhost:9200/logs/_settings -H "Content-Type: application/json" \
  -d '{"settings":{"index.blocks.write":false}}'

Update By Query

Updating multiple fields (including objects)

curl -X POST http://localhost:9200/logs/_update_by_query?wait_for_completion=true -H "Content-Type: application/json" \
  -d '{"script":{"source":"ctx._source.result=true;ctx._source.created_boy=params.user;","lang":"painless","params":{"user":{"id":"1","name":"User 1"}}}}'

Add field

curl -X POST http://localhost:9200/logs/_update_by_query?wait_for_completion=true -H "Content-Type: application/json" \
  -d {"script":{"source":"ctx._source.new_field='value_of_new_field'"}}

Remove field

curl -X POST http://localhost:9200/logs/_update_by_query?wait_for_completion=true -H "Content-Type: application/json" \
  -d {"script":{"source":"ctx._source.remove('new_field')"}}

Update Indice Settings

Set results window

curl -X PUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -H "Content-Type: application/json" \
	-d '{"index.max_result_window" : "100000"}'

Set max clause count

curl -X PUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -H "Content-Type: application/json" \
	-d '{"indices.query.bool.max_clause_count" : "100000"}'

Security

Create API Key

POST /_security/api_key
{
  "name": "<key-name>",
  "role_descriptors": { 
    "<role-name>": {
      "cluster": ["all"],
      "index": [
        {
          "names": ["<index-name>"],
          "privileges": ["read"]
        }
      ]
    }
  }
}