cheatsheet.elasticsearch.md

Elasticsearch Cheat Sheet

• Common queries
• When you need to reboot at least one node
• Explain why a shard is unallocated
• Fix an unallocated primary shard
• Force rerouting allocation of a shard
• Change number of replicas
• Change the refresh interval
• Increase recovery speed
• Update mapping dynamic setting
• Set transaction log (TRANSLOG) size
• Create a template
• Create a demo index

Common queries

curl http://localhost:9200/_settings?pretty
curl http://localhost:9200/_cluster/settings?pretty
curl http://localhost:9200/_cat/health?v
curl http://localhost:9200/_cat/nodes?v
curl http://localhost:9200/_cat/indices?v
curl http://localhost:9200/_cat/shards?v
curl http://localhost:9200/_template

When you need to reboot at least one node

1. Disable shard allocation.

   curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/_cluster/settings' -d '{
       "transient": {
           "cluster.routing.allocation.enable": "none"
       }
   }'

2. Run a synced flush on all indexes.

   curl -XPOST 'http://localhost:9200/*/_flush/synced'

3. Stop one node at a time, do your business and restart ES. Wait until it becomes green again. Finish all nodes.

4. Turn rebalancing back on.

   curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/_cluster/settings' -d '{
       "transient": {
           "cluster.routing.allocation.enable": "all"
       }
   }'

Explain why a shard is unallocated

$ curl -XGET 'http://localhost:9200/_cluster/allocation/explain' -d'{
  "index": "indexname",
  "shard": 0,
  "primary": true
}'

Fix an unallocated primary shard

Happens when a node temporarily leaves the cluster without replication turned on for the index.

curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/_cluster/reroute' -d '{
    "commands": [{
        "allocate_stale_primary": {
            "index": "indexname",
            "shard": 1,
            "node": "nodename",
            "accept_data_loss": true
        }
    }]
}'

Force rerouting allocation of a shard

Retry rerouting of failed shards:

curl -XPOST 'localhost:9200/_cluster/reroute?retry_failed

Change number of replicas

curl -X PUT "localhost:9200/INDEXNAME/_settings" -H 'Content-Type: application/json' -d'
{
    "index" : {
        "number_of_replicas" : 2
    }
}'

For all indexes:

curl -X PUT "localhost:9200/*/_settings" -H 'Content-Type: application/json' -d'
{
    "index" : {
        "number_of_replicas" : 2
    }
}'

Change the refresh interval

curl -X PUT "localhost:9200/INDEXNAME/_settings" -H 'Content-Type: application/json' -d'
{
    "index" : {
        "refresh_interval" : "1s"
    }
}'

For all indexes:

curl -X PUT "localhost:9200/*/_settings" -H 'Content-Type: application/json' -d'
{
    "index" : {
        "refresh_interval" : "1s"
    }
}'

Increase recovery speed

curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/_cluster/settings' -d '{
    "transient": {
        "indices.recovery.max_bytes_per_sec": "1000mb",
        "cluster.routing.allocation.node_concurrent_recoveries": "8"
    }
}'

Reset default settings:

curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/_cluster/settings' -d '{
    "transient": {
        "indices.recovery.max_bytes_per_sec": null,
        "cluster.routing.allocation.node_concurrent_recoveries": null
    }
}'

Update mapping dynamic setting

curl -X PUT "localhost:9200/INDEX/_mapping/_doc" -H 'Content-Type: application/json' -d'
{
  "dynamic": "false"
}
'

Valid values are: true, false, or strict.

Set transaction log (TRANSLOG) size

We don't want this setting to be too large or it makes shard relocation take FOREVER. It was previously set to 10GB for reference.

curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/v8_datasets_*/_settings' -d '{
    "index": {
        "translog.flush_threshold_size": "512mb"
    }
}'

Create a template

# Create a default template that applies to all indexes
curl -H 'Content-Type: application/json' -XPUT "localhost:9200/_template/template_default" -d'
{
  "template": "template_default",
  "index_patterns": ["*"],
  "order": 1,
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1,
    "refresh_interval": "10s"
  },
  "mappings": {
    "_doc": {
      "dynamic": "strict"
    }
  }    
}
'

# Remove template
curl -X DELETE "localhost:9200/_template/template_default"

Create a demo index

Download a sample data set and import it into ES.

# Download sample data
curl https://download.elastic.co/demos/kibana/gettingstarted/shakespeare_6.0.json >shakespeare.json

# Create data mapping
curl -H 'Content-Type: application/json' -XPUT "localhost:9200/shakespeare" -H 'Content-Type: application/json' -d'
{
 "mappings": {
  "doc": {
   "properties": {
    "speaker": {"type": "keyword"},
    "play_name": {"type": "keyword"},
    "line_id": {"type": "integer"},
    "speech_number": {"type": "integer"}
   }
  }
 }
}
'

# Bulk import
curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/doc/_bulk' --data-binary @shakespeare.json >shakespeare.output

# Remove index
curl -X DELETE "localhost:9200/shakespeare"