Elasticsearch "blocked by: [FORBIDDEN/12/index read-only / allow delete (api)" error

elastic_blocked_index.md

If you are getting the following error while indexing:

{
  "error": {
    "root_cause": [
      {
        "type": "cluster_block_exception",
        "reason": "blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"
      }
    ],
    "type": "cluster_block_exception",
    "reason": "blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"
  },
  "status": 403
}

Probably you are getting the following error in the logs:

flood stage disk watermark [95%] exceeded on [xxxxxxxx][yyyyy][zzzzzzzzz] free: 5.1gb[4.1%], all indices on this node will be marked read-only

It can be caused due to a low disk space, check it and then remove the lock OR you can remove the threshold:

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_cluster/settings -d '{ "transient": { "cluster.routing.allocation.disk.threshold_enabled": false } }'
curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": false}'