Skip to content

Instantly share code, notes, and snippets.

@tamros
Last active July 10, 2018 09:22
Show Gist options
  • Save tamros/1d4e00bfea7219e625a08a04d3a8456c to your computer and use it in GitHub Desktop.
Save tamros/1d4e00bfea7219e625a08a04d3a8456c to your computer and use it in GitHub Desktop.
Different ways to use ingest pipeline
#add the pipeline
PUT _ingest/pipeline/set_incremental
{
"processors": [
{
"set": {
"field": "incremental_value",
"value": 0
}
}
]
}
#---------------------
#using while ingesting
#indexing a document
PUT users/_doc/1/?pipeline=set_incremental
{
"email": "jane@me.com",
"user": "janedoe",
"name": "jane"
}
#check if the pipeline was excuted
GET users/_doc/1
#indexing using bulk
POST users/_doc/_bulk
{"index":{"_id":"2", "pipeline": "set_incremental"}}
{"email": "mack@me.com", "user": "mackdoe", "name": "mack"}
#check if the pipeline was excuted
GET users/_doc/2
#or
POST users/_doc/_bulk?pipeline=set_incremental
{"index":{"_id":"3"}}
{"email": "john@me.com", "user": "johndoe", "name": "john"}
#check if the pipeline was excuted
GET users/_doc/3
#----------------------
#adding a new processor
PUT _ingest/pipeline/add_incremental
{
"processors": [
{
"script": {
"source": """
ctx.incremental_value += 1;
"""
}
}
]
}
#---------------------
#using while updating
POST users/_update_by_query?pipeline=add_incremental
#check the value was added in all documents
GET users/_search
#---------------------
#using while reindex
POST _reindex
{
"source": {
"index": "users"
},
"dest": {
"index": "new_users",
"pipeline": "add_incremental"
}
}
#check the value was added in all documents in the new index
GET new_users/_search
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment