Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Logstash Elasticsearch Template
{
"template": "logstash-*",
"settings" : {
"number_of_shards" : 1,
"number_of_replicas" : 0,
"index" : {
"query" : { "default_field" : "@message" },
"store" : { "compress" : { "stored" : true, "tv": true } }
}
},
"mappings": {
"_default_": {
"_all": { "enabled": false },
"_source": { "compress": true },
"dynamic_templates": [
{
"string_template" : {
"match" : "*",
"mapping": { "type": "string", "index": "not_analyzed" },
"match_mapping_type" : "string"
}
}
],
"properties" : {
"@fields": { "type": "object", "dynamic": true, "path": "full" },
"@message" : { "type" : "string", "index" : "analyzed" },
"@source" : { "type" : "string", "index" : "not_analyzed" },
"@source_host" : { "type" : "string", "index" : "not_analyzed" },
"@source_path" : { "type" : "string", "index" : "not_analyzed" },
"@tags": { "type": "string", "index" : "not_analyzed" },
"@timestamp" : { "type" : "date", "index" : "not_analyzed" },
"@type" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}
curl -XPUT 'http://localhost:9200/_template/template_logstash/' -d @logstash-template.json
@thattommyhall

This comment has been minimized.

Copy link

commented Jan 18, 2013

Is this a pretty generic 'I was lazy and just used embedded but now I need it to be a bit quicker' config?

@kitchen

This comment has been minimized.

Copy link

commented Jan 23, 2013

@thattommyhall: I'll be using a slight variation of this template in production soon. I only adjusted the number of shards and replicas. Going from 1 shard to 2 shards cost me little in size but spreads out the data between my 2 ES nodes, and replicas: 1 means I can have redundancy)

@sunny2385

This comment has been minimized.

Copy link

commented Apr 12, 2013

Hi,
Could you please help converting this json into a yml format.
I wish to include these settings into elasticsearch.yml

Thanks

@porjo

This comment has been minimized.

Copy link

commented May 28, 2013

@sunny2385 create a directory called templates/ underneath your config directory and put the JSON file there. It will be auto-loaded next time Elasticsearch starts.

Note: there appears to be a bug whereby templates loaded from config are not visible (under some conditions) when using curl - see this post

@zhoutuo

This comment has been minimized.

Copy link

commented Jun 11, 2013

Could you explain "store" : { "compress" : { "stored" : true, "tv": true } } a little bit please, I could not found corresponding documentation

@rnavarro

This comment has been minimized.

Copy link

commented Jun 16, 2013

@eclipse9614 It's pretty basic, but the settings are mentioned here: http://www.elasticsearch.org/guide/reference/index-modules/store/

@gregmurphy

This comment has been minimized.

Copy link

commented Nov 29, 2013

Do you use Kibana to query your ES indexes?

I've tried your template against ES 0.90.5 and Kibana 3.0.0milestone4, and any indexes created using the template don't return any data to Kibana. The data is loaded into the index, and I run basic queries against it successfully myself, but if I try the query Kibana runs then no data is returned.

I'll carry on investigating myself, but was interested to know if you've seen the same problem.

@gregmurphy

This comment has been minimized.

Copy link

commented Dec 1, 2013

For anybody who is interested - the line that was preventing Kibana from querying the index is:
"query" : { "default_field" : "@message" },

Remove that, re-index and Kibana works fine.

@simonmulser

This comment has been minimized.

Copy link

commented Jul 7, 2014

FYI i added:
"numeric_detection" : true,
to get automatically numeric fields (integer, long, float...).

@thomasgogo

This comment has been minimized.

Copy link

commented Aug 21, 2015

@gregmruphy
how re-index? Please tell me the way

thanks !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.