ElasticSearch config for a write-heavy cluster

elasticsearch.yml

##################################################################
# /etc/elasticsearch/elasticsearch.yml
#
# Base configuration for a write heavy cluster
#

# Cluster / Node Basics
cluster.name: logng

# Node can have abritrary attributes we can use for routing
node.name: logsearch-01
node.datacenter: amsterdam

# Force all memory to be locked, forcing the JVM to never swap
bootstrap.mlockall: true

## Threadpool Settings ##

# Search pool
threadpool.search.type: fixed
threadpool.search.size: 20
threadpool.search.queue_size: 100

# Bulk pool
threadpool.bulk.type: fixed
threadpool.bulk.size: 60
threadpool.bulk.queue_size: 300

# Index pool
threadpool.index.type: fixed
threadpool.index.size: 20
threadpool.index.queue_size: 100

# Indices settings
indices.memory.index_buffer_size: 30%
indices.memory.min_shard_index_buffer_size: 12mb
indices.memory.min_index_buffer_size: 96mb

# Cache Sizes
indices.fielddata.cache.size: 15%
indices.fielddata.cache.expire: 6h
indices.cache.filter.size: 15%
indices.cache.filter.expire: 6h

# Indexing Settings for Writes
index.refresh_interval: 30s
index.translog.flush_threshold_ops: 50000

# Minimum nodes alive to constitute an operational cluster
discovery.zen.minimum_master_nodes: 2

# Unicast Discovery (disable multicast)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: [ "logsearch-01", "logsearch-02", "logsearch-03" ]

Is this a good working ES config or something you're testing?

Is this a good working ES config or something you're testing?

I will try

I will try

it is from this blog post: http://edgeofsanity.net/article/2012/12/26/elasticsearch-for-logging.html

it is from this blog post: http://edgeofsanity.net/article/2012/12/26/elasticsearch-for-logging.html

there is no "good settings", it always depends of your data and use cases, never copy/paste something you found on a blog with title "how handle billion document with elasticsearch" ^^

Try -> benchmark -> try -> benchmark

there is no "good settings", it always depends of your data and use cases, never copy/paste something you found on a blog with title "how handle billion document with elasticsearch" ^^

Try -> benchmark -> try -> benchmark