Skip to content

Instantly share code, notes, and snippets.

View renshuki's full-sized avatar
💭
🇫🇷 🇯🇵 🇺🇸

Francois-Clement Brossard renshuki

💭
🇫🇷 🇯🇵 🇺🇸
22 followers · 37 following
View GitHub Profile
@renshuki
renshuki / logstash.service
Created July 17, 2019 10:09
Logstash 7.x Systemd startup script
[Unit]
Description=logstash
[Service]
Type=simple
User=logstash
Group=logstash
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
@renshuki
renshuki / kibana.service
Created July 17, 2019 10:08
Kibana 7.x Systemd startup script
[Unit]
Description=Kibana
StartLimitIntervalSec=30
StartLimitBurst=3
[Service]
Type=simple
User=kibana
Group=kibana
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
@renshuki
renshuki / elasticsearch_watcher_percentile_ranks.md
Created July 8, 2019 10:49
Elasticsearch Watcher example using "Percentile Ranks"

This watcher trigger an alert when less than 80% of page responses are under 500ms. (Time range not present in the sample below and need to be added )

Index sample

POST latency/_doc
{
  "response_time": 100
}
@renshuki
renshuki / curator_delete_elastic_cloud_snapshots.md
Last active July 2, 2019 13:19
How to use Curator to automatically delete snapshots on Elastic Cloud (Elasticsearch Service)

Curator configuration

curator.yml

client:
  hosts:
    - https://your_cluster_url.cloud.es.io
  port: 9243
 use_ssl: True
@renshuki
renshuki / elasticsearch_set_search_index_slow_logs.md
Created July 1, 2019 07:10
Example on how to set Search and Index slow logs for an Elasticsearch index

slowlogs is the name of the index

PUT slowlogs/_settings
{
  "index": {
    "search": {
      "slowlog": {
        "level": "warn",
 "threshold": {
@renshuki
renshuki / reset_fix_monitoring_elastic_cloud.md
Created June 21, 2019 16:00
Reset / FIX Monitoring on Elastic Cloud (Elasticsearch Service) 
PUT _cluster/settings
{
  "persistent": {
    "xpack": {
      "monitoring": {
        "collection": {
          "enabled": "true",
          "interval": "10s"
 },
@renshuki
renshuki / kibana_create_index_pattern_curl_command_line.md
Last active June 5, 2019 08:42
Create a Kibana index pattern via cURL command line 
curl -XPOST -u elastic:password "http://localhost:5601/api/saved_objects/index-pattern/my-index-pattern-*" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" -d '
{
  "attributes": {
    "title": "my-index-pattern-*"
  }
}
'
@renshuki
renshuki / elasticsearch_slowlog_for_all_indices.md
Created May 11, 2019 08:51
How to enable slowlogs for all indices and tweak slowlog logger level

Slowlog thresholds for all indices

Add slowlog thresholds for all indices

PUT /_all/_settings 
{
"index.search.slowlog.threshold.query.warn": "10s",
"index.search.slowlog.threshold.query.info": "5s",
"index.search.slowlog.threshold.query.debug": "2s",
@renshuki
renshuki / elasticsearch_watcher_logstash_monitoring_events.md
Last active August 14, 2020 09:51
Elasticsearch Threshold Alert (Watcher) to check Logstash instance monitoring incoming events

This watcher check every 5 minutes if the number of Logstash monitoring events are below 1 which means Logstash monitoring is in trouble.

"watch" : {
    "trigger" : {
      "schedule" : {
        "interval" : "5m"
      }
    },
    "input" : {
@renshuki
renshuki / elasticsearch_watcher_logstash_heap_usage.md
Last active May 8, 2019 07:31
Elasticsearch Threshold Alert (Watcher) to monitor Logstash instance heap usage

This watcher check every 5 minutes if the average heap usage of the Logstash instance is above 80%.

An action could be added to send emails automatically

  "watch" : {
    "trigger" : {
      "schedule" : {
        "interval" : "5m"
      }