Skip to content

Instantly share code, notes, and snippets.

View markwalkom's full-sized avatar

Mark Walkom markwalkom

57 followers · 8 following
  • @warkolm
View GitHub Profile
@markwalkom
markwalkom / Step 1 - Elasticsearch and Kibana.md
Last active April 26, 2017 23:51
Monitoring Your Elastic Stack, with Beats

Download

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.1.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.3.1-darwin-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-5.3.1.zip

Extract

@markwalkom
markwalkom / Step 1 - Elasticsearch and Kibana.md
Last active April 26, 2017 06:46
Monitoring Your Elastic Stack with Beats

Download

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.1.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.3.1-darwin-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-5.3.1.zip

Extract

@markwalkom
markwalkom / README.md
Last active July 25, 2016 14:46
CollectD to ELK

This is an example of using ELK to parse and view collectd data.

Caveat - I haven't fully tested this mapping yet, it doesn't take into account any other fields that may be added with other collectd plugins, just the ones I have specified below.

@markwalkom
markwalkom / jqtips.md
Last active May 21, 2016 09:27
jq tips

Elasticsearch

Sum number of docs in a cluster

cat nodes_stats.json|jq '.nodes[].indices.docs.count'|awk '{s+=$0} END {print s}'

Sum total store size

cat nodes_stats.json|jq '.nodes[].indices.store.size_in_bytes'|awk '{s+=$0} END {print s}'

Working with the swapi data

Get a list of planets + key for translate lookup

cat people.json | jq -r '.[]|"\"\(.pk)\"" + ": " + "\"\(.fields.name)\""'

@markwalkom
markwalkom / missing-fields-query.json
Created May 18, 2016 05:13
Via Kibana, only show documents that have a missing field
# Via https://smelloworld.wordpress.com/2016/05/17/missing-fields-search-in-elasticsearch/
{“query”:{“filtered”:{“query”:{“match_all”:{}},”filter”:{“missing”:{“field”:”FIELDNAME”}}}}}
@markwalkom
markwalkom / Logstash all
Created February 5, 2016 03:44
$ logstash-2.2.0/bin/plugin list
logstash-codec-avro
logstash-codec-cef
logstash-codec-cloudfront
logstash-codec-cloudtrail
logstash-codec-collectd
logstash-codec-compress_spooler
logstash-codec-dots
logstash-codec-edn
logstash-codec-edn_lines
@markwalkom
markwalkom / kibana.conf
Last active December 31, 2015 19:29
Proxied kibana via nginx This saves your local desktop from connecting directly to the ES cluster.
server {
listen *:80;
server_name kibana.domain.com;
access_log /var/log/nginx/kibana_access.log;
error_log /var/log/nginx/kibana_error.log;
location /kibana {
root /var/www;
index index.html;
@markwalkom
markwalkom / gist:cd8b4a9f82c442079284
Created December 28, 2015 21:48
fail2ban patterns
F2B_DATE %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[ ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})
F2B_ACTION (\w+)\.(?:\w+)(\s+)?\:
F2B_JAIL \[(?<jail>\w+\-?\w+?)\]
F2B_LEVEL (?<level>\w+)\s+
@markwalkom
markwalkom / gist:f47a30e37cd402f2dc5d
Last active August 29, 2015 14:21
Export from ES to a json file
input {
elasticsearch {
hosts => [ "HOSTNAME_HERE" ]
port => "9200"
index => "INDEXNAME_HERE"
size => 500
scroll => "5m"
}
}
output {
@markwalkom
markwalkom / es-1.2-settings.md
Last active August 29, 2015 14:08 — forked from jprante/es-1.2-settings.md