Skip to content

Instantly share code, notes, and snippets.

Avatar

Aaron Mildenstein untergeek

View GitHub Profile
@untergeek
untergeek / elasticsearch-template.json
Created Aug 7, 2014
My current logstash/elasticsearch-index template
View elasticsearch-template.json
{
"template" : "logstash-*",
"settings" : {
"number_of_shards" : 5,
"index.refresh_interval" : "5s"
},
"mappings" : {
"_default_" : {
"_all" : {"enabled" : true},
"dynamic_templates" : [ {
View logstash.excerpt
if [type] == "apache_json" {
geoip { source => "clientip" }
if [useragent] != "" { useragent { source => "useragent" } }
if [auth] == "-" { mutate { remove_field => "auth" } }
if [ident] == "-" { mutate { remove_field => "ident" } }
if [referer] == "-" { mutate { remove_field => "referer" } }
}
@untergeek
untergeek / crontab
Created Jul 22, 2014
curator script info
View crontab
30 2 * * * ~/bin/curator.sh &> /dev/null
View collectd-logstash.conf
output {
stdout { debug => true debug_format => "json"}
if [type] == 'collectd' {
elasticsearch {
cluster => "elasticsearch"
}
}
}
@untergeek
untergeek / b.conf
Last active Aug 29, 2015
Grok debug
View b.conf
input { stdin {} }
filter {
grok {
match => [ "message", "\[%{RUBY_LOGLEVEL:loglevel}\] %{TIMESTAMP_ISO8601:logdate} %{ISO8601_TIMEZONE:timezone} \[%{NOTSPACE:somefield}\] \[%{IP:ipaddress}\] %{GREEDYDATA:therest}" ]
}
mutate {
gsub => [ "logdate", "\ ", "T" ]
replace => { "logdate" => "%{logdate}%{timezone}" }
}
date {
@untergeek
untergeek / logstash.conf
Created Jul 17, 2014
grok rule oddity...
View logstash.conf
input {
stdin {
type => "weblogic-server"
}
}
filter {
### weblogic-server
if [type] == "weblogic-server" {
multiline {
@untergeek
untergeek / a.txt
Created Jul 15, 2014
Curator with python strftime datestamp support
View a.txt
buh@BigMini (06:12 PM) [~/GIT/curator/curator] $ curl -XPUT 'http://localhost:9200/logstash-20140710/'
{"acknowledged":true}
View command-line.help
$ python test.py --help
usage: test.py [-h] [-v] [--host HOST] [--url_prefix URL_PREFIX] [--port PORT]
[--ssl] [-t TIMEOUT] [-n] [-D] [--loglevel LOG_LEVEL]
[-l LOG_FILE]
{allocation,bloom,close,delete,optimize,show_indices,snapshot}
...
Curator for Elasticsearch indices. See
http://github.com/elasticsearch/curator/wiki
@untergeek
untergeek / curator.log
Last active Aug 29, 2015
curator snapshot functionality...
View curator.log
$ ./curator.py --host blackbox --timeout 3600 --repository CuratorTest --create-repo --location "/Volumes/Repositories/CuratorTest" --snap-older 2
2014-05-21T19:16:29.800 INFO main:580 Job starting...
2014-05-21T19:16:29.800 INFO _new_conn:257 Starting new HTTP connection (1): blackbox
2014-05-21T19:16:29.806 INFO log_request_success:49 GET http://blackbox:9200/ [status:200 request:0.005s]
2014-05-21T19:16:29.806 INFO main:609 Creating repository CuratorTest...
2014-05-21T19:16:29.809 WARNING log_request_fail:68 GET /_snapshot/CuratorTest [status:404 request:0.003s]
2014-05-21T19:16:29.809 INFO log_request_fail:70 > None
2014-05-21T19:16:29.809 INFO _get_repository:376 Repository CuratorTest not found
2014-05-21T19:16:29.857 INFO log_request_success:49 PUT http://blackbox:9200/_snapshot/CuratorTest [status:200 request:0.047s]
2014-05-21T19:16:29.857 INFO _create_repository:393 R
@untergeek
untergeek / kibana.json
Last active Mar 27, 2019
Simple Kibana dashboard for collectd stats
View kibana.json
{
"title": "Collectd: Blackbox",
"services": {
"query": {
"list": {
"0": {
"query": "plugin:\"load\"",
"alias": "Load",
"color": "#7EB26D",
"id": 0,