Skip to content

Instantly share code, notes, and snippets.

@spuder
Last active August 29, 2015 13:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save spuder/10210424 to your computer and use it in GitHub Desktop.
Save spuder/10210424 to your computer and use it in GitHub Desktop.
logstash not sending to elasticsearch
I have logstash 1.4 configured to take everything frim /tmp/access_log and stdin and send it to both elasticsearch and stdout
Documentation page
http://logstash.net/docs/1.4.0/tutorials/getting-started-with-logstash
input {
stdin { }
file {
path => "/tmp/access_log"
start_position => beginning
}
}
filter {
if [path] =~ "access" {
mutate {
replace => { "type" => "apache_access" }
}
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
}
date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}
output {
elasticsearch {
host => localhost
}
stdout { codec => rubydebug }
}
Interestingly, elasticsearch only shows the inputs from stdin (herp and fooy)
{
"took" : 13,
"timed_out" : false,
"_shards" : {
"total" : 30,
"successful" : 30,
"failed" : 0
},
"hits" : {
"total" : 11944,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2014.04.08",
"_type" : "logs",
"_id" : "UXVVemhOSYaWmPtjOrTmuQ",
"_score" : 1.0, "_source" : {"message":"","@version":"1","@timestamp":"2014-04-08T23:00:15.885Z","host":"interactive"}
}, {
"_index" : "logstash-2014.04.08",
"_type" : "logs",
"_id" : "8vnLI6c1SvCsHrapLooUzw",
"_score" : 1.0, "_source" : {"message":"herp","@version":"1","@timestamp":"2014-04-08T23:05:07.433Z","host":"interactive","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2014.04.08",
"_type" : "logs",
"_id" : "h3tGy6OKT4K6qHJTpJDbPQ",
"_score" : 1.0, "_source" : {"message":"fooy","@version":"1","@timestamp":"2014-04-08T23:36:28.515Z","host":"interactive"}
}
... truncated a few erronious entries.
When I append apache log lines as suggested to /tmp_access.log, they show up on stdout, but not in elasticsearch.
Screenshot
http://cl.ly/image/2S3N302c3A1Q/Google%20Chrome.png
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment