Last active
August 29, 2015 13:58
-
-
Save spuder/10210424 to your computer and use it in GitHub Desktop.
logstash not sending to elasticsearch
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I have logstash 1.4 configured to take everything frim /tmp/access_log and stdin and send it to both elasticsearch and stdout | |
Documentation page | |
http://logstash.net/docs/1.4.0/tutorials/getting-started-with-logstash | |
input { | |
stdin { } | |
file { | |
path => "/tmp/access_log" | |
start_position => beginning | |
} | |
} | |
filter { | |
if [path] =~ "access" { | |
mutate { | |
replace => { "type" => "apache_access" } | |
} | |
grok { | |
match => { "message" => "%{COMBINEDAPACHELOG}" } | |
} | |
} | |
date { | |
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ] | |
} | |
} | |
output { | |
elasticsearch { | |
host => localhost | |
} | |
stdout { codec => rubydebug } | |
} | |
Interestingly, elasticsearch only shows the inputs from stdin (herp and fooy) | |
{ | |
"took" : 13, | |
"timed_out" : false, | |
"_shards" : { | |
"total" : 30, | |
"successful" : 30, | |
"failed" : 0 | |
}, | |
"hits" : { | |
"total" : 11944, | |
"max_score" : 1.0, | |
"hits" : [ { | |
"_index" : "logstash-2014.04.08", | |
"_type" : "logs", | |
"_id" : "UXVVemhOSYaWmPtjOrTmuQ", | |
"_score" : 1.0, "_source" : {"message":"","@version":"1","@timestamp":"2014-04-08T23:00:15.885Z","host":"interactive"} | |
}, { | |
"_index" : "logstash-2014.04.08", | |
"_type" : "logs", | |
"_id" : "8vnLI6c1SvCsHrapLooUzw", | |
"_score" : 1.0, "_source" : {"message":"herp","@version":"1","@timestamp":"2014-04-08T23:05:07.433Z","host":"interactive","tags":["_grokparsefailure"]} | |
}, { | |
"_index" : "logstash-2014.04.08", | |
"_type" : "logs", | |
"_id" : "h3tGy6OKT4K6qHJTpJDbPQ", | |
"_score" : 1.0, "_source" : {"message":"fooy","@version":"1","@timestamp":"2014-04-08T23:36:28.515Z","host":"interactive"} | |
} | |
... truncated a few erronious entries. | |
When I append apache log lines as suggested to /tmp_access.log, they show up on stdout, but not in elasticsearch. | |
Screenshot | |
http://cl.ly/image/2S3N302c3A1Q/Google%20Chrome.png | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment