Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jordansissel/c3d65c5c6e1ef2ca23e8603ef4750036 to your computer and use it in GitHub Desktop.
Save jordansissel/c3d65c5c6e1ef2ca23e8603ef4750036 to your computer and use it in GitHub Desktop.
Massive Parse Tree Failure in Logstash 5.0.0-alpha3
# Example packet:
% echo -n '<123>Aug 4 07:54:35 foo bar: { "hello": "world" }' | nc -vu 127.0.0.1 9514
% bin/logstash --version
logstash 5.0.0-alpha4
% bin/logstash -f logstash-config-broken.conf
Pipeline main started
{
"_data" => {
"hello" => "world"
},
"syslog_severity_code" => 3,
"syslog_facility" => "clock",
"syslog_facility_code" => 15,
"program" => "bar",
"message" => "<123>Aug 4 07:54:35 foo bar: { \"hello\": \"world\" }",
"type" => "syslog",
"logsource" => "foo",
"content" => "{ \"hello\": \"world\" }",
"syslog_severity" => "error",
"tags" => [
[0] "data",
[1] "json"
],
"@timestamp" => 2016-08-04T14:59:46.981Z,
"@version" => "1",
"host" => "127.0.0.1",
"timestamp" => "Aug 4 07:54:35"
}
input {
udp {
host => "127.0.0.1"
port => 9514
type => "syslog"
}
}
filter {
# This grok FAILS with a PARSE ERROR
grok {
match => { "message" => "<%{NONNEGINT:syslog_pri}>%{SYSLOGBASE} %{GREEDYDATA:content}" }
add_tag => "grok"
}
if "grok" in [tags] {
syslog_pri {}
}
mutate {
remove_field => [ "syslog_pri" ]
remove_tag => [ "grok" ]
}
}
filter {
if [content] {
json {
target => "[_data]"
source => "content"
add_tag => [ "data", "json" ]
}
}
}
filter {
if [program] == "ossec" and [_data] {
if [_data][component] {
grok {
match => { "[_data][component]" => "\(%{IPORHOST:src}\) %{WORD}->%{GREEDYDATA:src_file}" }
add_tag => [ "ossec_valid" ]
}
}
}
}
filter {
mutate {
remove_field => [ "_json" ]
}
}
output {
stdout {
codec => "rubydebug"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment