Skip to content

Instantly share code, notes, and snippets.

@reyjrar
Last active August 4, 2016 18:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save reyjrar/39fdd055c94b6c08d5a92aa687e2f2a3 to your computer and use it in GitHub Desktop.
Save reyjrar/39fdd055c94b6c08d5a92aa687e2f2a3 to your computer and use it in GitHub Desktop.
Massive Parse Tree Failure in Logstash 5.0.0-alpha3
input {
udp {
host => "127.0.0.1"
port => 9514
type => "syslog"
}
}
filter {
# This grok FAILS with a PARSE ERROR
grok {
match => { "message" => "<%{NONNEGINT:syslog_pri}>%{SYSLOGBASE} %{GREEDYDATA:content}" }
add_tag => "grok"
}
if "grok" in [tags] {
syslog_pri {}
}
mutate {
remove_field => [ "syslog_pri" ]
remove_tag => [ "grok" ]
}
}
filter {
if [content] {
json {
target => "[_data]"
source => "content"
add_tag => [ "data", "json" ]
}
}
}
filter {
if [program] == "ossec" and [_data] {
if [_data][component] {
grok {
match => { "[_data][component]" => "\(%{IPORHOST:src}\) %{WORD}->%{GREEDYDATA:src_file}" }
add_tag => [ "ossec_valid" ]
}
}
}
}
filter {
mutate {
remove_field => [ "_json" ]
}
}
output {
stdout {
codec => "rubydebug"
}
}
input {
udp {
host => "127.0.0.1"
port => 9514
type => "syslog"
}
}
filter {
grok {
match => { "message" => "<%{NONNEGINT:syslog_pri}>%{SYSLOGBASE} %{GREEDYDATA:content}" }
add_tag => "grok"
}
if "grok" in [tags] {
syslog_pri {}
}
mutate {
remove_field => [ "syslog_pri" ]
remove_tag => [ "grok" ]
}
}
filter {
if [content] {
json {
target => "[_data]"
source => "content"
add_tag => [ "data", "json" ]
}
}
}
filter {
if [program] == "ossec" and [_data] {
if [_data][component] {
# ADDING this mutate causes the previous GROKPARSE error to go away, docs are parsed,
# BUT NOTHING happens to the message in this mutate, ie, the tag is not added, but the
# rest of the config works
mutate {
add_tag => [ "noop" ]
}
grok {
match => { "[_data][component]" => "\(%{IPORHOST:src}\) %{WORD}->%{GREEDYDATA:src_file}" }
add_tag => [ "ossec_valid" ]
}
}
}
}
filter {
mutate {
remove_field => [ "_json" ]
}
}
output {
stdout {
codec => "rubydebug"
}
}
@reyjrar
Copy link
Author

reyjrar commented Jul 28, 2016

Possible Explanation: "Two filter operators become quantum entangled. Only one may run successfully, but both must be present for either to run successfully."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment