Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 7 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mzpqnxow/85a95010bedf8915056582fcfa34c3e7 to your computer and use it in GitHub Desktop.
Save mzpqnxow/85a95010bedf8915056582fcfa34c3e7 to your computer and use it in GitHub Desktop.
## Description
Sometimes you need to parse CEF with logstash but it is wrapped in some other format, such as JSON or syslog. In this case, you can't use the CEF codec in the input, so you're doomed to do it yourself since there is not currently a CEF codec for filter blocks. Try this logstash recipe, it works well. Just rmeember a comma will break it- so feel free to replace "," with a more rare character like "|" or something...
## Context
At this point, the message has been processed so its a full CEF message starting in cef_message. Syslog "headers" were already removed in a previous step.
## Stripped from production
filter {
if "cef" in [tags] {
mutate {
# CEF:0 is pipe delimited, split into individual fields
split => ["cef_message", "|"]
add_field => { "cef_version" => "%{cef_message[0]}" }
add_field => { "cef_device_vendor" => "%{cef_message[1]}" }
add_field => { "cef_device_product" => "%{cef_message[2]}" }
add_field => { "cef_device_version" => "%{cef_message[3]}" }
add_field => { "cef_sig_id" => "%{cef_message[4]}" }
add_field => { "cef_sig_name" => "%{cef_message[5]}" }
add_field => { "cef_sig_severity" => "%{cef_message[6]}" }
add_field => { "cef_kv_message" => "%{cef_message[7]}" }
}
# These next two (mutate and kv) are where the CEF is parsed
# The mutate is required to bypass a limitation, the kv is
# pretty straightforward key/value pair matching. It splits
# them into key/value pairs and pops them into what will later
# be renamed a device specific namespace. For now it is just
# called `event_container`
# Workaround because `kv` doesn't support values that contain spaces
# Transform and then use comma to separate instead of spaces. Then
# make sure you act on commas instead of spaces in the kv
mutate {
gsub => ["cef_kv_message", "(\S+=)", ", \1"]
}
kv {
source => "cef_kv_message"
trim_value => " "
trim_key => " "
value_split => "="
field_split => ","
target => "event_container"
add_tag => "cef_kv_success"
# remove_field => ["tmp_message", "cef_kv_message", "message", "cef_message"]
remove_field => ["cef_kv_message", "message", "cef_message"]
remove_tag => ["kv_section"]
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment