Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save geekpete/50470f7de18b432b12116617448ce2d1 to your computer and use it in GitHub Desktop.
Save geekpete/50470f7de18b432b12116617448ce2d1 to your computer and use it in GitHub Desktop.
Logstash split field solution.
Problem: We have a log line that includes a perl class that we want to log the class
and method in their respected fields. An example class and method in perl:
Animal::Dog::bark
In this example, "bark" is the method. "Animal::Dog" is the class.
After some searching and hacking, I found a solution that works with Logstash 1.4.2
Assume the input is "Animal::Dog::bark".
For completeness, I'm going to just add my entire configuration file I used for testing.
Comments are included and should explain what's going on.
input {
stdin { }
}
filter {
grok {
match => { "message" => "%{GREEDYDATA:api_class}" }
}
mutate {
# split the field on ::
split => ["api_class" , "::"]
# save the last element of the array as the api_method.
add_field => ["api_method", "%{[api_class][-1]}" ]
}
ruby {
# Go directly to the array and remove the last element.
code => "event['api_class'].pop()"
}
mutate {
# Join together whats left as the class name.
join => ["api_class", "::"]
}
}
output {
stdout { codec => rubydebug }
}
I tried to use mutate's remove_field to remove the last element of the array but it didn't work.
There are tickets created and possibly even a fix in the new version, however, this should continue
to work as long as the ruby filter is around.
The output:
{
"message" => "Animal::Dog::bark",
"@version" => "1",
"@timestamp" => "2014-12-09T13:38:58.178Z",
"host" => "host.example.com",
"api_class" => "Animal::Dog",
"api_method" => "bark"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment