Skip to content

Instantly share code, notes, and snippets.

@cdgraff
Last active September 25, 2023 23:53
Show Gist options
  • Star 7 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
  • Save cdgraff/8578424 to your computer and use it in GitHub Desktop.
Save cdgraff/8578424 to your computer and use it in GitHub Desktop.
logstash filter pattern for Icecast2
input {
file {
path => "/var/log/icecast/access.*"
type => "icecast"
start_position=>"beginning" # this be to import old logs
}
}
filter {
if [type] == "icecast" {
grok {
match => { "message" => "%{COMBINEDAPACHELOG} %{NUMBER:duration}" }
}
# this part is important to enable correct filters, as numbers, by default all grok result come as Strings
mutate {
convert => [ "bytes" ,"integer" ]
convert => [ "response", "integer" ]
convert => [ "duration", "integer" ]
}
date {
match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
target => "@timestamp"
}
if [auth] == "-" {
geoip {
source => "clientip"
target => "geoip"
# Can download from here: http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
database => "/etc/logstash/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float"]
}
if [agent] != "-" and [agent] != "" {
useragent {
source => "agent"
target => "ua"
add_tag => [ "UA" ]
}
}
if "UA" in [tags] {
if [device] == "Other" { mutate { remove_field => "device" } }
if [name] == "Other" { mutate { remove_field => "name" } }
if [os] == "Other" { mutate { remove_field => "os" } }
}
} else {
drop { }
}
}
}
output{
elasticsearch {
host => "elasticsearch.server.com"
}
}
@edglazer
Copy link

Minor issue, but you might want to change from "logstash filter pattern for Icecest2"
to
"logstash filter pattern for Icecast2"

@edglazer
Copy link

Thanks for posting this, btw!

@cdgraff
Copy link
Author

cdgraff commented Nov 19, 2015

thanks @edglazer, title updated, to better indexing ;)

Btw, are you using this? i'll love to listen about how you use...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment