Skip to content

Instantly share code, notes, and snippets.

@micheleorsi
Created January 22, 2015 10:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save micheleorsi/b7429e76eef4ba403637 to your computer and use it in GitHub Desktop.
Save micheleorsi/b7429e76eef4ba403637 to your computer and use it in GitHub Desktop.
Logstash configuration with Google App Engine logs
input {
file {
type => "gae"
path => ["/home/ubuntu/gae_logs/**/*log"]
tags => ["some","tags"]
stat_interval => 30
discover_interval => 180
}
}
filter {
grok {
match => { "message" => "%{COMMONAPACHELOG} (%{QS:referrer}|-) (%{QS:agent}|-) %{QS:hostname} ms=%{NUMBER:ms} cpu_ms=%{NUMBER:cpu_ms} (cpm_usd=%{NUMBER:cpm_usd}\s)*(loading_request=%{NUMBER:loading_request}\s)*(pending_ms=%{NUMBER:pending_ms}\s)*instance=%{WORD:instance} app_engine_release=(?<app_engine_release>[0-9].[0-9].[0-9]) trace_id=%{WORD:trace_id}" }
}
date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}
output {
stdout { codec => rubydebug }
s3 {
access_key_id => "<access-key>"
secret_access_key => "<secret-access-key>"
bucket => "<bucket-name>"
size_file => 16777216
time_file => 1
format => "plain"
canned_acl => "private"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment