Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
A grok pattern for Rails 3.2 logs for use with logstash. Assumes that you have a multiline filter to combine Rails logs into one line and only one worker is logging to a file (c.f. https://gist.github.com/mudge/5063930).
multiline {
tags => ["rails"]
pattern => "^Started"
negate => true
what => "previous"
}
RAILS3 (?m)Started %{WORD:verb} "%{URIPATHPARAM:request}" for %{IPORHOST:clientip} at (?<timestamp>%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second} %{ISO8601_TIMEZONE:timezone})\s*Processing by (?<controller>[^#]+)#(?<action>\w+) as (?<format>\S+)(?:\n Parameters: %{DATA:params}\n)?%{DATA}Completed %{NUMBER:response}%{DATA} in %{NUMBER:totalms}ms \(Views: %{NUMBER:viewms}ms \| ActiveRecord: %{NUMBER:activerecordms}ms%{GREEDYDATA}
@ese

This comment has been minimized.

Copy link

ese commented Apr 7, 2014

Some improvements:

RUUID \h{32}
RCONTROLLER (?<controller>[^#]+)#(?<action>\w+)
RAILS3HEAD (?m)Started %{WORD:verb} "%{URIPATHPARAM:request}" for %{IPORHOST:clientip} at (?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} %{ISO8601_TIMEZONE})
RPROCESSING \W*Processing by %{RCONTROLLER} as (?<format>\S+)(?:\W*Parameters: {%{DATA:params}}\W*)?
RAILS3FOOT Completed %{NUMBER:response}%{DATA} in %{NUMBER:totalms}ms %{RAILSPROFILE}%{GREEDYDATA}
RAILS3PROFILE (?:\(Views: %{NUMBER:viewms}ms \| ActiveRecord: %{NUMBER:activerecordms}ms|\(ActiveRecord: %{NUMBER:activerecordms}ms)?
RAILS3 %{RAILS3HEAD}(?:%{RPROCESSING})?(?<context>(?:%{DATA}\n)*)(?:%{RAILS3FOOT})?
@rhacker

This comment has been minimized.

Copy link

rhacker commented Apr 30, 2014

this is great. thanks guys !

@lwoodson

This comment has been minimized.

Copy link

lwoodson commented Aug 20, 2014

👍 Thanks.

@paulRbr

This comment has been minimized.

Copy link

paulRbr commented Nov 20, 2014

@ese thanks, there is a typo in your comment RAILSPROFILE doesn't exist

@ishaansutaria

This comment has been minimized.

Copy link

ishaansutaria commented Apr 8, 2015

can someone give more details into this...I have the ELK setup...I want to start with just basic rails log forwarded to the logstash server...which grok pattern should I use?

@kavimehta

This comment has been minimized.

Copy link

kavimehta commented Jun 3, 2016

What files do you edit and add the filter to?

@excalq

This comment has been minimized.

Copy link

excalq commented Dec 24, 2016

Here's a derivative version that I made, which uses Filebeat to combine multiline messages instead of Logstash (so LS groks the multiline message). Tested with :uuid enabled on Unicorn and Thin, with Rails 3.
https://gist.github.com/excalq/0c5023fce8af90089040daa6404525f2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.