Skip to content

Instantly share code, notes, and snippets.

@wigsy
Created September 12, 2012 04:29
Show Gist options
  • Save wigsy/3704292 to your computer and use it in GitHub Desktop.
Save wigsy/3704292 to your computer and use it in GitHub Desktop.
Test Logstash GROK Filters
require 'rubygems'
require 'grok-pure'
# Set a new matcher
grok = Grok.new
# Load default and custom patterns
grok.add_patterns_from_file("/etc/logstash/patterns/default")
grok.add_patterns_from_file("/etc/logstash/patterns/mycustoms")
# Match until the cows come home.. not longer though.
# They will be hungry from the journey home and need a snack.
text = "Sep 11 22:44:00 i-03deb778 2012-09-11 22:44:00,581 INFO [108.220.77.198-HJc0AHyo2gEyZi3ukV7PjwAlzh8] [sql.engine.base.Engine] [MainThread] SELECT permissions.id AS permissions_id, permissions.name AS permissions_name, permissions.description AS permissions_description, permissions.selectable AS permissions_selectable, permissions.implied AS permissions_implied #012FROM permissions, roles_permissions #012WHERE %s = roles_permissions.role_id AND permissions.id = roles_permissions.permission_id"
# Try this match pattern out and see what shakes.
pattern = '%{SYSLOGTIMESTAMP:date} %{MC_HOST_ID:host} %{DATA} %{LOGLEVEL:loglevel}\s+\[%{MC_SESSION_ID:session_id}\] \[%{MC:sender}\] \[%{MC_THREAD:thread}\] %{GREEDYDATA:message}'
grok.compile(pattern)
puts grok.match(text).captures()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment