Skip to content

Instantly share code, notes, and snippets.

@gansbrest
Created October 21, 2013 19:56
Show Gist options
  • Save gansbrest/7089886 to your computer and use it in GitHub Desktop.
Save gansbrest/7089886 to your computer and use it in GitHub Desktop.
Sample grok entry
1382383761.140 38.96.155.45 - 2013-10-21T19:29:21+00:00 GET /sites/all/modules/admin/includes/admin.toolbar.js?Q HTTP/1.1 200 1616 "http://www.fastcompany.com/account/skhaladzinski" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/28.0.1500.71 Chrome/28.0.1500.71 Safari/537.36" US 0.001 "-" www.fastcompany.com 6dfca709
input {
file {
'path' => '/path/to/file.log'
'type' => 'fcd6_nginx_access'
}
}
filter {
grok {
'patterns_dir' => '/opt/logstash/agent/etc/patterns'
'add_tag' => ['fcd6_origin_extracted']
'match' => ['message', '%{NUMBER:msec} %{IP:client_ip} - %{TIMESTAMP_ISO8601,time} %{WORD:method} %{URIPATHPARAM:uri} (?:HTTP/%{NUMBER:version}) %{NUMBER:status} (?:%{NUMBER:bytes_sent}) %{QS:referer} %{QS:agent} %{WORD,country_code} %{NUMBER,upstream_resp_time} %{QS:http_fwd_for} %{HOST:http_host} %{WORD:instance_id}']
'type' => 'fcd6_nginx_access'
}
}
@mjballard
Copy link

You've got a lot of extra space that you aren't taking into account.
Try:
%{NUMBER:msec}\s+%{IP:client_ip}\s+-\s+%{TIMESTAMP_ISO8601,time}\s+%{WORD:method}\s+%{URIPATHPARAM:uri}\s+(?:HTTP/%{NUMBER:version})\s+%{NUMBER:status}\s+(?:%{NUMBER:bytes_sent})\s+%{QS:referer}\s+%{QS:agent}\s+%{WORD,country_code}\s+%{NUMBER,upstream_resp_time}\s+%{QS:http_fwd_for}\s+%{HOST:http_host}\s+%{WORD:instance_id}

Great website for testing grok patterns: http://grokdebug.herokuapp.com/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment