Skip to content

Instantly share code, notes, and snippets.

@ocean
Last active August 29, 2015 14:10
Show Gist options
  • Save ocean/fcd1b3ec4c29f9eaa654 to your computer and use it in GitHub Desktop.
Save ocean/fcd1b3ec4c29f9eaa654 to your computer and use it in GitHub Desktop.
Logstash WILL NOT parse my files :`-(
# custom regex pattern for our URIs, allowing
# ampersands in the URL and ^s in the query string.
URI_EXTRA %{URIPROTO}://(?:%{USER}(?::[^@]*)?@)?(?:%{URIHOST})?(?:(?:/[A-Za-z0-9$.+!*'(){},~:;=&@#%_\-]*)+(?:\?[A-Za-z0-9$.+!*'|(){},~@#%&\^/=:;_?\-\[\]]*)?)?
} else if [path] =~ "drupal" {
mutate { replace => { "type" => "drupal_syslog" } }
grok {
break_on_match => false
match => [
"message", "%{SYSLOGBASE} %{GREEDYDATA:syslog_message}"
]
}
grok {
patterns_dir => "/Users/drewrobinson/Development/elk/logstash/patterns"
# break_on_match => false
match => [
"syslog_message", "%{URI:server}\|%{INT:unix_time}\|%{DATA:message_type}\|(%{IP:client})?\|%{URI_EXTRA:request}\|(%{URI_EXTRA:referer})?\|%{INT:uid}\|(%{DATA:link})?\|%{GREEDYDATA:log_message}"
]
}
grok {
patterns_dir => "/Users/drewrobinson/Development/elk/logstash/patterns"
# break_on_match => false
match => [
"syslog_message", "%{DATA:message_type}\|(%{IP:client})?\|%{URI_EXTRA:request}\|(%{URI_EXTRA:referer})?\|%{INT:uid}\|(%{DATA:link})? %{GREEDYDATA:log_message}"
]
}
grok {
patterns_dir => "/Users/drewrobinson/Development/elk/logstash/patterns"
# break_on_match => false
match => [
"syslog_message", "type=(%{QS:message_type}|%{DATA:message_type}) client=(%{IP:client})? request_uri=%{URI_EXTRA:request} referer=(%{URI_EXTRA:referer})? uid=%{INT:uid} link=(%{QS:link}|%{DATA:link})? message=%{GREEDYDATA:log_message}"
]
}
@ocean
Copy link
Author

ocean commented Nov 21, 2014

And I've tried with break_on_match as false and true

@ocean
Copy link
Author

ocean commented Nov 21, 2014

Changed the grok config to the above and it's not matching anything, yay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment