Logstash parser for ModSecurity/CRS entries in the Apache ErrorLog

gistfile1.txt

Logstash Configuration...

input {
  file {
    format => "plain"
    path => "/var/log/apache2/*error.log"
    type => "apacheerror"
  }
}
filter {
  grok {
    type => "apacheerror"
    pattern => [ "%{MODSECAPACHEERROR}", "%{GENERICAPACHEERROR}" ]
    patterns_dir => "/path/to/patterns/file/below"
  }
  date {
    type => "apacheerror"
    timestamp => "EEE MMM dd HH:mm:ss yyyy"
  }
}

Grok patterns...

APACHEERRORTIME %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}
APACHEERRORPREFIX \[%{APACHEERRORTIME:timestamp}\] \[%{NOTSPACE:apacheseverity}\] \[client %{IPORHOST:sourcehost}\]
GENERICAPACHEERROR %{APACHEERRORPREFIX} %{GREEDYDATA:message}

MODSECPREFIX %{APACHEERRORPREFIX} ModSecurity: %{NOTSPACE:modsecseverity}\. %{GREEDYDATA:modsecmessage}
MODSECRULEFILE \[file %{QUOTEDSTRING:rulefile}\]
MODSECRULELINE \[line %{QUOTEDSTRING:ruleline}\]
MODSECMATCHOFFSET \[offset %{QUOTEDSTRING:matchoffset}\]
MODSECRULEID \[id %{QUOTEDSTRING:ruleid}\]
MODSECRULEREV \[rev %{QUOTEDSTRING:rulerev}\]
MODSECRULEMSG \[msg %{QUOTEDSTRING:rulemessage}\]
MODSECRULEDATA \[data %{QUOTEDSTRING:ruledata}\]
MODSECRULESEVERITY \[severity %{QUOTEDSTRING:ruleseverity}\]
MODSECRULETAGS (?:\[tag %{QUOTEDSTRING:ruletag0}\] )?(?:\[tag %{QUOTEDSTRING:ruletag1}\] )?(?:\[tag %{QUOTEDSTRING:ruletag2}\] )?(?:\[tag %{QUOTEDSTRING:ruletag3}\] )?(?:\[tag %{QUOTEDSTRING:ruletag4}\] )?(?:\[tag %{QUOTEDSTRING:ruletag5}\] )?(?:\[tag %{QUOTEDSTRING:ruletag6}\] )?(?:\[tag %{QUOTEDSTRING:ruletag7}\] )?(?:\[tag %{QUOTEDSTRING:ruletag8}\] )?(?:\[tag %{QUOTEDSTRING:ruletag9}\] )?(?:\[tag %{QUOTEDSTRING}\] )*
MODSECHOSTNAME \[hostname %{QUOTEDSTRING:targethost}\]
MODSECURI \[uri %{QUOTEDSTRING:targeturi}\]
MODSECUID \[unique_id %{QUOTEDSTRING:uniqueid}\]

MODSECAPACHEERROR %{MODSECPREFIX} %{MODSECRULEFILE} %{MODSECRULELINE} (?:%{MODSECMATCHOFFSET} )?(?:%{MODSECRULEID} )?(?:%{MODSECRULEREV} )?(?:%{MODSECRULEMSG} )?(?:%{MODSECRULEDATA} )?(?:%{MODSECRULESEVERITY} )?%{MODSECRULETAGS}%{MODSECHOSTNAME} %{MODSECURI} %{MODSECUID}

Thanks for the patterns, it got me going on a logstash config for the full modsecurity audit logs, https://github.com/bitsofinfo/logstash-modsecurity

There is a bug in the date filter. Apache error uses milliseconds and the timestamp format should be:

EEE MMM dd HH:mm:ss.SSSSSS yyyy

If not logstash throws a warning because cannot parse the date.