Skip to content

Instantly share code, notes, and snippets.

@clay584
Created October 15, 2014 03:15
Show Gist options
  • Save clay584/5a75009ad571af3d0648 to your computer and use it in GitHub Desktop.
Save clay584/5a75009ad571af3d0648 to your computer and use it in GitHub Desktop.
Cisco Log Grok Expression
input {
file {
#if you are reading files that syslog-ng has written to.
path => ["/var/log/syslog-ng.log"]
type => "syslog"
tags => [ "network" ]
}
tcp {
#if syslog-ng is relaying to logstash on TCP/514
port => 514
type => "syslog"
tags => [ "network" ]
}
udp {
#if syslog-ng is relaying to logstash on UDP/514
port => 514
type => "syslog"
tags => [ "network" ]
}
} #end input block
filter {
#classify network syslog logs as cisco or other
if "network" in [tags] {
grep {
drop => false
match => [ "message", "%\S+-[0-9]-\S+:" ]
add_tag => [ "cisco" ]
}
} else {
mutate {
add_tag => [ "unknown_log" ]
}
}
if "cisco" in [tags] {
grok {
patterns_dir => "/opt/logstash/patterns"
#2014-06-26T18:05:06-07:00 10.8.60.66 62: Jun 26 18:05:05.129 PDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/6, changed state to up
#The standard CISCOTIMESTAMP pattern does not patch the timezone so you will need to add the following pattern to a new pattern file in <logstash_home>/patterns/custom
#CISCOTIMESTAMPTZ %{MONTH} +%{MONTHDAY}(?: %{YEAR})? %{TIME} %{TZ}
match => [ "message", "%{TIMESTAMP_ISO8601:syslog_ng_timestamp} %{IPORHOST:original_log_host} %{POSINT:cisco_number:int}: %{CISCOTIMESTAMPTZ:cisco_timestamp}: %%{DATA:facility}-%{POSINT:severity:int}-%{DATA:mnemonic}: %{GREEDYDATA:log_message}" ]
}
date {
timezone => "America/Los_Angeles"
#matches Cisco date/timestamps with timezone included. i.e - Jun 26 18:05:05.129 PDT
match => [ "cisco_timestamp", "MMM dd HH:mm:ss.SSS zzz" ]
}
if "_grokparsefailure" not in [tags] {
#if grok parse was successful, then delete message field as we have already extracted the data into individual fields, and this would be redundant and ultimately
#would take up twice the storage space. But if grok parse failed, we want to keep the message field so we can correct our match expression.
mutate {
delete_field => [ "message" ]
}
}
} else {
#do some stuff on logs that are not cisco logs
}
} #end filter block
output {
#send to elasticsearch or somewhere else
}
@metabsd
Copy link

metabsd commented Aug 17, 2017

can you share /opt/logstash/patterns for cisco pattern please.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment