Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Using PSAD and Logstash

PSAD and Logstash

Table of Contents

What are we doing here?

What is PSAD?

psad is a collection of three lightweight system daemons (two main daemons and one helper daemon) that run on Linux machines and analyze iptables log messages to detect port scans and other suspicious traffic. A typical deployment is to run psad on the iptables firewall where it has the fastest access to log data.

PSAD website

What is Logstash?

logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.

Logstash website

Using PSAD and Logstash

Using Logstash, you can process and tag PSAD events (logs) based on their significance. Also, as an alternative to using the PSAD email alerts, you could use one of Logstash' many output filters to send notifications of important events. For example, you could use Nagios to notify you of any port scans PSAD has detected. As the number of servers you install PSAD on grows, receiving email alerts from all of them can become cumbersome and could become a 'boy who cried wolf' situation. Logstash in combination with Kibana will let you chart all port scans and attempted attacks that PSAD can discover in a unified way.

This gist will explain how to configure both PSAD and Logstash to play nice together. This gist is focused on PSAD 2.2.4, Logstash 1.4.2 and Ubuntu Server 14.04; if you're using a different OS or different versions of Logstash/PSAD you may have to tweak the instructions a bit.

This gist assumes you have PSAD and Logstash up and running individually.

First, configure rsyslog

The first thing we need to do is configure rsyslog on Ubuntu to write any PSAD events to a separate logfile. Although it is perfectly possible to parse the /var/log/syslog directly for any PSAD messages, I prefer to write PSAD events to its own logfile; it makes it easier to determine its actions and makes our logstash configuration simpler as well, as we don't have to filter out any other kernel messages.

To do this, we need to tell rsyslog to send all messages that originate from PSAD to a different logfile. Create a file called /etc/rsyslog.d/30-psad.conf. The 30 is added to make sure the settings in this configuration file are parsed before the default settings, which start with 50-.

Add the following contents to the file:

if $programname == 'psad' then /var/log/psad.log # redirect psad log line to separate file
& stop # prevent log lines from application psad to be prcessed by any other filters

The first line ensures that any log lines from the application psad are sent to the file /var/log/psad.log. The second line ensures that any lines matching the previous criteria will not be processed by any other filters. This second line is needed to prevent the log lines from also showing up in your /var/log/messages or /var/log/kern.log file.

Note: with older versions of rsyslog, you may have to replace the word stop on the second line with the ~ character; on newer version of Ubuntu the ~ character as a stop indicator has been deprecated and will cause warning messages when (re)starting the rsyslog service.

Then, restart the rsyslog service to enable the changes:

sudo service rsyslog restart

Then, configure logstash patterns

To make processing PSAD messages easier, I have created a set of GROK patterns which should allow you to detect and act upon the various PSAD messages without too much effort.

First, you need to create a logstash patterns file. Depending on your logstash configuration, the location of these pattern files may vary but I store mine in /etc/logstash/patterns/. Here are the patterns:

# psad grok patterns
PSAD_PORT_RANGE (?:(%{DATA:[psad][start_port]}\-%{DATA:[psad][end_port]})|%{DATA:[psad][scan_port]})
PSAD_SCAN_DETECTED %{SYSLOGBASE} scan detected(:? \(%{DATA:[psad][scan_type]}\))?: %{IPORHOST:src_ip} -> %{IPORHOST:dst_ip} %{WORD:[psad][proto]}: \[%{PSAD_PORT_RANGE}\](:? flags: %{DATA:[psad][flags]})? %{WORD} pkts: %{NUMBER:[psad][proto_num_pkts]} DL: %{NUMBER:[psad][danger_level]}%{GREEDYDATA:message}
PSAD_SIGNATURE_MATCH %{SYSLOGBASE} src: %{IPORHOST:src_ip} signature match: \"%{DATA:[psad][signature]}\" \(sid: %{NUMBER:[psad][signature_id]}\) %{WORD:[psad][proto]} port: %{NUMBER:[psad][scan_port]}
PSAD_AUTO_BLOCK %{SYSLOGBASE} added iptables auto\-block against ${IPORHOST:src_ip} for %{NUMBER:[psad][block_time]} seconds
PSAD_AUTO_UNBLOCK %{SYSLOGBASE} removed iptables auto\-block against ${IPORHOST:src_ip}
PSAD_NOTICE %{SYSLOGBASE} (?=%{GREEDYDATA:message})(?:flushing|imported|received|domain)
PSAD_WARNING %{SYSLOGBASE} %{LOGLEVEL:[psad][log_level]}: (?=%{GREEDYDATA:message})

Let me briefly explain what each pattern does:

PSAD_PORT_RANGE

This pattern is only used by other patterns to prevent having to duplicate the port range part of the pattern Fields: start_port, end_port, scan_port

PSAD_SCAN_DETECTED

Used for port scan detections Fields: scan_type, src_ip, dst_ip, proto, flags, proto_num_pkts, danger_level, message

PSAD_SIGNATURE_MATCH

Used for psad/fwsnort signature matches Fields: src_ip, signature, signature_id, proto, scan_port

PSAD_AUTO_BLOCK

Used to detect auto-block actions Fields: src_ip, block_time

PSAD_AUTO_UNBLOCK

Used to detect auto-unblock actions Fields: src_ip

PSAD_NOTICE

Used to detect PSAD notice messages; usually generated by PSAD upon starting the service or updating the signatures Fields: message

PSAD_WARNING

Used to detect PSAD warning messages Fields: log_level, message

Available log parameters

Here's a list of all paramters which are available for use in Logstash (or elasticsearch, if that's what you're using to store your logs; these are also the fields you can use to create your dashboards in Kibana).

[psad][start_port] / [psad][end_port]

Start and end port of a port scan range

[psad][scan_port]

Port scanned or found in signature match

[psad][scan_type]

Indicates the type of scan, if any

[psad][proto]

Protocol used in scan (TCP/UDP)

[psad][flags]

Any attack flags found (SYN, etc)

[psad][proto_num_pkts]

The number of packets received in the attack/scan

[psad][danger_level]

The PSAD danger lever assigned to the attack/scan

[psad][signature]

The psad/fwsnort signature detected

[psad][signature_id]

The psad/fwsnort signature ID detected

[psad][block_time]

The time an IP address was auto-blocked

[psad][log_level]

The log level of a PSAD warning message

message

A generic message for the logline; varies per type of log line

src_ip

The source IP of the attack/scan

dst_ip

The destination IP of the attack/scan

Note: Why not place the src_ip and dst_ip in the [psad] array as well? Since I use logstash to process almost all my logfiles, I prefer to keep things consistent. This way I can create a separate GEOIP filter to process all fields called src_ip and dst_ip and I don't have to create a separate filter for each logfile.

Next up: logstash input

The logstash input filter will assure that our psad log is parsed by logstash. Here's what it looks like:

input {
    file {
        path => '/var/log/psad.log' # the location of our psad logfile
        type => 'psad' # assign a type so we can easily process this in the filters
    }
}

Are you using logstash-forwarder?

No problem, use this instead in your config.json file:

{
  "files": [
    {
      "paths": [ "/var/log/psad.log" ],
      "fields": { "type": "psad" }
    }
  ]
}

On to: logstash filter

Now that we've made sure logstash has access to our PSAD logfile, we need to tell it what to do with each line. I've placed comments in the code to guide you through each part.

filter {
	# only process lines which are of type 'psad'; this is set in the input section
    if [type] == "psad" {

        # parse for detected scans
        grok {
            patterns_dir    => "/etc/logstash/patterns/" # specify the folder where logstash can find your patterns file
            match           => [ "message", "%{PSAD_SCAN_DETECTED}" ]
            add_tag         => [ "psad", "scan-detected" ] # add tags so we can determine what to do with each type of line
            tag_on_failure  => [] # do not add tags on failure just yet, first check the other line types
        }

        # parse for signature match
        if "psad" not in [tags] {
            grok {
                patterns_dir    => "/etc/logstash/patterns/"
                match           => [ "message", "%{PSAD_SIGNATURE_MATCH}" ]
                add_tag         => [ "psad", "signature-match" ]
                tag_on_failure  => []
            }
        }

        # parse for auto block
        if "psad" not in [tags] {
            grok {
                patterns_dir    => "/etc/logstash/patterns/"
                match           => [ "message", "%{PSAD_AUTO_BLOCK}" ]
                add_tag         => [ "psad", "iptables-auto-block" ]
                tag_on_failure  => []
            }
        }

        # parse for auto unblock
        if "psad" not in [tags] {
            grok {
                patterns_dir    => "/etc/logstash/patterns/"
                match           => [ "message", "%{PSAD_AUTO_UNBLOCK}" ]
                add_tag         => [ "psad", "iptables-auto-unblock" ]
                tag_on_failure  => []
            }
        }

        # parse for psad warning
        if "psad" not in [tags] {
            grok {
                patterns_dir    => "/etc/logstash/patterns/"
                match           => [ "message", "%{PSAD_WARNING}" ]
                add_tag         => [ "psad", "warning" ]
                tag_on_failure  => []
            }
        }

        # parse for psad notice
        if "psad" not in [tags] {
            grok {
                patterns_dir    => "/etc/logstash/patterns/"
                match           => [ "message", "%{PSAD_NOTICE}" ]
                add_tag         => [ "psad", "service-notice" ]
                tag_on_failure  => []
            }
        }

        # check for unmatched loglines
        # if you find these often, I may have missed a line type and we will need to add a
        # pattern for it; please send me any such lines so I can create the grok pattern
        if "psad" not in [tags] {
            mutate {
                add_tag         => [ "psad", "unmatched-logline" ]
            }
        }

    }
}

Roundup

Now you've configured rsyslog to write your psad messages to a separate file, told logstash where to find this file and what to do with each type of log line. Use your creativity to send your logstash filtered logs to any of the available outputs; I personally use elasticsearch to store my logs and kibana to visualize them. In a next gist I will likely publish my Kibana PSAD dashboard, but I still need to do some finetuning there first.

If you have any questions, please feel free to contact me: https://www.github.com/netson

Links

Useful links with information about PSAD:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment