Skip to content

Instantly share code, notes, and snippets.

@Tokugero
Last active December 8, 2021 10:35
Show Gist options
  • Save Tokugero/f013c0a97dc1982074b94d05b7bb8d1e to your computer and use it in GitHub Desktop.
Save Tokugero/f013c0a97dc1982074b94d05b7bb8d1e to your computer and use it in GitHub Desktop.
Telegraf with pfSense 2.4.4 and pfBlockerNG

How to Install Telegraf logparser for pfBlockerNG

On pfSense

Step 1: Install pfBlockerNg-devel

System > Package Manager > Available Packages > pfBlockerNG-devel (As of this writing this was 2.2.5_19) (Optional, run the /usr/local/bin/geoipupdate.sh script)

[2.4.4-RELEASE][root@pfSense.lab]/root: /usr/local/bin/geoipupdate.sh
Fetching GeoIP.dat and GeoIPv6.dat...
/usr/local/share/GeoIP/GeoIPupdate.VOmIvG/GeoI100% of  694 kB 6683 kBps 00m00s
/usr/local/share/GeoIP/GeoIPupdate.TeBBKT/GeoI100% of 1171 kB 6629 kBps 00m00s

Configure your favorite DNSBL blocks

Step 2: Install Telegraf

System > Package Manager > Telegraf (As of this writing this was 0.6) Services > Telegraf > [X] Enable Telegraf Telegraf Output = (your favorite database, I use InfluxDB) InfluxDB Server = (IP/hostname of your database host) InfluxDB Database = (This can be the same as other telegraf configurations on your network, or an entirely new one, or leave blank to default to "telegraf"

Step 3: Install the latest Telegraf (At least version 1.8 which has the SysLog time parser fix)

NOTE This is for a 64 bit system, you may need to find the right version for your hardware ssh pfsense host

cd ~
curl -O "https://dl.influxdata.com/telegraf/releases/telegraf-1.8.3_freebsd_amd64.tar.gz"
tar xzfv telegraf-1.8.3_freebsd_amd64.tar.gz
service telegraf onestop
mv /usr/local/bin/telegraf ~/telegraf.bak
mv ~/telegraf/usr/bin/telegraf /usr/local/bin/telegraf
service telegraf onestart

Step 4: Update the Telegraf config generator

vi /usr/local/pkg/telegraf.inc

#Somewhere in the [[inputs. ]] section

[[inputs.logparser]]
  files = ["/var/log/pfblockerng/dnsbl.log"]
  from_beginning=true
  [inputs.logparser.grok]
    measurement = "dnsbl_log"
    patterns = ["^%{WORD:BlockType}-%{WORD:BlockSubType},%{SYSLOGTIMESTAMP:timestamp:ts-syslog},%{IPORHOST:destination:tag},%{IPORHOST:source:tag},%{GREEDYDATA:call},%{WORD:BlockMethod},%{WORD:BlockList},%{IPORHOST:tld:tag},%{WORD:DefinedList:tag},%{GREEDYDATA:hitormiss}"]
    timezone = "Local"
    [inputs.logparser.tags]
      value = "1"

Step 5: Generate new telegraf config

Services > Telegraf > Save

In Grafana <Note that configuring influx & grafana are out of scope for this guide>

The below is an example Dashboard JSON that reads the tags put in the previous Step 4: https://i.redd.it/l03ed4yuybx11.png

{
  "aliasColors": {},
  "bars": false,
  "dashLength": 10,
  "dashes": false,
  "datasource": "Lab Telegraf",
  "fill": 1,
  "gridPos": {
    "h": 9,
    "w": 12,
    "x": 0,
    "y": 0
  },
  "id": 2,
  "legend": {
    "alignAsTable": true,
    "avg": false,
    "current": false,
    "max": false,
    "min": false,
    "rightSide": true,
    "show": true,
    "sort": "total",
    "sortDesc": true,
    "total": true,
    "values": true
  },
  "lines": true,
  "linewidth": 1,
  "links": [],
  "nullPointMode": "null as zero",
  "percentage": false,
  "pointradius": 5,
  "points": false,
  "renderer": "flot",
  "seriesOverrides": [],
  "spaceLength": 10,
  "stack": false,
  "steppedLine": false,
  "targets": [
    {
      "alias": "[[tag_destination]]",
      "groupBy": [
        {
          "params": [
            "$__interval"
          ],
          "type": "time"
        },
        {
          "params": [
            "destination"
          ],
          "type": "tag"
        }
      ],
      "measurement": "dnsbl_log",
      "orderByTime": "ASC",
      "policy": "default",
      "refId": "A",
      "resultFormat": "time_series",
      "select": [
        [
          {
            "params": [
              "hitormiss"
            ],
            "type": "field"
          },
          {
            "params": [],
            "type": "count"
          }
        ]
      ],
      "tags": []
    }
  ],
  "thresholds": [],
  "timeFrom": null,
  "timeShift": null,
  "title": "DNSBL Blocks by Destination",
  "tooltip": {
    "shared": true,
    "sort": 2,
    "value_type": "individual"
  },
  "transparent": true,
  "type": "graph",
  "xaxis": {
    "buckets": null,
    "mode": "time",
    "name": null,
    "show": true,
    "values": []
  },
  "yaxes": [
    {
      "format": "short",
      "label": null,
      "logBase": 1,
      "max": null,
      "min": null,
      "show": true
    },
    {
      "format": "short",
      "label": null,
      "logBase": 1,
      "max": null,
      "min": null,
      "show": true
    }
  ]
}
@kubedzero
Copy link

getting set up after a big move

Hope the move went well!

I did chmod the dnsbl.log file and that worked for 24h actually! Then it rolled over and got recreated, went back to the original owner-only permissions -rw------- 1 root wheel 2.6M May 27 15:52 /var/log/pfblockerng/dnsbl.log and I lost access again.

I've since rebooted + upgraded to 2.4.5 pfSense and the issue went away. Not sure if it was the reboot or the upgrade that fixed it, I have to keep pfSense online for as long as possible so rebooting isn't always an option. We'll see if it was just a fluke or if it continues to be a problem.

I can confirm that mine was running as root as well. 90071 root 14 20 0 166M 81772K uwait 1 5:27 0.06% telegraf

@Tokugero
Copy link
Author

Very strange, I'm on 2.3.5-RELEASE-p2 and do not seem to have this issue... Glad you got it worked out, deals with demons or otherwise. Time for me to update as well and see how my luck fares!

@Tokynet
Copy link

Tokynet commented Aug 31, 2020

FWIW, I'm running into this permissions issue too. I've been running 2.4.5 from the beginning. I'm going to try a reboot and see if that fixes it.

edit:
Reboot did fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment