Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Telegraf with pfSense 2.4.4 and pfBlockerNG

How to Install Telegraf logparser for pfBlockerNG

On pfSense

Step 1: Install pfBlockerNg-devel

System > Package Manager > Available Packages > pfBlockerNG-devel (As of this writing this was 2.2.5_19) (Optional, run the /usr/local/bin/geoipupdate.sh script)

[2.4.4-RELEASE][root@pfSense.lab]/root: /usr/local/bin/geoipupdate.sh
Fetching GeoIP.dat and GeoIPv6.dat...
/usr/local/share/GeoIP/GeoIPupdate.VOmIvG/GeoI100% of  694 kB 6683 kBps 00m00s
/usr/local/share/GeoIP/GeoIPupdate.TeBBKT/GeoI100% of 1171 kB 6629 kBps 00m00s

Configure your favorite DNSBL blocks

Step 2: Install Telegraf

System > Package Manager > Telegraf (As of this writing this was 0.6) Services > Telegraf > [X] Enable Telegraf Telegraf Output = (your favorite database, I use InfluxDB) InfluxDB Server = (IP/hostname of your database host) InfluxDB Database = (This can be the same as other telegraf configurations on your network, or an entirely new one, or leave blank to default to "telegraf"

Step 3: Install the latest Telegraf (At least version 1.8 which has the SysLog time parser fix)

NOTE This is for a 64 bit system, you may need to find the right version for your hardware ssh pfsense host

cd ~
curl -O "https://dl.influxdata.com/telegraf/releases/telegraf-1.8.3_freebsd_amd64.tar.gz"
tar xzfv telegraf-1.8.3_freebsd_amd64.tar.gz
service telegraf onestop
mv /usr/local/bin/telegraf ~/telegraf.bak
mv ~/telegraf/usr/bin/telegraf /usr/local/bin/telegraf
service telegraf onestart

Step 4: Update the Telegraf config generator

vi /usr/local/pkg/telegraf.inc

#Somewhere in the [[inputs. ]] section

[[inputs.logparser]]
  files = ["/var/log/pfblockerng/dnsbl.log"]
  from_beginning=true
  [inputs.logparser.grok]
    measurement = "dnsbl_log"
    patterns = ["^%{WORD:BlockType}-%{WORD:BlockSubType},%{SYSLOGTIMESTAMP:timestamp:ts-syslog},%{IPORHOST:destination:tag},%{IPORHOST:source:tag},%{GREEDYDATA:call},%{WORD:BlockMethod},%{WORD:BlockList},%{IPORHOST:tld:tag},%{WORD:DefinedList:tag},%{GREEDYDATA:hitormiss}"]
    timezone = "Local"
    [inputs.logparser.tags]
      value = "1"

Step 5: Generate new telegraf config

Services > Telegraf > Save

In Grafana <Note that configuring influx & grafana are out of scope for this guide>

The below is an example Dashboard JSON that reads the tags put in the previous Step 4: https://i.redd.it/l03ed4yuybx11.png

{
  "aliasColors": {},
  "bars": false,
  "dashLength": 10,
  "dashes": false,
  "datasource": "Lab Telegraf",
  "fill": 1,
  "gridPos": {
    "h": 9,
    "w": 12,
    "x": 0,
    "y": 0
  },
  "id": 2,
  "legend": {
    "alignAsTable": true,
    "avg": false,
    "current": false,
    "max": false,
    "min": false,
    "rightSide": true,
    "show": true,
    "sort": "total",
    "sortDesc": true,
    "total": true,
    "values": true
  },
  "lines": true,
  "linewidth": 1,
  "links": [],
  "nullPointMode": "null as zero",
  "percentage": false,
  "pointradius": 5,
  "points": false,
  "renderer": "flot",
  "seriesOverrides": [],
  "spaceLength": 10,
  "stack": false,
  "steppedLine": false,
  "targets": [
    {
      "alias": "[[tag_destination]]",
      "groupBy": [
        {
          "params": [
            "$__interval"
          ],
          "type": "time"
        },
        {
          "params": [
            "destination"
          ],
          "type": "tag"
        }
      ],
      "measurement": "dnsbl_log",
      "orderByTime": "ASC",
      "policy": "default",
      "refId": "A",
      "resultFormat": "time_series",
      "select": [
        [
          {
            "params": [
              "hitormiss"
            ],
            "type": "field"
          },
          {
            "params": [],
            "type": "count"
          }
        ]
      ],
      "tags": []
    }
  ],
  "thresholds": [],
  "timeFrom": null,
  "timeShift": null,
  "title": "DNSBL Blocks by Destination",
  "tooltip": {
    "shared": true,
    "sort": 2,
    "value_type": "individual"
  },
  "transparent": true,
  "type": "graph",
  "xaxis": {
    "buckets": null,
    "mode": "time",
    "name": null,
    "show": true,
    "values": []
  },
  "yaxes": [
    {
      "format": "short",
      "label": null,
      "logBase": 1,
      "max": null,
      "min": null,
      "show": true
    },
    {
      "format": "short",
      "label": null,
      "logBase": 1,
      "max": null,
      "min": null,
      "show": true
    }
  ]
}
@kestouf

This comment has been minimized.

Copy link

@kestouf kestouf commented Apr 21, 2020

Hello
I encounter a fairly simple problem, I do not see the telegraf package in the list of available packages.
Do you know why this package is no longer available?

Thx
Best regards

@Tokugero

This comment has been minimized.

Copy link
Owner Author

@Tokugero Tokugero commented Apr 21, 2020

I believe there is/was an older telegraf binary for pfsense at the time but you can download the PKG from telegraf itself https://github.com/influxdata/telegraf/releases

Note step 3 in this doc for the install steps I used at the time, they should still work

@kubedzero

This comment has been minimized.

Copy link

@kubedzero kubedzero commented May 15, 2020

I followed these instructions, I'm running pfSense 2.4.4 on Intel x86 hardware. I installed the latest Telegraf, curl -O "https://dl.influxdata.com/telegraf/releases/telegraf-1.14.2_freebsd_amd64.tar.gz"Looking in the /var/log/telegraf/telegraf.log I see [inputs.logparser] Error in plugin: open /var/log/pfblockerng/dnsbl.log: permission denied repeated a few times.

Any thoughts on what could be causing this?

@Tokugero

This comment has been minimized.

Copy link
Owner Author

@Tokugero Tokugero commented May 27, 2020

Sorry for the delay, just getting set up after a big move.

I followed these instructions, I'm running pfSense 2.4.4 on Intel x86 hardware. I installed the latest Telegraf, curl -O "https://dl.influxdata.com/telegraf/releases/telegraf-1.14.2_freebsd_amd64.tar.gz"Looking in the /var/log/telegraf/telegraf.log I see [inputs.logparser] Error in plugin: open /var/log/pfblockerng/dnsbl.log: permission denied repeated a few times.

Any thoughts on what could be causing this?

This may be caused by your telegraf user running as a non-priviliged user. Here's a top output of how mine is running as root:
52000 root 25 20 0 69720K 47068K uwait 1 0:03 0.00% telegraf

You can try chmoding your dnsbl file to 666 or checking your run scripts for telegraf to ensure it's running as the root user

@kubedzero

This comment has been minimized.

Copy link

@kubedzero kubedzero commented May 27, 2020

getting set up after a big move

Hope the move went well!

I did chmod the dnsbl.log file and that worked for 24h actually! Then it rolled over and got recreated, went back to the original owner-only permissions -rw------- 1 root wheel 2.6M May 27 15:52 /var/log/pfblockerng/dnsbl.log and I lost access again.

I've since rebooted + upgraded to 2.4.5 pfSense and the issue went away. Not sure if it was the reboot or the upgrade that fixed it, I have to keep pfSense online for as long as possible so rebooting isn't always an option. We'll see if it was just a fluke or if it continues to be a problem.

I can confirm that mine was running as root as well. 90071 root 14 20 0 166M 81772K uwait 1 5:27 0.06% telegraf

@Tokugero

This comment has been minimized.

Copy link
Owner Author

@Tokugero Tokugero commented May 27, 2020

Very strange, I'm on 2.3.5-RELEASE-p2 and do not seem to have this issue... Glad you got it worked out, deals with demons or otherwise. Time for me to update as well and see how my luck fares!

@Tokynet

This comment has been minimized.

Copy link

@Tokynet Tokynet commented Aug 31, 2020

FWIW, I'm running into this permissions issue too. I've been running 2.4.5 from the beginning. I'm going to try a reboot and see if that fixes it.

edit:
Reboot did fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.