using https://yetiops.net/posts/openbsd-snmp-exporter/
The goal is to present basic metrics such as bytes passed through the external interface of the router. The main idea is to run SNMP on the OpenBSD router that can get the metrics from PF. Then SNMP exporter from Prometheus will convert the data to Prometheus format that can be scraped by Grafana.
The constraint is that this should be done in reasonably secure way: the IoT devices are in separate VLAN, however the router should have miminal exposure. Using snmpd
with privilege separation and in flight data protection is sufficient. Further, the data collection and graphing will be done on single machine.
This assumes that the PF configuration on the OpenBSD router has something like this:
set loginterface $ExtIF
and that PF is enabled and active. Also, there are PF rules in place that allow SNMP traffic to the interface of the IoT network/VLAN.
- create
snmpd
setup:
echo snmpd_flags= >> /etc/rc.conf.local
cat << EOF >/etc/snmpd.conf
## in OpenBSD 7.0 it should be possible to say 'listen on any' and rely on PF rules.
listen on 127.0.0.1 read
# IoT
listen on 172.40.0.1 read
# Adjust the local system information
#system contact "Charlie Root (root@myhost.example.com)"
#system description "Powered by OpenBSD"
#system location "Rack A1-24, Room 13"
system services 74
seclevel enc
user "grafana" authkey "bae8aX3i" enc aes enckey "AhChaBa8"
EOF
- either reboot or run:
snmpd
This assumes Raspberry Pi setup ala https://github.com/vladak/pirig/blob/master/weatherpi.md
First confirm that SNMP works:
snmpwalk -l authPriv -a SHA -A bae8aX3i -x AES -X AhChaBa8 -u grafana -v 3 gw
This should produce the output like:
iso.3.6.1.2.1.1.1.0 = STRING: "OpenBSD ... 6.9 GENERIC.MP#551 octeon"
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.30155.23.1
iso.3.6.1.2.1.1.3.0 = Timeticks: (13084198) 1 day, 12:20:41.98
iso.3.6.1.2.1.1.4.0 = STRING: "root@..."
iso.3.6.1.2.1.1.5.0 = STRING: "..."
iso.3.6.1.2.1.1.6.0 = ""
iso.3.6.1.2.1.1.7.0 = INTEGER: 74
iso.3.6.1.2.1.1.8.0 = Timeticks: (0) 0:00:00.00
iso.3.6.1.2.1.1.9.1.1.1 = INTEGER: 1
...
First create ~/snmp/generator.yml
:
modules:
openbsd_pf:
walk:
# counters from HOST-RESOURCES-MIB (http://www.net-snmp.org/docs/mibs/host.html)
- hrSystemUptime
- hrSystemDate
- hrProcessorTable
- hrStorageTable
# PF counters
- pfRunning
- pfRuntime
- pfHostid
- pfStateCount
- pfStateSearches
- pfStateInserts
- pfStateRemovals
- pfLogIfName
- pfLogIfIpBytesIn
- pfLogIfIpBytesOut
- pfLogIfIpPktsInPass
- pfLogIfIpPktsInDrop
- pfLogIfIpPktsOutPass
- pfLogIfIpPktsOutDrop
- pfIfIn4PassPkts
- pfIfIn4PassBytes
- pfIfIn4BlockPkts
- pfIfIn4BlockBytes
- pfIfOut4PassPkts
- pfIfOut4PassBytes
- pfIfOut4BlockPkts
- pfIfOut4BlockBytes
- pfIfDescr
- pfIfIndex
version: 3
lookups:
- source_indexes: [pfIfIndex]
lookup: pfIfDescr
drop_source_indexes: false
auth:
username: "grafana"
security_level: authPriv
password: "bae8aX3i"
auth_protocol: SHA
priv_protocol: AES
priv_password: "AhChaBa8"
then generate ~/snmp/snmp.yml
config file for snmp_exporter
:
sudo apt-get install -y golang
sudo apt-get install -y libsnmp-dev # delivers net-snmp/net-snmp-config.h needed by the Go generator
mkdir ~/golang
export GOPATH=~/golang
go get github.com/prometheus/snmp_exporter/generator
cd ~/snmp
export MIBDIRS=/usr/share/snmp/mibs:/usr/share/snmp/mibs/iana:/usr/share/snmp/mibs/ietf:/usr/share/mibs/site:/usr/share/snmp/mibs:/usr/share/mibs/iana:/usr/share/mibs/ietf:/usr/share/mibs/netsnmp:$PWD/mibs
sudo apt-get install -y snmp-mibs-downloader # needed for base MIBs (in /usr/share/snmp/mibs/) referenced by OpenBSD MIBs
scp -r gw:/usr/share/snmp/mibs mibs # gw is OpenBSD
$GOPATH/bin/generator generate
Use https://gist.github.com/vladak/eae9a33c4f1b4dd7d0c92aeea4a519a2 to setup SNMP exporter service.
Append the generated file to the snmp.yml
file used by the SNMP exporter service and restart the service.
- verify that the data can be retrieved:
curl 'http://localhost:9116/snmp?target=gw&module=openbsd_pf'
The curl
output should contain data such as:
# HELP pfIfOut4BlockBytes The number of outgoing IPv4 bytes blocked. - 1.3.6.1.4.1.30155.1.8.128.1.13
# TYPE pfIfOut4BlockBytes counter
pfIfOut4BlockBytes{pfIfDescr="all",pfIfIndex="1"} 0
pfIfOut4BlockBytes{pfIfDescr="carp",pfIfIndex="2"} 0
pfIfOut4BlockBytes{pfIfDescr="cnmac0",pfIfIndex="3"} 659404
pfIfOut4BlockBytes{pfIfDescr="cnmac1",pfIfIndex="4"} 0
pfIfOut4BlockBytes{pfIfDescr="cnmac2",pfIfIndex="5"} 0
...
update /etc/prometheus/prometheus.yml
with (because the service running on port 9116 requires the target
and module
parameters, these are added through relabeling):
scrape_configs:
- job_name: openbsd_pf_snmp
static_configs:
- targets:
- gw
scrape_interval: 15s
metrics_path: /snmp
params:
module: [openbsd_pf]
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9116
- restart Prometheus:
sudo systemctl restart prometheus
- use
journalctl _SYSTEMD_UNIT=prometheus.service
to check that the configuration was reloaded correctly (unlike Solaris' SMF there does not seem to be a concept of degraded service if the configuration was not parsed successfully).
setup new dashboard and add a few panels referencing metrics like pfLogIfIpBytesIn
or pfLogIfIpBytesOut
.