Send traffics stats by ip from pfsense to influxdb

rate_to_influx.sh

#!/bin/sh
#
# Author: Luis Felipe Domínguez Vega <ldominguezvega@gmail.com>
# https://gist.github.com/lfdominguez/08de623d9f9c0fa84e6b4d5d0d25025c
# Program to use with the rate software to send to InfluxDB the
# rate of each IP on interface
#
# rate_to_influx <influxdb_url> <influxdb_database> <local_network> <interface>
# rate_to_influx http://127.0.0.1:8086 network 192.168.0.0/24 re1

#- - - - - - - - - - - - - - - - - - -
# rate -i re1 -r 1 -e -nl -Ab -a 255 -c 192.168.0.0/24 -d
# rate -i $4 -r 1 -e -nl -Ab -a 255 -c $3 -d
#192.168.0.40: 0:1424 :0:1424: 0:1: 0:1
#

logger "$0 [I] Started rate estimator"

URL="$1/write?db=$2&precision=s"

DATA_INFLUX=""

PROGRAM='
{
  base_1="network_metric,host="
  base_2=",direction="
  base_3="byte_rate="

  split($0, data, ":")

  print base_1 data[1] base_2 "download " base_3 data[4] " " ts
  print base_1 data[1] base_2 "upload " base_3 data[5] " " ts
  printf " "
}
'

FLAG_TS=1
rate -i $4 -r 1 -e -nl -Ab -a 255 -c $3 -d | while read -r line; do

FIRST_LINE=`echo $line | head -c 1`
if [ x"$FIRST_LINE" == x"-" ]; then
    curl -i -s -k -o /dev/null -XPOST "$URL" --data-binary "$DATA_INFLUX" || {
    logger "$0 [W] Can't write to InfluxDB Database"
    }
    DATA_INFLUX=""
    FLAG_TS=1
else
    if [ $FLAG_TS -eq 1 ]; then
        TS=`date +%s`
        FLAG_TS=0
    fi

    DATA_FROM_RATE=`echo "$line" | awk -v ts="$TS" "$PROGRAM"`
    DATA_INFLUX="$DATA_INFLUX$DATA_FROM_RATE"
fi
done

Hey there,
How does one implement this? and would it be possible to incorporate hostnames via nslookup and send that to influxdb instead of the ip address?

From the shell of your pfSense save this script, chmod it, and run it.

curl https://gist.github.com/crawc/1669f655a0472e76723d9fd631c9cd75 --output rate_to_influx.sh
chmod +x rate_to_influx.sh
./rate_to_influx http://192.168.100.10:8086 pfsense-lan 192.168.0.0/24 igb1

rate_to_influx <influxdb_url> <influxdb_database> <local_network>

This example shows the Influx database pfsense-lan is running at 192.168.100.10

Here is a modified version that should work do DNS lookups for each Ip before sending it to Influxdb, but untested.
https://gist.github.com/crawc/c375754f80c346a633f2f9e6c956976a

Hey @crawc - Thank you.

This new script, has ticks at the start and end, are they needed or did GIT place them there with the insert code?
With them present, the script does not find 'done'.
I removed the ticks and my output looks like this when running:

`
./rate_to_influx.sh http://10.0.0.250:8086 pfsense 10.0.0.0/24 re1

• ./rate_to_influx.sh: =10.0.0.165: not found
• ./rate_to_influx.sh: =10.0.0.167: not found
• ./rate_to_influx.sh: =10.0.0.109: not found
• ./rate_to_influx.sh: =10.0.0.164: not found

`
my script::
https://github.com/benisai/rate-to-influxdb-pfsense/blob/main/Script%20for%20pfsense

Without the ticks, I updated the gist.

your hostname script works nicely. Thank you.