This gist answers a Discord question. It explains my approach to collecting CPU temperatures by sending the data as an MQTT payload.
I like to think of this approach as working with the MING (Mosquitto, InfluxDB, Node-RED, Grafana) paradigm rather than fighting against it (eg using approaches like SSH calling out of the Node-RED container).
The approach is not tied to the Raspberry Pi that is running IOTstack. If you have several Raspberry Pis, they can all log their temperatures to the Raspberry Pi running IOTstack using exactly the same mechanism.
The concept can also be extended to other types of computer. I'm using it to collect CPU temperatures for macOS systems too.
Be very careful about copying and pasting text from this gist. Unless you take precautions, Windows will add its CR+LF line-endings and those will cause problems.
-
Assumed path name:
~/.local/bin/publish_rpi_temperature
-
Script content:
#!/usr/bin/env bash mkdir -p ~/Logs mosquitto_pub -h mosquitto.mydomain.com -t "/mytopic/computer/pi" -m "{\ \"host\": \"$HOSTNAME\", \ \"temp\": $(vcgencmd measure_temp | cut -c 6-9)\ }"
The script assumes the Mosquitto clients are installed:
$ sudo apt install mosquitto-clients
You will need to customise:
- The target host "mosquitto.mydomain.com" using the fully-qualified domain name, or host name, or multicast DNS name, or IP address of the Raspberry Pi running IOTstack.
- The topic "/mytopic/computer/pi". Although I did not need to use it, I originally added the "/pi" suffix to the topic string in case I needed to know the type of computer that had generated the payload. My Raspberry Pi's use "/pi"; my macOS machines "/mac".
This script is specific to the Raspberry Pi. To port the script to a different type of host you will need to:
- find an alternative for
vcgencmd
; and - make sure that the HOSTNAME environment variable contains a sensible value (eg on macOS, it contains the fully-qualified domain name rather than the host name).
The MQTT messages sent by this script look like this:
/mytopic/computer/mac {"host": "bauxite", "temp": 46.5}
/mytopic/computer/mac {"host": "magnetite", "temp": 28.8}
/mytopic/computer/mac {"host": "gravel", "temp": 35.2}
/mytopic/computer/mac {"host": "marble", "temp": 34.2}
/mytopic/computer/pi {"host": "octopi", "temp": 41.3}
/mytopic/computer/pi {"host": "new-dev", "temp": 43.3}
/mytopic/computer/pi {"host": "iot-hub", "temp": 46.2}
/mytopic/computer/pi {"host": "sec-dev", "temp": 45.2}
$ mkdir -p ~/Logs
The collection script does not normally produce output unless there is an error condition. If the script does not appear to be working, the place to start looking is:
$ tail ~/Logs/publish_rpi_temperature.log
-
Preamble (should be common to all crontab files):
SHELL=/bin/bash HOME=/home/pi PATH=/home/pi/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
-
Event trigger:
# report system temperature every 5 minutes */5 * * * * publish_rpi_temperature >>./Logs/publish_rpi_temperature.log 2>&1
The InfluxDB database needs to be initialised before you can write to it. In this example, the database name is "computer".
$ docker exec -it influxdb influx
> create database computer
> exit
$
Within the "computer" database, each host computer logging temperatures generates its own series:
temperature,system=«hostname»
where "«hostname»" corresponds with the "host" key in the MQTT payload.
A Node-RED three-node flow handles the messages:
-
an MQTT-in node subscribing to the appropriate topic. In this example the topic is "/mytopic/computer/+" where "+" is a single-level wildcard.
-
a Change node to prepare the payload for insertion into InfluxDB:
[ { "temp": payload.temp },{ "system": payload.host } ]
In words:
- temperature is added as a field (data);
- the name of the host reporting the temperature is added as a tag (metadata)
-
an InfluxDB-out node inserts the prepared payload into the database named "computer".
Given the incoming MQTT message:
/mytopic/computer/pi {"host": "iot-hub", "temp": 46.2}
The overall effect of the flow is:
> USE computer
> INSERT temperature,system='iot-hub' temp=46.2
> exit
You should be able to copy the JSON content below and use the "Import" command in the main menu (three horizontal bars "≡" at the top, right of the Node-Red window) to create a new flow containing the three nodes.
Depending on how your system is set up, the flow may need some tinkering before it works.
[
{
"id": "cc5ffba5.fdc7c8",
"type": "tab",
"label": "CPU Temperatures",
"disabled": false,
"info": ""
},
{
"id": "796d8452.225294",
"type": "change",
"z": "cc5ffba5.fdc7c8",
"name": "prepare fields",
"rules": [
{
"t": "set",
"p": "payload",
"pt": "msg",
"to": "[\t {\t \"temp\": payload.temp\t },{\t \"system\": payload.host\t }\t]\t",
"tot": "jsonata"
}
],
"action": "",
"property": "",
"from": "",
"to": "",
"reg": false,
"x": 340,
"y": 100,
"wires": [
[
"d27128f3.4050c8"
]
]
},
{
"id": "d27128f3.4050c8",
"type": "influxdb out",
"z": "cc5ffba5.fdc7c8",
"influxdb": "626e8bc5.4c702c",
"name": "write to InfluxDB",
"measurement": "temperature",
"precision": "",
"retentionPolicy": "",
"database": "",
"retentionPolicyV18Flux": "",
"org": "",
"bucket": "",
"x": 570,
"y": 100,
"wires": []
},
{
"id": "39543010.e0258",
"type": "mqtt in",
"z": "cc5ffba5.fdc7c8",
"name": "/mytopic/computer/+",
"topic": "/mytopic/computer/+",
"qos": "2",
"datatype": "json",
"broker": "c5d29fb5.89907",
"x": 130,
"y": 100,
"wires": [
[
"796d8452.225294"
]
]
},
{
"id": "626e8bc5.4c702c",
"type": "influxdb",
"hostname": "influxdb",
"port": "8086",
"protocol": "http",
"database": "computer",
"name": "",
"usetls": false,
"tls": "",
"influxdbVersion": "1.x"
},
{
"id": "c5d29fb5.89907",
"type": "mqtt-broker",
"name": "Docker MQTT",
"broker": "mosquitto",
"port": "1883",
"clientid": "",
"usetls": false,
"compatmode": false,
"protocolVersion": 4,
"keepalive": "60",
"cleansession": true,
"birthTopic": "",
"birthQos": "0",
"birthRetain": "false",
"birthPayload": "",
"closeTopic": "",
"closeQos": "0",
"closeRetain": "false",
"closePayload": "",
"willTopic": "",
"willQos": "0",
"willRetain": "false",
"willPayload": ""
}
]
$ docker exec -it influxdb influx -precision=rfc3339
Connected to http://localhost:8086 version 1.8.9
InfluxDB shell version: 1.8.9
> use computer
Using database computer
> show series
key
---
temperature,system=bauxite
temperature,system=gravel
temperature,system=iot-hub
temperature,system=magnetite
temperature,system=marble
temperature,system=new-dev
temperature,system=octopi
temperature,system=sec-dev
temperature,system=sjgair
> select * from temperature where time > now() - 10m tz('Australia/Sydney')
name: temperature
time system temp
---- ------ ----
2021-08-22T23:40:00.271972208+10:00 marble 35.2
2021-08-22T23:40:00.674021388+10:00 magnetite 29.6
2021-08-22T23:40:00.937395236+10:00 gravel 36.6
2021-08-22T23:40:01.065496792+10:00 bauxite 46
2021-08-22T23:40:01.220100822+10:00 sec-dev 46.7
2021-08-22T23:40:01.87568606+10:00 iot-hub 47.7
2021-08-22T23:40:01.892511832+10:00 octopi 40.8
2021-08-22T23:40:02.021640681+10:00 new-dev 42.8
2021-08-22T23:45:00.210631999+10:00 magnetite 28.9
2021-08-22T23:45:00.453033759+10:00 gravel 35.6
2021-08-22T23:45:00.545580449+10:00 bauxite 46.4
2021-08-22T23:45:00.840558319+10:00 marble 35.6
2021-08-22T23:45:01.133290607+10:00 new-dev 43.3
2021-08-22T23:45:01.319572925+10:00 sec-dev 46.2
2021-08-22T23:45:01.977801647+10:00 iot-hub 46.7
2021-08-22T23:45:01.99638355+10:00 octopi 41.3
> exit
$
Shows a mixture of Raspberry Pi and macOS CPU temperatures.
🥳 - graphics!
Aside from the standard benefits of containerisation (someone else does all the maintenance work, easy to pin to earlier versions if an update breaks something, or test new versions in a controlled manner before committing to them, "no surprises" because two processes try to futz with the same part of your OS - they can't because containerisation stops them), you should think about the efficiency of traffic flows. A device like the sonoff speaks. Its MQTT traffic arrives at the RPi where Docker is listening on port 1883. Docker uses Network Address Translation (NAT) to forward the packet to the container's port 1883. The entire TCP exchange in each direction goes through NAT and the packets are routed (Layer Three) in and out of container space. If NodeRed and Influx are both running as containers like Mosquitto then all the remaining traffic interactions occur within containerspace so it's like a Layer Two switch - unicast traffic is point-to-point across the internal subnet. If NodeRed runs outside container space then its subscriptions to Mosquitto and its "insert" instructions to Influx all have to undergo bi-directional NAT and be routed in and out of containerspace. It works but ... yech.
Also, NodeRed graphics are a good way to get started but I think you'll quickly grow tired of their limitations. I bet the question you'll be asking in a few days is "why don't the measurements shown in the graphs persist if I restart NodeRed?" Because they don't. You can make NodeRed do it but it's quite ugly.
Grafana, on the other hand, has all this down pat. One of the things I really like is having a running dashboard of last week's voltage being updated every 10 seconds. If I notice spike I can just rubber-band the area and zoom in on the period in question without a second thought. The histogram associated with the graph of voltage by time just follows along giving me the distribution. Quite neat, really. It's a great app.
I suppose the best advice I can give you is to not waste too much time trying to make NodeRed do graphics. You have devices. They send telemetry to Mosquitto - it's the concentrator. Node-Red is the traffic cop saying where the data should be stored in Influx. Grafana is the display engine, ferreting data out of influx tables and combining it in interesting ways. Node-Red is good at things like alerts but Grafana has those too so it's a matter of taste and features. Influx is also good at chucking data away or aggregating it in useful ways. Example, do I really need 10-second voltage data from five years ago or would the maximum voltage in each minute, five minutes, 30 minutes, hour, etc be more appropriate? Influx can handle that. Each component in the MING stack is well worth the learning curve.