Skip to content

Instantly share code, notes, and snippets.

@arnesund
Last active September 3, 2023 13:40
  • Star 6 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save arnesund/29ffa1cdabacabe323d3bc45bc7db3fb to your computer and use it in GitHub Desktop.
Fetch Netatmo Weather Station measurements and store in InfluxDB
#!/usr/bin/env python
import os
import sys
import json
import time
import requests
# Get your client ID and secret by creating an App at https://dev.netatmo.com/
NETATMO_CLIENT_ID = ""
NETATMO_CLIENT_SECRET = ""
NETATMO_USERNAME = ""
NETATMO_PASSWORD = ""
# Ensure that the hostname and database name are correct for your InfluxDB
INFLUX_HOST = 'localhost'
INFLUX_PORT = '8086'
INFLUX_DATABASE = 'netatmo'
INFLUXDB_WRITE_URL = 'http://{}:{}/write?precision=s&db={}'.format(INFLUX_HOST, INFLUX_PORT, INFLUX_DATABASE)
print('Will write measurements to InfluxDB at endpoint {}'.format(INFLUXDB_WRITE_URL))
data = dict(grant_type='password', client_id=NETATMO_CLIENT_ID,
client_secret=NETATMO_CLIENT_SECRET, username=NETATMO_USERNAME,
password=NETATMO_PASSWORD, scope='read_station')
resp = requests.post('https://api.netatmo.com/oauth2/token', data=data)
if resp.status_code == 200:
token = resp.json()
token['expiry'] = int(time.time()) + token['expires_in']
while True:
# Check if token needs refresh
if token['expiry'] - int(time.time()) < 600:
data = dict(grant_type='refresh_token', refresh_token=token['refresh_token'], client_id=NETATMO_CLIENT_ID, client_secret=NETATMO_CLIENT_SECRET)
resp = requests.post('https://api.netatmo.com/oauth2/token', data=data)
if resp.status_code == 200:
token = resp.json()
token['expiry'] = int(time.time()) + token['expires_in']
# Fetch measurements
resp = requests.get('https://api.netatmo.com/api/getstationsdata?access_token=' + token['access_token'])
if resp.status_code == 200:
data = resp.json()
payload = ""
for device in data['body']['devices']:
timestamp = device['dashboard_data']['time_utc']
# Normalize station name and module names to ensure they become valid InfluxDB label values
stationname = device['station_name'].replace(' ', '_').encode('utf-8')
modulename = device['module_name'].replace(' ', '_').encode('utf-8')
for datatype in device['data_type']:
payload += '{0},station_name={1},module_name={2} value={3} {4}\n'.format(datatype, stationname, modulename, device['dashboard_data'][datatype], timestamp)
for module in device['modules']:
modulename = module['module_name'].replace(' ', '_').encode('utf-8')
for datatype in module['data_type']:
if datatype == 'Wind':
for subtype in ['WindStrength', 'WindAngle', 'GustStrength', 'GustAngle']:
payload += '{0},station_name={1},module_name={2} value={3} {4}\n'.format(subtype, stationname, modulename, module['dashboard_data'][subtype], timestamp)
else:
payload += '{0},station_name={1},module_name={2} value={3} {4}\n'.format(datatype, stationname, modulename, module['dashboard_data'][datatype], timestamp)
# Debug output to ensure payload contains data and has valid InfluxDB format
print('Writing the following data points to InfluxDB:')
print(payload)
# Write to InfluxDB
resp = requests.post(INFLUXDB_WRITE_URL, data=payload)
# New data arrives every 10 min so the sleep period is a bit shorter to ensure you get it all
time.sleep(480)
@vlsoft
Copy link

vlsoft commented Jun 19, 2019

hi ,

since this Wednesday 17h, the module does not work anymore, I have the following error message:

File "/opt/InputDataInflux/netatmo2.py", line 46, in
     timestamp = device ['dashboard_data'] ['time_utc']
KeyError: 'dashboard_data'

do you have an idea.

thank you

@arnesund
Copy link
Author

Hi!

That is hard to tell without insight into your dataset. The error says “dashboard_data” is not found, and to debug that I suggest you start by printing the “device” data structure right before the line that fails so you can see what it looks like. It might contain an error from the Netatmo API that leads you in the right direction.

Arne

@vlsoft
Copy link

vlsoft commented Jun 20, 2019

OK thank you,

I found,
A module was disconnected.
and no info was on the netatmo website.
I reconnect the module and now it works .

lionel

@jpatriarca
Copy link

Thank you for your great script.

I have the same issue with a faulty module on my installation and I see that the script don't execute if an error occurs. There is any way to ignore faulty modules extractions and collect the remaining sensors data to Grafana?

Sorry if I'm asking too much but it should be a great improvement on the script.

Thank you

@arnesund
Copy link
Author

Hi! Yes, that should be fairly easy to do. Where in the script do you get an exception? Please post the exception here so I can add some error handling.

@jpatriarca
Copy link

Hi,

thank you for your reply.

The error that I receive when your script is unable to retrieve values from one of the stations and restarts are:

Aug 27 16:44:16 jpserver python[18494]: Will write measurements to InfluxDB at endpoint http://influxdb.internal.patriarca.pt:8086/write?precision=s&db=netatmo
Aug 27 16:44:16 jpserver python[18494]: Traceback (most recent call last):
Aug 27 16:44:16 jpserver python[18494]: File "/home/jpatriarca/scripts/fetch_netatmo.py", line 60, in
Aug 27 16:44:16 jpserver python[18494]: payload += '{0},station_name={1},module_name={2} value={3} {4}\n'.format(datatype, stationname, modulename, module['dashboard_data'][datatype], timestamp)
Aug 27 16:44:16 jpserver python[18494]: KeyError: 'dashboard_data'
Aug 27 16:44:16 jpserver systemd[1]: netatmo.service: Main process exited, code=exited, status=1/FAILURE
Aug 27 16:44:16 jpserver systemd[1]: netatmo.service: Failed with result 'exit-code'.
Aug 27 16:46:16 jpserver systemd[1]: netatmo.service: Service hold-off time over, scheduling restart.
Aug 27 16:46:16 jpserver systemd[1]: netatmo.service: Scheduled restart job, restart counter is at 114.
Aug 27 16:46:16 jpserver systemd[1]: Stopped Netatmo.
Aug 27 16:46:16 jpserver systemd[1]: Starting Netatmo...
Aug 27 16:46:16 jpserver systemd[1]: Started Netatmo.

Despite the script states "KeyError: 'dashboard_data' I can confirm that this is not related to InfluxDB or Grafana availability but because one of the stations were offline at the moment the script run (station connectivity issue). As soon as this issue were resolved and the station was online again, the script runs smoothly. Your script as set as a service in my setup.

Having the change to support one or multiple stations unavailable will permit that the remaining online stations to upload data to InfluxDB.

Thank you

@sbrooke
Copy link

sbrooke commented Mar 15, 2020

I'm working on setting up a docker container to run this and your script has been invaluable. I'm also running into the same issue with offline modules. I'm trying to learn python, but pretty early in the process. Any suggestions for doing the error handling you mentioned above?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment