Get Netatmo Weather station data

README.md

Unfortunately the Netatmo node https://github.com/guidone/node-red-contrib-netatmo-dashboard is not working : Nodered crashes 5/6 times a day if you use it. The owner is not willing to maintain his code.

Here is a method to get the Netatmo Data without having to install a module, just using http requests. Here is how it works :

1. Enter your token and refresh token + client id and client secret in the node Set Netatmo Creds & Tokens. You can use https://flows.nodered.org/node/node-red-contrib-oauth2, or info here https://github.com/guidone/node-red-contrib-netatmo-dashboard#authentication to get it or any other methods

2. Trigger the inject node Set first info

3. Then, the inject node Start can be triggered either automatically after a deploy like I configured it (prefered method), or manually if needed

4. It will get the token, refresh token, and will refresh them after the expiration (returned in the payload by Netatmo) - 5 minutes

5. If error in getting the token it will retry after 1 hour. You can add any kind of alert (email, etc) after node Delay 1 hour on error to alert you that there is an error and token was not returned (time out etc)

6. The node Next refresh in... displays the next time the refresh will occur. This one and the associated "delay 1s" can be deleted safely if not needed

7. The token is stored in the global variable NetatmoAccessToken and is used in the http request after the inject node Get Netatmo Data.

flow.json

[{"id":"48c9ba7a4d6a5359","type":"http request","z":"dab956b5.a43dd8","name":"Send refresh request","method":"POST","ret":"obj","paytoqs":"ignore","url":"https://api.netatmo.com/oauth2/token","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"other","valueValue":"application/x-www-form-urlencoded"}],"x":920,"y":1260,"wires":[["1d54981a5bd62dc8"]]},{"id":"493b859e5318d28d","type":"inject","z":"dab956b5.a43dd8","name":"Start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":"1","topic":"","payload":"","payloadType":"date","x":150,"y":1220,"wires":[["fef4ac6631a48776"]]},{"id":"fef4ac6631a48776","type":"change","z":"dab956b5.a43dd8","name":"Set param request refresh","rules":[{"t":"set","p":"payload","pt":"msg","to":"{\"grant_type\":\"refresh_token\",\"refresh_token\":$globalContext(\"NetatmoRefreshToken\"),\"client_id\":$globalContext(\"NetatmoClientId\"),\"client_secret\":$globalContext(\"NetatmoClientSecret\")}","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":670,"y":1260,"wires":[["48c9ba7a4d6a5359"]]},{"id":"65f4b6541bc97008","type":"change","z":"dab956b5.a43dd8","name":"Store tokens & delay","rules":[{"t":"set","p":"NetatmoAccessToken","pt":"global","to":"payload.access_token","tot":"msg"},{"t":"set","p":"NetatmoRefreshToken","pt":"global","to":"payload.refresh_token","tot":"msg"},{"t":"set","p":"delay","pt":"msg","to":"payload.expires_in*1000-300000","tot":"jsonata"},{"t":"set","p":"status","pt":"msg","to":"OK","tot":"str"},{"t":"set","p":"NetatmoLastUpdateTime","pt":"global","to":"","tot":"date"}],"action":"","property":"","from":"","to":"","reg":false,"x":1340,"y":1200,"wires":[["55d78fcc3650ef5c","97c519fc69b9b6e9"]]},{"id":"371e46c1b4744f71","type":"change","z":"dab956b5.a43dd8","name":"Set Netatmo Creds & Tokens","rules":[{"t":"set","p":"NetatmoClientId","pt":"global","to":"Your_client_id","tot":"str"},{"t":"set","p":"NetatmoClientSecret","pt":"global","to":"Your_client_secret","tot":"str"},{"t":"set","p":"NetatmoAccessToken","pt":"global","to":"Your_first_token","tot":"str"},{"t":"set","p":"NetatmoRefreshToken","pt":"global","to":"Your_first_refres_token","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":660,"y":1180,"wires":[[]]},{"id":"acb161c2964a5538","type":"inject","z":"dab956b5.a43dd8","name":"Set first info","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":130,"y":1180,"wires":[["371e46c1b4744f71"]]},{"id":"5dd80c83a73ef692","type":"inject","z":"dab956b5.a43dd8","name":"Stop","props":[{"p":"reset","v":"1","vt":"num"},{"p":"status","v":"STOP","vt":"str"},{"p":"delay","v":"1","vt":"num"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":150,"y":1300,"wires":[["55d78fcc3650ef5c","97c519fc69b9b6e9","586be2751752f395"]]},{"id":"1d54981a5bd62dc8","type":"switch","z":"dab956b5.a43dd8","name":"All ok ?","property":"payload","propertyType":"msg","rules":[{"t":"hask","v":"access_token","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1120,"y":1260,"wires":[["65f4b6541bc97008"],["ffd6c9bccc7f0c84"]],"outputLabels":["Response OK","Error"]},{"id":"ffd6c9bccc7f0c84","type":"change","z":"dab956b5.a43dd8","name":"Delay 1 hour on error","rules":[{"t":"set","p":"delay","pt":"msg","to":"3600000","tot":"num"},{"t":"set","p":"status","pt":"msg","to":"KO","tot":"str"},{"t":"set","p":"NetatmoLastUpdateTime","pt":"global","to":"","tot":"date"}],"action":"","property":"","from":"","to":"","reg":false,"x":1340,"y":1320,"wires":[["55d78fcc3650ef5c","97c519fc69b9b6e9"]]},{"id":"55d78fcc3650ef5c","type":"delay","z":"dab956b5.a43dd8","name":"Refresh","pauseType":"delayv","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":460,"y":1260,"wires":[["fef4ac6631a48776"]]},{"id":"97c519fc69b9b6e9","type":"delay","z":"dab956b5.a43dd8","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"allowrate":false,"outputs":1,"x":460,"y":1320,"wires":[["586be2751752f395"]]},{"id":"586be2751752f395","type":"function","z":"dab956b5.a43dd8","name":"Next refresh in...","func":"var d = new Date();\nvar n = d.getTime();\nvar start_time = global.get('NetatmoLastUpdateTime')||0;\nvar duration = msg.delay/1000;\nvar elapsed_time = Math.round((n - start_time)/1000);\n\nvar remains = new Date(null);\nremains.setSeconds(duration - elapsed_time); // specify value for SECONDS here\nvar remains_h = remains.toISOString().substr(11, 8);\nif (msg.status == \"STOP\") {\n    node.status({});\n    msg.remains = \"00:00:00\";\n    msg.payload = 0;\n    return [null, msg];\n}\nif (elapsed_time < duration) {\n    node.status({fill:msg.status == \"OK\" ? \"blue\" : \"red\",shape:\"dot\", text:remains_h});\n    msg.remains = remains_h;\n    return [ msg, null ];\n}\n\n","outputs":2,"noerr":0,"initialize":"","finalize":"","libs":[],"x":480,"y":1380,"wires":[["97c519fc69b9b6e9"],[]]},{"id":"944f8def659f9d52","type":"change","z":"dab956b5.a43dd8","name":"","rules":[{"t":"set","p":"token","pt":"msg","to":"\"Bearer \" & $globalContext(\"NetatmoAccessToken\")\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":1460,"wires":[["b96f28348953dfca"]]},{"id":"b96f28348953dfca","type":"http request","z":"dab956b5.a43dd8","name":"Netatmo Https","method":"GET","ret":"obj","paytoqs":"ignore","url":"https://api.netatmo.com/api/getstationsdata","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"other","keyValue":"Authorization","valueType":"msg","valueValue":"token"}],"x":680,"y":1460,"wires":[["7df73f6e51fbc4a5"]]},{"id":"7f8ab2a7eb362ac8","type":"inject","z":"dab956b5.a43dd8","name":"Get Netatmo Data","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":1460,"wires":[["944f8def659f9d52"]]},{"id":"7df73f6e51fbc4a5","type":"debug","z":"dab956b5.a43dd8","name":"Netatmo Data","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":900,"y":1460,"wires":[]}]

You're very welcome. I'm happy to help!
As you are calling the URLs regularly, the tokens do not change

For save storing of the credentials to share the flow, I recommend node-red-contrib-credentials. It stores node information in flows.json and any values setup as credentials into an encrypted separate flows_cred.json file of your node-red instance.

Change of flow from @matz33 is simple:

Today, Netatmo informed about upcoming authentication changes, starting May 29, 2024:

Authentication update (second notice)

This is a reminder regarding our authentication update following the release initially planned for last fall which we decided to postpone in order to give our users more time.

Dear Netatmo developer,
Today, when you refresh an Access Token using the associated endpoint https://api.netatmo.com/oauth2/token, the Netatmo servers respond with a pair of tokens: an Access Token and a Refresh Token.

If the previous Access Token is still valid, the newly returned access token is identical but has an expiration time extended by 3 hours.

The Refresh Token is not renewed.

Starting May 29, 2024, this behavior will change to comply with the OAuth2 Authorization Framework RFC recommendations (section 10.4) and improve the security of our users' data.

When refreshing the tokens, the Access Token and Refresh Token will be different from the previous ones and the old tokens will be invalidated.

What does this mean for you?

If you already store the tokens returned by the API when refreshing your tokens, this change will not affect you.

If you do not update the tokens when refreshing them, your users will be logged out because the old tokens will be invalidated.

If for a given user, you have stored the same token in different places (for example: an application and its widget), you must have a way to synchronize them because the new token will be applied for the entire application.

Sincerely,

Legrand - Netatmo - Bticino

Does this impact our current workflow? Thanks!

Hi, thanks for alerting on this.
The workflow complies to the Oauth2 requirments : in the node "Store tokens & delay", it stores the tokens returned by Netatmo when refreshing them :

Thanks for confirming @matz33 ! Only to be safe ;) BR/JO!

Thx a lot for this great flow @matz33 !

You're welcome :)

Hi @matz33

FYI:
Netatmo applied the OAuth2 change yesterday and not on the 29th of May as initially communicated to developers.
I only want to let you know about...

For me it is working right now but I did not restarted Node-Red yet.
Starting with yesterday I faced many issues where the Netatmo API is used (Homebridge plugins, CARROT weather app,…)
After restarting the services of different plugins or restarting apps with are using the API, the connection was impacted.

I did not verified this yet on your workflow.
Has anybody impacts right now?

Here, the latest info from the DEV-Support about it.

Hello Johannes,

We just did a modification on the token retrieval process :

When you refreshed an access_token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers responded with a couple of tokens : an access_token and a refresh_token. If the previous access_token was still valid, the refresh_token value was never renewed

Starting from yesterday, this behavior changed to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users
When refreshing tokens, access_token and refresh_token values will be automatically renewed and former tokens invalidated

So, if you do not update and use the new refresh_token value when refreshing your access_token, your users will be disconnected after 3 hours and you will retrieve an “invalid_grant” error
To fix it, you need to update the tokens value as soon as you get the newly generated ones

Please note that we are not linked to third-party apps or to their developments. It’s up to them to follow our token renewal process

Have a good day,
Leslie - Community Manager

Thanks and BR/JO!

Hi @jostrasser
On my side the flow is still working well and updates the data correctly.
Regards
Matz33

Hi @jostrasser On my side the flow is still working well and updates the data correctly. Regards Matz33

Also after a restart of Node-Red? ;)

@jostrasser
Yes, for sure, if you follow the instructions in the comments - exactly what you posted there on 07.December 2023:

settings.js of Nod-Red:

contextStorage: { default: { module:"localfilesystem" }, },

Now the keys are stored in the folder of Node-Red under "context" and a restart of the Node-Red instance isn´t breaking the workflow!

Hi @jostrasser On my side the flow is still working well and updates the data correctly. Regards Matz33

Also after a restart of Node-Red? ;)

I've just restarted it, and yes, still working. As soon as the context are stored locally , why wouldn't it work ?

@matz33 & @kailausberg

Great, thanks! Yes, the context is correctly stored on my side.
I am only a little bit afraid to test this on a vacation trip and I only wanted to verify it :-P

A big thanks to you :)

Can confirm that ;) Yesterday Netatmo changed their api to the new authorizatzion process finally. Lot of users had to get a new token pair from the dev portal after the change - me as well. After this everythings works fine again :)

Another update from Netatmo, but not relevant here I think... only FYI ;)

Thanks for forwarding the info ! Indeed I asked the teams to add in this deployment also the possibility to have multi-sessions (otherwise, any other connection would have reset the tokens and so making all devices unmanageable via the API). So it took a bit of time for this development

Have a good day,
Leslie - Community Manager

Yesterday it stopped working completely. Tried a restart today and it still doesn't work.

EDIT: after generating the token again, everything is fine and it continues to work. :)

Can confirm that ;) Yesterday Netatmo changed their api to the new authorizatzion process finally. Lot of users had to get a new token pair from the dev portal after the change - me as well. After this everythings works fine again :)

Yesterday it stopped working completely. Tried a restart today and it still doesn't work.

@S474N can you try to create a fresh token pair?
See: https://gist.github.com/matz33/843165d0046cac534864b65e633801a5?permalink_comment_id=5078029#gistcomment-5078029

After generating the token again, everything is fine and it continues to work. :) Thank you!

It's strange that you had to regenerate a pair of tokens. For my part, I didn't have to.

Does someone has same issue as me? When I reboot NR server it does not work afterward: Invalid Access Token, if I generate new tokens on Netatmo dev. and put them to the Set first info node and then press Start it works again until next reboot. I of course have auto inject of Set first info node enabled after there reboot, so all the tokens are filled in to global variables automatically after restart. Thanks for implementation anyway!

@Kositch As maybe already seen by you, I am experiencing the same with the Homebridge-Eveatmo plugin but not with this Node-Red workflow (I restarted only one time since the OAuth2 change and it worked). I am on vacation at the moment but will resume testing when I am back at home.

skrollme/homebridge-eveatmo#81 (comment)

@matz33
First of all, many thanks for the great work! In principle it really works great.

But either I am missing something or there is a (quite big) drawback. Every time I deplay, I have to recreate a new token pair, otherwise I get the error "invalid token". In Addition, I manually have to restart the refresh manually (which can be forgotten easily when you are working on other flows).

Do you have any hint for me to overcome these litte problems?

Does not work for me. On the first refresh I get by "Send refresh request":

error: "invalid_request"
error_description: "Invalid grant_type parameter or parameter missing"

where "Set param request refresh" look ok to me:

refresh_token: "xx|xx"
client_id: "xx"
client_secret: "xx"

Any suggestion how to fix this?

Does not work for me. On the first refresh I get by "Send refresh request":

error: "invalid_request" error_description: "Invalid grant_type parameter or parameter missing"

where "Set param request refresh" look ok to me:

refresh_token: "xx|xx" client_id: "xx" client_secret: "xx"

Any suggestion how to fix this?

It seems that you are doing something (what ever) wrong. The flow itself works flawless.

In my "Set Netatmo Creds & Tokens" node, there is another order with 4 (not 3) credentials:
NetatmoClientId: "xx"
NetatmoClientSecret: "xx"
NetatmoAccessToken: "xx|xx"
NetatmoRefreshToken: "xx|xx"

Btw: you need to scroll in the node to see all...

Can someone please confirm, that after NR server restart it works automatically as expected? Not sure if there is something wrong at me or it is "normal" behaviour that I need to enter new tokens upon NR reboot. Thanks!

Can someone please confirm, that after NR server restart it works automatically as expected? Not sure if there is something wrong at me or it is "normal" behaviour that I need to enter new tokens upon NR reboot. Thanks!

If you need to enter tokens upon NR reboot, that means that you don't store variable on filesystem, so they are lost when you restart. Have you applied the procedure explained here ? https://gist.github.com/matz33/843165d0046cac534864b65e633801a5?permalink_comment_id=4785938#gistcomment-4785938

Can someone please confirm, that after NR server restart it works automatically as expected? Not sure if there is something wrong at me or it is "normal" behaviour that I need to enter new tokens upon NR reboot. Thanks!

i confirm than it's works after reboot NR server with the store variables on local filesystem.

Can someone please confirm, that after NR server restart it works automatically as expected? Not sure if there is something wrong at me or it is "normal" behaviour that I need to enter new tokens upon NR reboot. Thanks!

If you need to enter tokens upon NR reboot, that means that you don't store variable on filesystem, so they are lost when you restart. Have you applied the procedure explained here ? https://gist.github.com/matz33/843165d0046cac534864b65e633801a5?permalink_comment_id=4785938#gistcomment-4785938

Sorry, not see that, will try when I fill learn how to edit those files NR config files manully outside NR.

But one more question - if I set global variables with current tokens after NR reboot using injection of "Set info first" the tokens should work, no? The issue is that I need to generate new ones on dev.netatmo after reboot, than I inject them using "Set info first" to global variables and it works again... Mayby I am not right, just to understand that.

@Kositch: If the tokens are stored in the local NR filesystem you can reboot your system any time and have to do nothing more.
Only if your reboot is later than the token is valid (for eg. if you stop the system for some hours and make a reboot), you have to get some new tokens from the dev portal and start the process in NR again with the first step.
To put it in a nutshell: A normal reboot from the running NR system needs no further action...

Hi, i am really a beginner, got my station working before they changed it. It copied the flows, imported them and put all the data in. I created new tokens on the NetAtmo page. Unfortunately, I don't get it working. Could you please help me because I really need the NetAtmo data in my Node red system. This error appears at the end 12.6.2024, 20:59:14node: Netatmo Data
msg : Object
object
_msgid: "ec0bc468.810068"
payload: object
error: object
code: 1
message: "Access token is missing"

Thanks very much for your support.

@morpheus1984

can you try to configure it again (as described in the Readme, read carefully)… you maybe made a mistake at the configuration.

@jostrasser Mhh, i copied the complete flow and added my different tokens. So i think i cant do so much wrong. 🤷🏼 Do i need the additional Oauth2 nodes to get the tokens? Until now i just used the Flow i copied from here

@jostrasser Mhh, i copied the complete flow and added my different tokens. So i think i cant do so much wrong. 🤷🏼 Do i need the additional Oauth2 nodes to get the tokens? Until now i just used the Flow i copied from here

No, this flow and a fresh token pair should be enough. And you are always getting a „Access token is missing“?

@jostrasser checked all again, got new tokens from Netatmo, always same error. Is there something special when working with variables? I don't do that for other things in node red right now.

13.6.2024, 21:13:55node: Netatmo Data
msg : Object
object
payload: object
error: object
code: 1
message: "Access token is missing"
topic: ""
_msgid: "a28c1e48.a4dee"
token: "Bearer 5656ab5465d1c471498b463a|e7344516a7ffbb99XXXXXXXXXXXX"
statusCode: 400
headers: object
responseUrl: "https://api.netatmo.com/api/getstationsdata"
redirectList: array[0]

Thanks for helping.

Update: it all works fine with the original code provided above. I used another oauth2 node called node-red-contrib-oauth2. that does not work. somehow it generates another refresh and access token pair (not sure how that is even possible) and that fails after a while.
I was in contact with a Netatmo supporter, he told me that they deactivated recently the usage of parallel token pair usage. Now only one pair will be valid, all others invalid. That is why this method worked before but now not anymore.
so all fine on my side :) happy with the code provide. thanks @matz33

Again problem with token:

Tested regenerating token, but it doesn't solve the problem.

Not here... works flawless.

Same here,... I also restarted the whole Node-Red installation (because I updated to Node-Red 4.0). All is working flawless.

Again regenerated tokens and it change also first two credetinals. Seems to be ok.

i have the same problem as @morpheus1984 "Access token is missing"... do i need to update Node-RED ?

i have the same problem as @morpheus1984 "Access token is missing"... do i need to update Node-RED ?

yes - you should!