Get Netatmo Weather station data

README.md

Unfortunately the Netatmo node https://github.com/guidone/node-red-contrib-netatmo-dashboard is not working : Nodered crashes 5/6 times a day if you use it. The owner is not willing to maintain his code.

Here is a method to get the Netatmo Data without having to install a module, just using http requests. Here is how it works :

1. Enter your token and refresh token + client id and client secret in the node Set Netatmo Creds & Tokens. You can use https://flows.nodered.org/node/node-red-contrib-oauth2, or info here https://github.com/guidone/node-red-contrib-netatmo-dashboard#authentication to get it or any other methods

2. Trigger the inject node Set first info

3. Then, the inject node Start can be triggered either automatically after a deploy like I configured it (prefered method), or manually if needed

4. It will get the token, refresh token, and will refresh them after the expiration (returned in the payload by Netatmo) - 5 minutes

5. If error in getting the token it will retry after 1 hour. You can add any kind of alert (email, etc) after node Delay 1 hour on error to alert you that there is an error and token was not returned (time out etc)

6. The node Next refresh in... displays the next time the refresh will occur. This one and the associated "delay 1s" can be deleted safely if not needed

7. The token is stored in the global variable NetatmoAccessToken and is used in the http request after the inject node Get Netatmo Data.

flow.json

[{"id":"48c9ba7a4d6a5359","type":"http request","z":"dab956b5.a43dd8","name":"Send refresh request","method":"POST","ret":"obj","paytoqs":"ignore","url":"https://api.netatmo.com/oauth2/token","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"other","valueValue":"application/x-www-form-urlencoded"}],"x":920,"y":1260,"wires":[["1d54981a5bd62dc8"]]},{"id":"493b859e5318d28d","type":"inject","z":"dab956b5.a43dd8","name":"Start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":"1","topic":"","payload":"","payloadType":"date","x":150,"y":1220,"wires":[["fef4ac6631a48776"]]},{"id":"fef4ac6631a48776","type":"change","z":"dab956b5.a43dd8","name":"Set param request refresh","rules":[{"t":"set","p":"payload","pt":"msg","to":"{\"grant_type\":\"refresh_token\",\"refresh_token\":$globalContext(\"NetatmoRefreshToken\"),\"client_id\":$globalContext(\"NetatmoClientId\"),\"client_secret\":$globalContext(\"NetatmoClientSecret\")}","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":670,"y":1260,"wires":[["48c9ba7a4d6a5359"]]},{"id":"65f4b6541bc97008","type":"change","z":"dab956b5.a43dd8","name":"Store tokens & delay","rules":[{"t":"set","p":"NetatmoAccessToken","pt":"global","to":"payload.access_token","tot":"msg"},{"t":"set","p":"NetatmoRefreshToken","pt":"global","to":"payload.refresh_token","tot":"msg"},{"t":"set","p":"delay","pt":"msg","to":"payload.expires_in*1000-300000","tot":"jsonata"},{"t":"set","p":"status","pt":"msg","to":"OK","tot":"str"},{"t":"set","p":"NetatmoLastUpdateTime","pt":"global","to":"","tot":"date"}],"action":"","property":"","from":"","to":"","reg":false,"x":1340,"y":1200,"wires":[["55d78fcc3650ef5c","97c519fc69b9b6e9"]]},{"id":"371e46c1b4744f71","type":"change","z":"dab956b5.a43dd8","name":"Set Netatmo Creds & Tokens","rules":[{"t":"set","p":"NetatmoClientId","pt":"global","to":"Your_client_id","tot":"str"},{"t":"set","p":"NetatmoClientSecret","pt":"global","to":"Your_client_secret","tot":"str"},{"t":"set","p":"NetatmoAccessToken","pt":"global","to":"Your_first_token","tot":"str"},{"t":"set","p":"NetatmoRefreshToken","pt":"global","to":"Your_first_refres_token","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":660,"y":1180,"wires":[[]]},{"id":"acb161c2964a5538","type":"inject","z":"dab956b5.a43dd8","name":"Set first info","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":130,"y":1180,"wires":[["371e46c1b4744f71"]]},{"id":"5dd80c83a73ef692","type":"inject","z":"dab956b5.a43dd8","name":"Stop","props":[{"p":"reset","v":"1","vt":"num"},{"p":"status","v":"STOP","vt":"str"},{"p":"delay","v":"1","vt":"num"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":150,"y":1300,"wires":[["55d78fcc3650ef5c","97c519fc69b9b6e9","586be2751752f395"]]},{"id":"1d54981a5bd62dc8","type":"switch","z":"dab956b5.a43dd8","name":"All ok ?","property":"payload","propertyType":"msg","rules":[{"t":"hask","v":"access_token","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1120,"y":1260,"wires":[["65f4b6541bc97008"],["ffd6c9bccc7f0c84"]],"outputLabels":["Response OK","Error"]},{"id":"ffd6c9bccc7f0c84","type":"change","z":"dab956b5.a43dd8","name":"Delay 1 hour on error","rules":[{"t":"set","p":"delay","pt":"msg","to":"3600000","tot":"num"},{"t":"set","p":"status","pt":"msg","to":"KO","tot":"str"},{"t":"set","p":"NetatmoLastUpdateTime","pt":"global","to":"","tot":"date"}],"action":"","property":"","from":"","to":"","reg":false,"x":1340,"y":1320,"wires":[["55d78fcc3650ef5c","97c519fc69b9b6e9"]]},{"id":"55d78fcc3650ef5c","type":"delay","z":"dab956b5.a43dd8","name":"Refresh","pauseType":"delayv","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":460,"y":1260,"wires":[["fef4ac6631a48776"]]},{"id":"97c519fc69b9b6e9","type":"delay","z":"dab956b5.a43dd8","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"allowrate":false,"outputs":1,"x":460,"y":1320,"wires":[["586be2751752f395"]]},{"id":"586be2751752f395","type":"function","z":"dab956b5.a43dd8","name":"Next refresh in...","func":"var d = new Date();\nvar n = d.getTime();\nvar start_time = global.get('NetatmoLastUpdateTime')||0;\nvar duration = msg.delay/1000;\nvar elapsed_time = Math.round((n - start_time)/1000);\n\nvar remains = new Date(null);\nremains.setSeconds(duration - elapsed_time); // specify value for SECONDS here\nvar remains_h = remains.toISOString().substr(11, 8);\nif (msg.status == \"STOP\") {\n    node.status({});\n    msg.remains = \"00:00:00\";\n    msg.payload = 0;\n    return [null, msg];\n}\nif (elapsed_time < duration) {\n    node.status({fill:msg.status == \"OK\" ? \"blue\" : \"red\",shape:\"dot\", text:remains_h});\n    msg.remains = remains_h;\n    return [ msg, null ];\n}\n\n","outputs":2,"noerr":0,"initialize":"","finalize":"","libs":[],"x":480,"y":1380,"wires":[["97c519fc69b9b6e9"],[]]},{"id":"944f8def659f9d52","type":"change","z":"dab956b5.a43dd8","name":"","rules":[{"t":"set","p":"token","pt":"msg","to":"\"Bearer \" & $globalContext(\"NetatmoAccessToken\")\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":1460,"wires":[["b96f28348953dfca"]]},{"id":"b96f28348953dfca","type":"http request","z":"dab956b5.a43dd8","name":"Netatmo Https","method":"GET","ret":"obj","paytoqs":"ignore","url":"https://api.netatmo.com/api/getstationsdata","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"other","keyValue":"Authorization","valueType":"msg","valueValue":"token"}],"x":680,"y":1460,"wires":[["7df73f6e51fbc4a5"]]},{"id":"7f8ab2a7eb362ac8","type":"inject","z":"dab956b5.a43dd8","name":"Get Netatmo Data","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":1460,"wires":[["944f8def659f9d52"]]},{"id":"7df73f6e51fbc4a5","type":"debug","z":"dab956b5.a43dd8","name":"Netatmo Data","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":900,"y":1460,"wires":[]}]

Hm, ok. In my case it‘s work fine. Have you attached an debug node at the end of the response from the refresh request?

I put debug nodes wherever appropriate. while I have enabled the old way, it works for now, the engineers rolled it back.

Don´t worry, they will change it again... :(

Don´t worry, they will change it again... :(

sure

@jostrasser

@AleksCee The workflow is running fine since a few days but after a restart of my Node-Red instance today I had to set a fresh token set to get it working again. Is there a way to prevent this? Thanks!

I had the same issue after updating the nodered server today. Perhaps, the global vars are not restart persistent and should save on disk. This seams to address this: https://gist.github.com/nygma2004/a886a507c57ea83a3a5260338091505b

This seams to be a better way: https://notenoughtech.com/home-automation/preserving-variables-in-nodered/

Ok, I have update my settings.json with the following section. Will see what’s happened on next restart….

contextStorage: {
        default: {
            module:"localfilesystem",
            config: {
              dir:"/data",
              base:"context",
              cache: true,
              flushInterval: 30
            }
        },
    }, 

Ok, I have update my settings.json with the following section. Will see what’s happened on next restart….

it's possible to make this changes via NR web ui?

Ok, I have update my settings.json with the following section. Will see what’s happened on next restart….

it's possible to make this changes via NR web ui?

Sorry, no. This file is the main config file for nodeRED and is not accessible via GUI. In this file are also user/Passwort data and so…

Ok, I have update my settings.json with the following section. Will see what’s happened on next restart….

it's possible to make this changes via NR web ui?

Sorry, no. This file is the main config file for nodeRED and is not accessible via GUI. In this file are also user/Passwort data and so…

thks, i try this been locally

@AleksCee

You are my hero, this did it! Thanks!!

I changed the settings.js file of Node-Red as follows:

contextStorage: {
         default: {
             module:"localfilesystem"
         },
     },

Now my keys are stored in the folder of Node-Red under "context" and a restart of the Node-Red instance isn´t breaking the workflow!

@AleksCee

Now my keys are stored in the folder of Node-Red under "context" and a restart of the Node-Red instance isn´t breaking the workflow!

I have been monitoring locally saved tokens since last night. I see that the time and date of the file has changed, but the tokens inside have not.

@AleksCee

Now my keys are stored in the folder of Node-Red under "context" and a restart of the Node-Red instance isn´t breaking the workflow!

I have been monitoring locally saved tokens since last night. I see that the time and date of the file has changed, but the tokens inside have not.

@mrmaximas Ure right. I monitored it also right know and I can see that the keys which are stored at context/global/global.json are not updated after the Next refresh timer expires... :(

Maybe an issue with the workflow? I can see that the refreshed keys are configured to set them global (global.set) in the node Store tokens & delay but the fresh tokens are not reflected there after an update and that's maybe the reason why they are not reflected in the global.json file. :/

I just updated the nodered version and everything works fine. Perhaps the server flush the data only if necessary. After restart the global.json is from now. Wochenende should watch this out. Sorry I’m also not a nodered specialist… :-(

Hello all

Maybe I should have said that it’s better to set options to store variables on disk. Afaic I would always set this option on by default. Don’t know why it is not by default.

Thanks @matz33 !
Then we should be fine after changing the contextStorage. ;) thank you!

From where you have NetatmoAccessToken? I have only NetatmoClientId and NetatmoClientSecret.

From where you have NetatmoAccessToken? I have only NetatmoClientId and NetatmoClientSecret.

This is explained here : https://github.com/guidone/node-red-contrib-netatmo-dashboard#authentication

From where you have NetatmoAccessToken? I have only NetatmoClientId and NetatmoClientSecret.

This is explained here : https://github.com/guidone/node-red-contrib-netatmo-dashboard#authentication

This might be helpful if your client is a Mac and you can use Paw.
The siplest way if you're on Windows is go to Netatmo Connect and create new credentials for your app.
That will give you the tokens as well.

FlorianDessloch thank you, so created new app. It works, but all nodered's payloads are different :D

@S474N what do you mean by "all nodered payloads are different" ?

Have this my own parser:

@matz33 Thank you very much for this solution, I'm get all the data, since I'm not familia with node-red, can you help me how to get the temperature of the outdoor model? as an example.
Thanks in advance.

Hi @ChriWoPS ,

First activate the debug pane on the right :

Then, in my flow, you can click on the button on the left of the node "Get Netatmo Data" :

This part of the flow will start and you will get the data in the node "Netatmo data". To see the data, go to the debug pane. You will see something like this :

Click on the little arrow and develop until you reach the temperature of the module you need. Here it is like this :

On the right of the value you need, click on "Copy path" icon. In my exemple the copied value is :
payload.body.devices[0].modules[0].dashboard_data.Temperature

This means that the Temperature is stored in msg.payload.body.devices[0].modules[0].dashboard_data.Temperature. You can use this in a change node for exemple :

This exemple will assign the temperature to the global variable Temperature. You can see it in the "Context data" pane :

You can refresh this value each 5 minutes by editing the "Get Netatmo data" node properties like this :

Tell me if that answers to your question

Dear @matz33,
Thank you very much! This is exactly what I needed.
Now I can get all values I need. 👍😀

I can't thank you enough. For 2 days, there was no article I did not read and no method I did not try. With your method, it has been working smoothly for about 12 hours. Since my technical knowledge is not very sufficient, I don't know exactly what I'm doing, but I think it prevents the first access token and refresh token from changing. I wonder if it works with logic like keepalive? Thanks a lot again.

You're very welcome. I'm happy to help!
As you are calling the URLs regularly, the tokens do not change

For save storing of the credentials to share the flow, I recommend node-red-contrib-credentials. It stores node information in flows.json and any values setup as credentials into an encrypted separate flows_cred.json file of your node-red instance.

Change of flow from @matz33 is simple:

Today, Netatmo informed about upcoming authentication changes, starting May 29, 2024:

Authentication update (second notice)

This is a reminder regarding our authentication update following the release initially planned for last fall which we decided to postpone in order to give our users more time.

Dear Netatmo developer,
Today, when you refresh an Access Token using the associated endpoint https://api.netatmo.com/oauth2/token, the Netatmo servers respond with a pair of tokens: an Access Token and a Refresh Token.

If the previous Access Token is still valid, the newly returned access token is identical but has an expiration time extended by 3 hours.

The Refresh Token is not renewed.

Starting May 29, 2024, this behavior will change to comply with the OAuth2 Authorization Framework RFC recommendations (section 10.4) and improve the security of our users' data.

When refreshing the tokens, the Access Token and Refresh Token will be different from the previous ones and the old tokens will be invalidated.

What does this mean for you?

If you already store the tokens returned by the API when refreshing your tokens, this change will not affect you.

If you do not update the tokens when refreshing them, your users will be logged out because the old tokens will be invalidated.

If for a given user, you have stored the same token in different places (for example: an application and its widget), you must have a way to synchronize them because the new token will be applied for the entire application.

Sincerely,

Legrand - Netatmo - Bticino

Does this impact our current workflow? Thanks!

Hi, thanks for alerting on this.
The workflow complies to the Oauth2 requirments : in the node "Store tokens & delay", it stores the tokens returned by Netatmo when refreshing them :

Thanks for confirming @matz33 ! Only to be safe ;) BR/JO!

Thx a lot for this great flow @matz33 !

You're welcome :)