Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
#!/usr/bin/python3
import requests
import json
import re
import paho.mqtt.client as mqtt
import paho.mqtt.publish as publish
# Regex to match the hidden input on the initial log in page
request_validation_re = re.compile(r'<input name="__RequestVerificationToken" type="hidden" value="([^"]*)" />')
# The serial number of your ecowater device
dsn = { "dsn": 'serialnumber' }
# The initial form data
payload = {
"Email" : "username",
"Password" : "password",
"Remember" : 'false'
}
# The headers needed for the JSON request
headers = {
'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language' : 'en-US,en;q=0.5',
'X-Requested-With': 'XMLHttpRequest',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0'
}
# MQTT server details
mqtt_host = '127.0.0.1'
mqtt_port = 1883
mqtt_user = 'username'
mqtt_pass = 'password'
def querySite():
with requests.Session() as s:
# Initial GET request
try:
g = s.get('https://www.wifi.ecowater.com/Site/Login')
except requests.exceptions.RequestException as e:
print(f"Problem connecting to ecowater.com: {e}")
raise
# Grab the token from the hidden input
tokens = request_validation_re.findall(g.text)
payload['__RequestVerificationToken'] = tokens[0]
# Log in to the site
try:
login = s.post('https://www.wifi.ecowater.com/Site/Login', data=payload)
except requests.exceptions.RequestException as e:
print(f"Problem logging in to ecowater.com: {e}")
raise
# Add the correct Referer header
headers['Referer'] = login.url + '/' + dsn['dsn']
# Query the JSON endpoint for the data that we actually want
try:
data = s.post('https://www.wifi.ecowater.com/Dashboard/UpdateFrequentData', data=dsn, headers=headers)
except requests.exceptions.RequestException as e:
print(f"Problem getting JSON from ecowater.com: {e}")
raise
if data.status_code != 200:
print("Status code from ecowater.com was not 200")
raise ValueError("Status code from ecowater.com was not 200")
parseData(data.text)
def parseData(text):
# Load the data in to json
j = json.loads(text)
# Ensure at least one of the returned values makes sense (sanity check)
# All values were "0" once or twice.
assert j['water_flow'] >= 0, "Water flow does not appear to be a real number."
# Placeholder for each message
messages = []
# Extract the next recharge date, into something usable.
# False if we match 'Not Scheduled', else True
# Only sets this topic if we have the initial regex match.
nextRecharge_re = "device-info-nextRecharge'\)\.html\('(?P<nextRecharge>.*)'"
nextRecharge_result = re.search(nextRecharge_re, j['recharge'])
if nextRecharge_result:
msg = {
'topic': 'ecowater/rechargeTomorrow',
'payload': False if nextRecharge_result.group('nextRecharge') == 'Not Scheduled' else True
}
messages.append(msg)
# Delete the elements that we don't want
del j['out_of_salt']
del j['out_of_salt_days']
del j['water_units']
del j['time']
del j['recharge']
# Format each piece of json data in to a mqtt 'message'
for d in j:
msg = {
'topic': 'ecowater/' + d,
'payload': j[d]
}
messages.append(msg)
publishMessages(messages)
def publishMessages(messages):
# Publish the message, consisting of all of the json data
if len(messages) > 0:
publish.multiple(messages, hostname=mqtt_host, port=mqtt_port, auth={'username': mqtt_user, 'password': mqtt_pass})
else:
print('Messages is empty - nothing to publish.')
if __name__ == '__main__':
querySite()
@kylejohnson
Copy link
Author

kylejohnson commented May 18, 2020

Hey everyone, I'm going to spend some time today trying to get this working in HomeAssistant directly. It isn't how I use the script - I just call it happily from cron - but I'll see what I can do, and report back.

@kylejohnson
Copy link
Author

kylejohnson commented May 20, 2020

I don't want to go down the AppDaemon route and python_script: can't use imports, so it looks like the best option is for me to make this in to a custom integration.
In the mean time, it looks like it is fairly easy to add a cronjob to a docker host, so that should be able to hold some of you over.

@kchaney9
Copy link

kchaney9 commented May 20, 2020

If you make a custom integration that would be awesome! I have no experience with cron so I'm not sure where to begin with that but I'll try to look into it. I really appreciate you taking the time to work on this.

@blidegn
Copy link

blidegn commented May 29, 2020

I agree that a custom integration is the right way forward. I have in the mean time created a solution using Appdaemon.
The solution is based on a modified version of the script ecowater.py that @kylejohnson has written.

Below what I have done to get it running:

  1. Install AppDaemon Add-on via the Supervisor and start it
  2. Upload the modified version of ecowater.py and place in this folder: /config/appdaemon/apps (the file can just be uploaded via the File Editor integration in HA)
  3. Add the content of apps.yaml to /config/appdaemon/apps/apps.yaml
  4. Add Ecowater credentials and DSN to the secrets file /config/secrets.yaml
    ecowater_user:
    ecowater_password:
    ecowater_dsn:
  5. Define the schedule via the parameters below in apps.yaml. The schedule below is hourly 15:00 min past and daily at 23:59:00. The schedules can be disabled by removing the parameters from the file.
    dailyruntime: [23, 0, 59]
    hourlyruntime: [0, 15, 0]
  6. Restart AppDaemon in order to read the information from the secrets file

Currently there are two log statements (self.log...) in ecowater.py - so it is possible to see if the app is running and able to fetch data. The log output can be seen from the AppDaemon plugin (Log tab) via the Supervisor. AppDaemon will reload the code whenever one of the files are saved, so it is easy to check if it is working by just setting the hourlyruntime a minute later than current time and save the file.

I am new to Python, AppDaemon and github so the code is not perfect and might have to be done differently - but it works in my HA setup.

@kchaney9
Copy link

kchaney9 commented Jun 4, 2020

Thank you @blidegn for this! I was able to get it working, but I was wondering how I could get it to run much more frequently so I can accurately monitor current water usage.

@kylejohnson
Copy link
Author

kylejohnson commented Jun 5, 2020

Unfortunately, water_flow is not very for monitoring water usage, at is only shows the water flow at the time that the softener reached out and uploaded to the API; water_today is much more useful in that regard.

I've also been attempting to connect a NodeMCU directly to the ecowater's flow sensor, but have had only inaccurate results thus far.

@kchaney9
Copy link

kchaney9 commented Jun 5, 2020

Well I was thinking if I could get it to do a callback every 10 seconds or so it would work for what I'm trying to do. I was able to adjust the hourly runtime for the next minute and ran the sink and it registered the flow because, as you said, it only shows the water flow for the moment it reached out to the API. So if it runs more frequently then it will give me a rough idea of the current water flow.

@kchaney9
Copy link

kchaney9 commented Jun 5, 2020

So I changed part of the modified ecowater.py from run_hourly to run_minutely so it's running once a minute which is much better. I tried to use run_every but I guess I couldn't get the formatting right (I'm very new to all of this) because I kept getting error messages with the time and interval part. I did some guessing and checking and googling but couldn't get it working so I've just settled for once a minute until I can get it working more often.

@kylejohnson
Copy link
Author

kylejohnson commented Jun 8, 2020

I wouldn't recommend running it more than once a minute. For one, the script takes 20+ seconds to run and return any data. If you run it every 20 seconds, in this case, then it'll be overlapping on itself. Additionally, EcoWater could crack down on this, and block attempt to block us, if they think we're abusing their (poor excuse for an) API.

@kchaney9
Copy link

kchaney9 commented Jun 8, 2020

Yeah, after living with it for a while, I think once a minute is enough for me anyways.

@blidegn
Copy link

blidegn commented Jun 8, 2020

@kylejohnson: I fully agree with you that it will not make sense to run the script more than once a minute due to the execution time and the fact that EcoWater might block access.

@kchaney9: I think that we need a proper API from EcoWater or somehow be able to poll the device directly in order to monitor the current water flow.

I have installed InfluxDB and Grafana in order to visualise the daily water usage via the sensor water_today and that works very well.

@quinten94b
Copy link

quinten94b commented Jul 28, 2020

@blidegn: I have your code running via appdaemon on HA, can you explain to me how I set it to pull data every 5 or 15 minutes instead every hour?

@blidegn
Copy link

blidegn commented Jul 30, 2020

@quinten94b: The code needs to be changed in order to pull data more often. Will see if I can get it done within the next week.

@blidegn
Copy link

blidegn commented Aug 2, 2020

@quinten94b: A new parameter "runevery" has been added to ecowater.py and apps.yaml so it is possible to schedule the app to run based on number of minutes instead of hours.

@quinten94b
Copy link

quinten94b commented Aug 17, 2020

@quinten94b: A new parameter "runevery" has been added to ecowater.py and apps.yaml so it is possible to schedule the app to run based on number of minutes instead of hours.

Sorry for the late response: Thanks for adding the minute schedule. I'm heavily invested in Home Assistant but fairly new to the scripting part. I'm more of a PLC logic guy and I do understand the scripting but for the moment I'm not able to write it myself.

Do you know if there is a way to pull data so I know if my water softener is set to regenerate the next night or not? Some where in the website there is a code hidden with a string behind telling that the softener is set or not

<script type="text/javascript">$('#device-info-nextRecharge').html('Niet ingesteld');</script>
('Niet ingesteld' is Dutch for 'Not planned')

@kylejohnson
Copy link
Author

kylejohnson commented Aug 18, 2020

Do you know if there is a way to pull data so I know if my water softener is set to regenerate the next night or not? Some where in the website there is a code hidden with a string behind telling that the softener is set or not

I should be able to get this working.
What I see when there is not a recharge scheduled is $('#device-info-nextRecharge').html('Not Scheduled');. Do you know what is reported when there is a recharge scheduled?

@quinten94b
Copy link

quinten94b commented Aug 18, 2020

Do you know if there is a way to pull data so I know if my water softener is set to regenerate the next night or not? Some where in the website there is a code hidden with a string behind telling that the softener is set or not

I should be able to get this working.
What I see when there is not a recharge scheduled is $('#device-info-nextRecharge').html('Not Scheduled');. Do you know what is reported when there is a recharge scheduled?

If I remember correct it's 'Gepland' in Dutch or probably 'Scheduled' in English. But I'm not sure. I will be on the next regeneration in +-10 days.

@kylejohnson
Copy link
Author

kylejohnson commented Aug 18, 2020

Turns out I don't need the answer to my above question. I added some code which essentially looks for Not Scheduled and sets rechargeTomorrow to false, else true.

Is that basically what you're looking for?

[
  {
    "topic": "ecowater/rechargeTomorrow",
    "payload": false
  },
  {
    "topic": "ecowater/online",
    "payload": true
  },
  {
    "topic": "ecowater/salt_level",
    "payload": 3
  },
  {
    "topic": "ecowater/salt_level_percent",
    "payload": 37.5
  },
  {
    "topic": "ecowater/water_today",
    "payload": 5
  },
  {
    "topic": "ecowater/water_avg",
    "payload": 77
  },
  {
    "topic": "ecowater/water_avail",
    "payload": 689
  },
  {
    "topic": "ecowater/water_flow",
    "payload": 0
  },
  {
    "topic": "ecowater/rechargeEnabled",
    "payload": true
  }
]

@quinten94b
Copy link

quinten94b commented Aug 18, 2020

Yes I'm looking to something like that. If I know it's not planned, I also know when it's planned.

@kylejohnson
Copy link
Author

kylejohnson commented Aug 18, 2020

I've updated the gist with my most recent version. I've restructured the code (it still produces the same results), and also added in the ecowater/rechargeTomorrow topic. From there, you could set up a mqtt binary sensor in home assistant with something like the following:

binary_sensor:
  - platform: mqtt
    name: "Water Softener Recharge Tomorrow"
    state_topic: "ecowater/rechargeTomorrow"
    payload_on: True
    payload_off: False

Hope this helps.

@BradleyFord
Copy link

BradleyFord commented Aug 25, 2020

I'm just in the market for a water softener, if we can get this working in HA that would actually push me towards buying their product

@barleybobs
Copy link

barleybobs commented Aug 25, 2021

@kylejohnson I have developed a Python library for interfacing with Ecowater softeners and was wondering if you are ok with me publishing it as it uses your code as a base. You will be attributed and I am intending to release it under the MIT license. Thanks!

@kylejohnson
Copy link
Author

kylejohnson commented Aug 25, 2021

@kylejohnson I have developed a Python library for interfacing with Ecowater softeners and was wondering if you are ok with me publishing it as it uses your code as a base. You will be attributed and I am intending to release it under the MIT license. Thanks!

That sounds great! I'd be honored. Mind if I take a look at what you wrote?

Side note, I've been meaning to put out a video on getting real-time water usage from the softener via hardware modification.

@barleybobs
Copy link

barleybobs commented Aug 25, 2021

That sounds great! I'd be honored. Mind if I take a look at what you wrote?

Sure I'll share the pypi and github links here when I get it uploaded!

@barleybobs
Copy link

barleybobs commented Aug 25, 2021

@kylejohnson

Thanks for all of your work regarding pulling data from Ecowater water softeners. You have been credited on both the Github and PyPI pages!

You can find the library on:
Github - https://github.com/barleybobs/ecowater-softener
PyPI - https://pypi.org/project/ecowater-softener/

@iamelk
Copy link

iamelk commented Sep 1, 2021

Just found this! Is this available as a HA integration / HACS install by any chance or do I need to run appdaemon or a cron job to trigger the script?

@barleybobs
Copy link

barleybobs commented Sep 7, 2021

@iamelk

It is almost a HACS intergration. I'm just finishing some stuff off regarding it. I'll let you know when it releases!

@iamelk
Copy link

iamelk commented Sep 7, 2021

Thats amazing - thanks @barleybobs ! Ready and willing to test when its out!

@PanMat
Copy link

PanMat commented Jan 27, 2022

The code seems to be working like a charm for me in HA...thanks for publishing it!

The "ecowater/recharge" topic seems to be publishing a list which seems to be confusing the HA sensor, attached are value of payload on this topic when ecowater is recharging and when it is not.
Ecowater

Is there a way to clean up the payload (for ecowater/recharge) in the script or create another binary sensor that shows the current status of water softener recharging at any given moment?

@Benny-Git
Copy link

Benny-Git commented Apr 24, 2022

Thanks for that script. I updated it to set the last messages to be retained, so a new started subscriber always gets the latest values immediately: https://gist.github.com/Benny-Git/0a73a280116fc61f8e0a21e9d6e62d8d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment