Skip to content

Instantly share code, notes, and snippets.

@kylejohnson
Last active October 26, 2023 19:03
Show Gist options
  • Star 13 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save kylejohnson/89784569ad613387346c6844432274f5 to your computer and use it in GitHub Desktop.
Save kylejohnson/89784569ad613387346c6844432274f5 to your computer and use it in GitHub Desktop.
#!/usr/bin/python3
import requests
import json
import re
import paho.mqtt.client as mqtt
import paho.mqtt.publish as publish
# Regex to match the hidden input on the initial log in page
request_validation_re = re.compile(r'<input name="__RequestVerificationToken" type="hidden" value="([^"]*)" />')
# The serial number of your ecowater device
dsn = { "dsn": 'serialnumber' }
# The initial form data
payload = {
"Email" : "username",
"Password" : "password",
"Remember" : 'false'
}
# The headers needed for the JSON request
headers = {
'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language' : 'en-US,en;q=0.5',
'X-Requested-With': 'XMLHttpRequest',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0'
}
# MQTT server details
mqtt_host = '127.0.0.1'
mqtt_port = 1883
mqtt_user = 'username'
mqtt_pass = 'password'
def querySite():
with requests.Session() as s:
# Initial GET request
try:
g = s.get('https://www.wifi.ecowater.com/Site/Login')
except requests.exceptions.RequestException as e:
print(f"Problem connecting to ecowater.com: {e}")
raise
# Grab the token from the hidden input
tokens = request_validation_re.findall(g.text)
payload['__RequestVerificationToken'] = tokens[0]
# Log in to the site
try:
login = s.post('https://www.wifi.ecowater.com/Site/Login', data=payload)
except requests.exceptions.RequestException as e:
print(f"Problem logging in to ecowater.com: {e}")
raise
# Add the correct Referer header
headers['Referer'] = login.url + '/' + dsn['dsn']
# Query the JSON endpoint for the data that we actually want
try:
data = s.post('https://www.wifi.ecowater.com/Dashboard/UpdateFrequentData', data=dsn, headers=headers)
except requests.exceptions.RequestException as e:
print(f"Problem getting JSON from ecowater.com: {e}")
raise
if data.status_code != 200:
print("Status code from ecowater.com was not 200")
raise ValueError("Status code from ecowater.com was not 200")
parseData(data.text)
def parseData(text):
# Load the data in to json
j = json.loads(text)
# Ensure at least one of the returned values makes sense (sanity check)
# All values were "0" once or twice.
assert j['water_flow'] >= 0, "Water flow does not appear to be a real number."
# Placeholder for each message
messages = []
# Extract the next recharge date, into something usable.
# False if we match 'Not Scheduled', else True
# Only sets this topic if we have the initial regex match.
nextRecharge_re = "device-info-nextRecharge'\)\.html\('(?P<nextRecharge>.*)'"
nextRecharge_result = re.search(nextRecharge_re, j['recharge'])
if nextRecharge_result:
msg = {
'topic': 'ecowater/rechargeTomorrow',
'payload': False if nextRecharge_result.group('nextRecharge') == 'Not Scheduled' else True
}
messages.append(msg)
# Delete the elements that we don't want
del j['out_of_salt']
del j['out_of_salt_days']
del j['water_units']
del j['time']
del j['recharge']
# Format each piece of json data in to a mqtt 'message'
for d in j:
msg = {
'topic': 'ecowater/' + d,
'payload': j[d]
}
messages.append(msg)
publishMessages(messages)
def publishMessages(messages):
# Publish the message, consisting of all of the json data
if len(messages) > 0:
publish.multiple(messages, hostname=mqtt_host, port=mqtt_port, auth={'username': mqtt_user, 'password': mqtt_pass})
else:
print('Messages is empty - nothing to publish.')
if __name__ == '__main__':
querySite()
@kchaney9
Copy link

kchaney9 commented Jun 5, 2020

So I changed part of the modified ecowater.py from run_hourly to run_minutely so it's running once a minute which is much better. I tried to use run_every but I guess I couldn't get the formatting right (I'm very new to all of this) because I kept getting error messages with the time and interval part. I did some guessing and checking and googling but couldn't get it working so I've just settled for once a minute until I can get it working more often.

@kylejohnson
Copy link
Author

I wouldn't recommend running it more than once a minute. For one, the script takes 20+ seconds to run and return any data. If you run it every 20 seconds, in this case, then it'll be overlapping on itself. Additionally, EcoWater could crack down on this, and block attempt to block us, if they think we're abusing their (poor excuse for an) API.

@kchaney9
Copy link

kchaney9 commented Jun 8, 2020

Yeah, after living with it for a while, I think once a minute is enough for me anyways.

@blidegn
Copy link

blidegn commented Jun 8, 2020

@kylejohnson: I fully agree with you that it will not make sense to run the script more than once a minute due to the execution time and the fact that EcoWater might block access.

@kchaney9: I think that we need a proper API from EcoWater or somehow be able to poll the device directly in order to monitor the current water flow.

I have installed InfluxDB and Grafana in order to visualise the daily water usage via the sensor water_today and that works very well.

@quinten94b
Copy link

@blidegn: I have your code running via appdaemon on HA, can you explain to me how I set it to pull data every 5 or 15 minutes instead every hour?

@blidegn
Copy link

blidegn commented Jul 30, 2020

@quinten94b: The code needs to be changed in order to pull data more often. Will see if I can get it done within the next week.

@blidegn
Copy link

blidegn commented Aug 2, 2020

@quinten94b: A new parameter "runevery" has been added to ecowater.py and apps.yaml so it is possible to schedule the app to run based on number of minutes instead of hours.

@quinten94b
Copy link

@quinten94b: A new parameter "runevery" has been added to ecowater.py and apps.yaml so it is possible to schedule the app to run based on number of minutes instead of hours.

Sorry for the late response: Thanks for adding the minute schedule. I'm heavily invested in Home Assistant but fairly new to the scripting part. I'm more of a PLC logic guy and I do understand the scripting but for the moment I'm not able to write it myself.

Do you know if there is a way to pull data so I know if my water softener is set to regenerate the next night or not? Some where in the website there is a code hidden with a string behind telling that the softener is set or not

<script type="text/javascript">$('#device-info-nextRecharge').html('Niet ingesteld');</script>
('Niet ingesteld' is Dutch for 'Not planned')

@kylejohnson
Copy link
Author

Do you know if there is a way to pull data so I know if my water softener is set to regenerate the next night or not? Some where in the website there is a code hidden with a string behind telling that the softener is set or not

I should be able to get this working.
What I see when there is not a recharge scheduled is $('#device-info-nextRecharge').html('Not Scheduled');. Do you know what is reported when there is a recharge scheduled?

@quinten94b
Copy link

quinten94b commented Aug 18, 2020

Do you know if there is a way to pull data so I know if my water softener is set to regenerate the next night or not? Some where in the website there is a code hidden with a string behind telling that the softener is set or not

I should be able to get this working.
What I see when there is not a recharge scheduled is $('#device-info-nextRecharge').html('Not Scheduled');. Do you know what is reported when there is a recharge scheduled?

If I remember correct it's 'Gepland' in Dutch or probably 'Scheduled' in English. But I'm not sure. I will be on the next regeneration in +-10 days.

@kylejohnson
Copy link
Author

Turns out I don't need the answer to my above question. I added some code which essentially looks for Not Scheduled and sets rechargeTomorrow to false, else true.

Is that basically what you're looking for?

[
  {
    "topic": "ecowater/rechargeTomorrow",
    "payload": false
  },
  {
    "topic": "ecowater/online",
    "payload": true
  },
  {
    "topic": "ecowater/salt_level",
    "payload": 3
  },
  {
    "topic": "ecowater/salt_level_percent",
    "payload": 37.5
  },
  {
    "topic": "ecowater/water_today",
    "payload": 5
  },
  {
    "topic": "ecowater/water_avg",
    "payload": 77
  },
  {
    "topic": "ecowater/water_avail",
    "payload": 689
  },
  {
    "topic": "ecowater/water_flow",
    "payload": 0
  },
  {
    "topic": "ecowater/rechargeEnabled",
    "payload": true
  }
]

@quinten94b
Copy link

Yes I'm looking to something like that. If I know it's not planned, I also know when it's planned.

@kylejohnson
Copy link
Author

I've updated the gist with my most recent version. I've restructured the code (it still produces the same results), and also added in the ecowater/rechargeTomorrow topic. From there, you could set up a mqtt binary sensor in home assistant with something like the following:

binary_sensor:
  - platform: mqtt
    name: "Water Softener Recharge Tomorrow"
    state_topic: "ecowater/rechargeTomorrow"
    payload_on: True
    payload_off: False

Hope this helps.

@BradleyFord
Copy link

I'm just in the market for a water softener, if we can get this working in HA that would actually push me towards buying their product

@barleybobs
Copy link

@kylejohnson I have developed a Python library for interfacing with Ecowater softeners and was wondering if you are ok with me publishing it as it uses your code as a base. You will be attributed and I am intending to release it under the MIT license. Thanks!

@kylejohnson
Copy link
Author

@kylejohnson I have developed a Python library for interfacing with Ecowater softeners and was wondering if you are ok with me publishing it as it uses your code as a base. You will be attributed and I am intending to release it under the MIT license. Thanks!

That sounds great! I'd be honored. Mind if I take a look at what you wrote?

Side note, I've been meaning to put out a video on getting real-time water usage from the softener via hardware modification.

@barleybobs
Copy link

That sounds great! I'd be honored. Mind if I take a look at what you wrote?

Sure I'll share the pypi and github links here when I get it uploaded!

@barleybobs
Copy link

@kylejohnson

Thanks for all of your work regarding pulling data from Ecowater water softeners. You have been credited on both the Github and PyPI pages!

You can find the library on:
Github - https://github.com/barleybobs/ecowater-softener
PyPI - https://pypi.org/project/ecowater-softener/

@iamelk
Copy link

iamelk commented Sep 1, 2021

Just found this! Is this available as a HA integration / HACS install by any chance or do I need to run appdaemon or a cron job to trigger the script?

@barleybobs
Copy link

@iamelk

It is almost a HACS intergration. I'm just finishing some stuff off regarding it. I'll let you know when it releases!

@iamelk
Copy link

iamelk commented Sep 7, 2021

Thats amazing - thanks @barleybobs ! Ready and willing to test when its out!

@PanMat
Copy link

PanMat commented Jan 27, 2022

The code seems to be working like a charm for me in HA...thanks for publishing it!

The "ecowater/recharge" topic seems to be publishing a list which seems to be confusing the HA sensor, attached are value of payload on this topic when ecowater is recharging and when it is not.
Ecowater

Is there a way to clean up the payload (for ecowater/recharge) in the script or create another binary sensor that shows the current status of water softener recharging at any given moment?

@Benny-Git
Copy link

Thanks for that script. I updated it to set the last messages to be retained, so a new started subscriber always gets the latest values immediately: https://gist.github.com/Benny-Git/0a73a280116fc61f8e0a21e9d6e62d8d

@BillyFKidney
Copy link

BillyFKidney commented Nov 15, 2022

I agree that a custom integration is the right way forward. I have in the mean time created a solution using Appdaemon. The solution is based on a modified version of the script ecowater.py that @kylejohnson has written.

Below what I have done to get it running:

  1. Install AppDaemon Add-on via the Supervisor and start it
  2. Upload the modified version of ecowater.py and place in this folder: /config/appdaemon/apps (the file can just be uploaded via the File Editor integration in HA)
  3. Add the content of apps.yaml to /config/appdaemon/apps/apps.yaml
  4. Add Ecowater credentials and DSN to the secrets file /config/secrets.yaml
    ecowater_user:
    ecowater_password:
    ecowater_dsn:
  5. Define the schedule via the parameters below in apps.yaml. The schedule below is hourly 15:00 min past and daily at 23:59:00. The schedules can be disabled by removing the parameters from the file.
    dailyruntime: [23, 0, 59]
    hourlyruntime: [0, 15, 0]
  6. Restart AppDaemon in order to read the information from the secrets file

Currently there are two log statements (self.log...) in ecowater.py - so it is possible to see if the app is running and able to fetch data. The log output can be seen from the AppDaemon plugin (Log tab) via the Supervisor. AppDaemon will reload the code whenever one of the files are saved, so it is easy to check if it is working by just setting the hourlyruntime a minute later than current time and save the file.

I am new to Python, AppDaemon and github so the code is not perfect and might have to be done differently - but it works in my HA setup.

FYI: apps.yaml has the ecowater_website is incorrectly listed, the correct site would be https://wifi.ecowater.com/Site/Login (no www)

Looks like ecowater.py had www listed too:
# Query the JSON endpoint for the data that we actually want
data = s.post('https://wifi.ecowater.com/Dashboard/UpdateFrequentData', data=dsn, headers=headers)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment