Skip to content

Instantly share code, notes, and snippets.

@ndfred

ndfred/glowmqtt.md

Last active Feb 28, 2021
Embed
What would you like to do?

Here is a way to expose most of the Glow Display MQTT information to home assistant through templates. Please make sure you ask Hildebrand support to enable your account for MQTT access before setting everything up.

The first step is to connect Home Assistant to MQTT. If you don't have your own broker hooked up yet, go ahead and set it up (I would suggest using the Integrations UI) and specify your Glow username and password as well as glowmqtt.energyhive.com as the server and 8883 as the port to make sure we're establishing a secure SSL connection.

If you already have your own MQTT server hooked up to Home Assistant, like I do, you will have to set up a bridge between your MQTT server and the Glow one, after that your server will mirror the MQTT messages from the Glow server. You have to do this because Home Assistant does not support connecting to more than one MQTT server.

I will assume you are using Mosquitto as an MQTT server. Start by creating a file named /etc/mosquitto/conf.d/glow.conf with the following content (replace GLOW_USERNAME, GLOW_PASSWORD and GLOW_DEVICE_ID with your relevant information):

connection glowmqtt
address glowmqtt.energyhive.com:8883
remote_username GLOW_USERNAME
remote_password GLOW_PASSWORD
bridge_capath /etc/ssl/certs/
topic SMART/HILD/GLOW_DEVICE_ID in 0

Once that is done, just restart Mosquitto to reload its configuration:

sudo systemctl restart mosquitto

Now you should be able to see your Glow messages over MQTT by using your client of choice.

If that works all right, you can open the Home Assistant configuration.yaml file (for me it is located at /home/homeassistant/.homeassistant/configuration.yaml) and add these new sensors under the sensor section (replace GLOW_DEVICE_ID with the relevant information for your device):

sensor:
  - platform: mqtt
    name: "Home Instant Electricity"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'W'
    value_template: "{{ value_json['elecMtr']['0702']['04']['00']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Electricity"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'kWh'
    value_template: "{{ value_json['elecMtr']['0702']['00']['00']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Daily Electricity"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'kWh'
    value_template: "{{ value_json['elecMtr']['0702']['04']['01']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Weekly Electricity"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'kWh'
    value_template: "{{ value_json['elecMtr']['0702']['04']['30']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Monthly Electricity"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'kWh'
    value_template: "{{ value_json['elecMtr']['0702']['04']['40']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Gas"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'm3'
    value_template: "{{ value_json['gasMtr']['0702']['00']['00']|int(base=16) * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Daily Gas"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'kWh'
    value_template: "{{ value_json['gasMtr']['0702']['0C']['01']|int(base=16)  * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Weekly Gas"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'kWh'
    value_template: "{{ value_json['gasMtr']['0702']['0C']['30']|int(base=16)  * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'
  - platform: mqtt
    name: "Home Monthly Gas"
    state_topic: "SMART/HILD/GLOW_DEVICE_ID"
    unit_of_measurement: 'kWh'
    value_template: "{{ value_json['gasMtr']['0702']['0C']['40']|int(base=16)  * value_json['gasMtr']['0702']['03']['01']|int(base=16) / value_json['gasMtr']['0702']['03']['02']|int(base=16) }}"
    icon: 'mdi:flash'

Now restart your Home Assistant instance, and you should be good to go! All the sensors should appear in the developer tools menu, for instance sensor.home_instant_energy, and update in real time.

If you'd like to play with the MQTT API, you can also check out my python code to connect and fetch data directly from the MQTT server.

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Jul 23, 2020

This is great, thanks.
I’m guessing [03][01]/[03][02]
Is conversion factors ?

Any chance you could add the export bits to your example, I’m struggling to get my head round the conversion from what I suspect is a 2s complement number. (Signed) ?

@ndfred

This comment has been minimized.

Copy link
Owner Author

@ndfred ndfred commented Jul 23, 2020

As I mentioned at the end of the gist, here is where I put my python code to parse it all, it also includes the reference to all the fields I could gather from the ZigBee spec:
https://gist.github.com/ndfred/b373eeafc4f5b0870c1b8857041289a9
["0702"]["03"]["01"] is indeed the multiplier (Metering > Formatting > Multiplier) and ["0702"]["03"]["02"] is the divisor (Metering > Formatting > Divisor). All of these are hexadecimal numbers hence the int(base=16) filter.

@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Jul 23, 2020

Fantastic work, this was just what I needed. For anyone like me running Home Assistant Supervised, glow.conf needs to be saved in /share/mosquitto

I'm with OVO and to get the gas sensor to work I had to change:

value_json['gasMtr']['0702']['0C']['01']
to
value_json['gasMtr']['0702']['04']['01']

@ndfred

This comment has been minimized.

Copy link
Owner Author

@ndfred ndfred commented Jul 23, 2020

Great! I'm with e.on and and my gas meter's consumption is exposed in the Alternative Historical Consumption field (0C) rather than the Historical Consumption field (04), I don't really know why. The electricity consumption shows up as Historical Consumption.

Here is a full dump with the serial numbers scrubbed if people want to compare:

{
  "elecMtr": {
    "0702": {
      "03": {
        "01": "00000001",
        "04": "00",
        "02": "000003E8",
        "07": "XXXXXXXXXXXXX", # Scrubbed
        "03": "FB",
        "08": "",
        "00": "00",
        "06": "00"
      },
      "00": {
        "07": "00000000",
        "01": "000000000000",
        "00": "000000019FAC",
        "14": "02",
        "02": "000000000000"
      },
      "04": {
        "01": "00259B",
        "40": "017A5D",
        "30": "00A547",
        "00": "00000174"
      },
      "02": {
        "00": "10"
      }
    },
    "0705": {
      "00": {
        "01": "FFEB3E62",
        "00": "0084"
      }
    },
    "0708": {
      "01": {
        "01": "E.ON"
      }
    }
  },
  "gasMtr": {
    "0702": {
      "03": {
        "01": "00000001",
        "12": "00",
        "02": "000003E8",
        "07": "XXXXXXXXXX", # Scrubbed
        "03": "2B",
        "08": "",
        "00": "01",
        "06": "80"
      },
      "00": {
        "00": "000000002BD8",
        "14": "02"
      },
      "0C": {
        "01": "003F52",
        "40": "021924",
        "30": "00F6A5"
      },
      "02": {
        "00": "00"
      }
    },
    "0705": {
      "00": {
        "01": "00000000",
        "00": "0C94"
      }
    },
    "0708": {
      "01": {
        "01": ""
      }
    }
  },
  "ts": "2020-07-23 20:46:58",
  "hversion": "GLOW-IHD-01-1v4-SMETS2",
  "time": "5F19F742",
  "zbSoftVer": "1.2.5",
  "gmtime": 1595537218,
  "pan": {
    "rssi": "DC",
    "status": "joined",
    "nPAN": "00",
    "join": "0",
    "lqi": "FF"
  },
  "smetsVer": "SMETS2",
  "ets": "2000-01-01 00:00:00",
  "gid": "XXXXXXXXXXXXXXXX" # Scrubbed
}
@marcoelgordo

This comment has been minimized.

Copy link

@marcoelgordo marcoelgordo commented Aug 5, 2020

Thanks works like a charm for me as well. I believe you should start your gist by saying that MQTT is not enabled by default and that it needs to be requested to support team of the glow box. As soon as they enabled it data started to flow.
Great work.
Merci beaucoup ;-)

@ndfred

This comment has been minimized.

Copy link
Owner Author

@ndfred ndfred commented Aug 24, 2020

Updated the intro to the note, thanks for pointing it out and glad it worked for you!

@sourbrambles

This comment has been minimized.

Copy link

@sourbrambles sourbrambles commented Oct 9, 2020

Afternoon,

I followed the steps above and it was working flawlessly for me for a few days, now I'm just getting a flat line in my graphs and even when it was working I was seeing the below info every few seconds in the mqtt logs, anyone get these errors?

1602249800: Connecting bridge glowmqtt (glowmqtt.energyhive.com:8883)
1602249804: Socket error on client local.core-mosquitto.glowmqtt, disconnecting.

@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Oct 9, 2020

I am also now getting the “Socket error on client” error message every six seconds in my logs. This only started happening a few weeks ago and I emailed Hildebrand who said they couldn’t see any issues at their end. Unlike sourbrambles, I am still getting all the data through to home assistant.

@sourbrambles

This comment has been minimized.

Copy link

@sourbrambles sourbrambles commented Oct 9, 2020

Data has started coming through again sometime in the last hour. So might of been a local issue to me.

@davidjac

This comment has been minimized.

Copy link

@davidjac davidjac commented Dec 4, 2020

I am still waiting for a CAD but understand that I should already be able to get data as I do in Bright APP having signed up authorities, etc.
I have installed the HA plugin for Hildebrand Glow, input the APP Id, user name and password but get nothing, no entities at all. Is there another stage to this. Have I missed something? Thanks

@timsavory

This comment has been minimized.

Copy link

@timsavory timsavory commented Dec 9, 2020

I am trying to achieve the same via the HA plugin, getting the same problem, I am rewriting some of the code and will report back if any luck (if you check your HA logs you may see an error is reported (unbound local error in sensor.py), this is due to the plugin looking for difference resource types than my smart meters)......

@timsavory

This comment has been minimized.

Copy link

@timsavory timsavory commented Dec 9, 2020

OK I figured out the above but for some reason I cannot receive the current readings using the /current resource api but can get it on the resources /readings api. Not very reliable as huge data lag so will move to MQTT once I get the CAD....

A question though, I already have MQTT setup on my HA (listening to a topic from another running docker container which uses user and password generated in HA), how do I configure it to also listen for the Glow MQTT topic which of course will not be on localhost?

Thanks in advance!

@ndfred

This comment has been minimized.

Copy link
Owner Author

@ndfred ndfred commented Dec 9, 2020

how do I configure it to also listen for the Glow MQTT topic which of course will not be on localhost?

You can't do that. What you can do instead, and what I described in this guide (which is a bit counter-intuitive at first), is to merge another server's traffic into your localhost MQTT server, and then connect to that to read all the messages (your local traffic, and the Glow stuff). That's what the /etc/mosquitto/conf.d/glow.conf file does in the guide. If you use HA's embedded MQTT server (is that still a thing?) then you will need to move to Mosquitto as this is a fairly advanced setup.

@sourbrambles, @pg-nut: I would suggest connecting manually with Mosquitto's command line tools or any other client to check that it does work. Haven't had any issues on my end, but I am not checking every day.

@davidjac, @timsavory: you don't need to install the HA plugin, it collects data through the REST API which doesn't give you as much data and is delayed whereas MQTT will send updates right away. Oddly enough the Bright app uses the REST API, which is why you will see HA get data more quickly than their own app (maybe firewalls and scaling is why they aren't going with MQTT with their app). You will need your CAD and registration with your meter to see anything I would assume.

@timsavory

This comment has been minimized.

Copy link

@timsavory timsavory commented Dec 9, 2020

how do I configure it to also listen for the Glow MQTT topic which of course will not be on localhost?

You can't do that. What you can do instead, and what I described in this guide (which is a bit counter-intuitive at first), is to merge another server's traffic into your localhost MQTT server, and then connect to that to read all the messages (your local traffic, and the Glow stuff). That's what the /etc/mosquitto/conf.d/glow.conf file does in the guide. If you use HA's embedded MQTT server (is that still a thing?) then you will need to move to Mosquitto as this is a fairly advanced setup.

@sourbrambles, @pg-nut: I would suggest connecting manually with Mosquitto's command line tools or any other client to check that it does work. Haven't had any issues on my end, but I am not checking every day.

@davidjac, @timsavory: you don't need to install the HA plugin, it collects data through the REST API which doesn't give you as much data and is delayed whereas MQTT will send updates right away. Oddly enough the Bright app uses the REST API, which is why you will see HA get data more quickly than their own app (maybe firewalls and scaling is why they aren't going with MQTT with their app). You will need your CAD and registration with your meter to see anything I would assume.

Thanks very much, so looks like i would be better shifting MQTT to host computer (or in a docker container) then so it can be the bridge, then removing the HA one and put connection in the HA config yaml.

I am aware of the need (or lack of) to use the HA plugin, but I have not received my CAD yet, they have sold out and am on the waiting list!

@davidjac

This comment has been minimized.

Copy link

@davidjac davidjac commented Jan 26, 2021

I'm using HomeAssistant OS (rather than docker, etc) i'm not sure how to get to the directory /etc/mosquitto/conf.d/ I cant see it through Samba, guess its hidden in this type of install? Also where do I get GLOW_DEVICE_ID is it the app id that is standard?
Thanks in advance

@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Jan 26, 2021

For HomeAssistant OS, try /share/mosquitto

@davidjac

This comment has been minimized.

Copy link

@davidjac davidjac commented Jan 26, 2021

Yeah I thought that and yes can access the share folder but the Mosquitto folder doesn't reside there, unless its hidden.

@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Jan 26, 2021

It’s been a while since I set mine up, but I think you need to create the sub directory

@davidjac

This comment has been minimized.

Copy link

@davidjac davidjac commented Jan 26, 2021

Ok I'll try just creating a sub directory and put that file in it. Many thanks for advice pg-nut

@davidjac

This comment has been minimized.

Copy link

@davidjac davidjac commented Jan 27, 2021

That did'nt work but Jane says that I need the Cad before using MQTT anyway.

@danbayliss

This comment has been minimized.

Copy link

@danbayliss danbayliss commented Feb 24, 2021

Hi. Thanks for sharing this! I've managed to get this working and have a few questions

  1. How can I round this to 0 decimal places?
  2. Is there no way to get instant gas?
  3. What is 'Home Electricity' & 'Home Gas'. Is this yearly usage?

Thanks!

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Feb 24, 2021

You can just put
| int
In the end of any of the templates.

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Feb 24, 2021

Instant gas wouldn’t mean much as the instant readings would be very small.
you could take an average over 15 mins or something.

@danbayliss

This comment has been minimized.

Copy link

@danbayliss danbayliss commented Feb 24, 2021

Thanks @B-Hartley. That makes sense about instant gas. I'm still having issues with the rounding. I've put | int at the end and also tried | int | round(2) but both of them still don't make any difference. Any idea?

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Feb 24, 2021

@danbayliss

This comment has been minimized.

Copy link

@danbayliss danbayliss commented Feb 24, 2021

Code is value_template: "{{ value_json['elecMtr']['0702']['04']['01']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16) | int | round(2) }}"

@danbayliss

This comment has been minimized.

Copy link

@danbayliss danbayliss commented Feb 25, 2021

For those of you getting a socket error message when using bridge mode then you need to add the following to mosquitto.conf remote_clientid adduniquename

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Feb 25, 2021

you could change that to:

value_template: "{{ (value_json['elecMtr']['0702']['04']['01']|int(base=16) * value_json['elecMtr']['0702']['03']['01']|int(base=16) / value_json['elecMtr']['0702']['03']['02']|int(base=16)) | int  }}
@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Feb 25, 2021

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Feb 25, 2021

Are you connecting directly to their broker? or bridging it into your broker?
Are you running the MQTT add on?
or a seperate server running a broker?

@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Feb 25, 2021

@danbayliss

This comment has been minimized.

Copy link

@danbayliss danbayliss commented Feb 25, 2021

Thanks @B-Hartley that has sorted it for me.

@pg-nut remote_clientid is the option and you need to set it with a unique name. If you just set it to something someone else might use you'll get a socket error message. Your issue could be different to mine but that's what fixed it for me

@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Feb 25, 2021

@danbayliss

This comment has been minimized.

Copy link

@danbayliss danbayliss commented Feb 25, 2021

This is my full config if you want to compare. When i remove remote_clientid I start to get the socket error message again. The characters at the end of the two clientid's are me just bashing the keyboard to make sure it's something unique on the energyhive.com MQTT server

connection glowmqtt
address glowmqtt.energyhive.com:8883
remote_username bright@email.com
remote_password brightpassword
bridge_capath /etc/ssl/certs
topic SMART/HILD/XXXXXXXXX in 0
local_clientid homeassistantqqwrrw352
remote_clientid danbglow1112335

@pg-nut

This comment has been minimized.

Copy link

@pg-nut pg-nut commented Feb 25, 2021

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Feb 25, 2021

This is mine..........

connection glowmqtt
  address glowmqtt.energyhive.com:8883
  topic SMART/HILD/AAAAA9A999AA in 0
  try_private false
  notifications true
  start_type automatic
  remote_clientid glow
  remote_username my.email@domain.co.uk
  remote_password Password1
  keepalive_interval 300
  cleansession true
  bridge_protocol_version mqttv311
  local_clientid homeassistant
  bridge_cafile /etc/ssl/certs/ca-certificates.crt
@danbayliss

This comment has been minimized.

Copy link

@danbayliss danbayliss commented Feb 26, 2021

Hi. Is it possible to show rolling energy usage? I'm looking to see if I can have a sensor that always shows the last 7 days of usage rather than the sensor which currently shows the current weeks usage and resets at the end of the week?

@B-Hartley

This comment has been minimized.

Copy link

@B-Hartley B-Hartley commented Feb 26, 2021

You could probably achieve this with a statistics sensor.
https://www.home-assistant.io/integrations/statistics/

@BertrumUK

This comment has been minimized.

Copy link

@BertrumUK BertrumUK commented Feb 28, 2021

I like to record my weekly meter reads into an app which I compare when I get my monthly bill. I only got my SMETS2 meters last week and my CAD on Thursday and its working lovely via MQTT and the code above. Instead of having to go into the garage to scroll through the readings and get the totals for each meter it would be nice if the data could be pulled from DCC or the meter. I have checked the feeds and do not see it anywhere - has anyone else figured out if this is possible to pull ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment