esphome, esp32 ble tracker and Home Assistant mqtt_room sensors

esphome_ble_config.yaml

# MQTT broker configuration
mqtt:
  broker: !secret mqtt_broker
  username: !secret mqtt_username
  password: !secret mqtt_password
  discovery: False # Only if you use the HA API usually
  id: mqtt_client

# Define the room for this ESP32 node
substitutions:
  room_name: bedroom

# Push the room name into a global
globals:
  - id: room_topic
    type: std::string
    initial_value: '"room_presence/${room_name}"'

# Configure the esp32_ble_tracker to push beacon advertisements to MQTT
esp32_ble_tracker:
  on_ble_advertise:
    - then:
        - lambda: |-
            if (x.get_ibeacon().has_value()) {
                std::string uuid;
                esp_bt_uuid_t raw_uuid = x.get_ibeacon().value().get_uuid().get_uuid();

                char sbuf[64];
                char *bpos = sbuf;

                switch (raw_uuid.len) {
                    case ESP_UUID_LEN_128:
                        for (int8_t i = 0; i <= 15; i++) {
                            sprintf(bpos, "%02x", raw_uuid.uuid.uuid128[i]);
                            bpos += 2;

                            if (i == 6 || i == 8 || i == 10 || i == 12)
                                  sprintf(bpos++, "-");
                        }

                        sbuf[47] = '\0';
                        uuid.assign(sbuf);
                        break;
                    default:
                        uuid = x.get_ibeacon().value().get_uuid().to_string();
                        std::transform(uuid.begin(), uuid.end(), uuid.begin(), [](unsigned char c){ return std::tolower(c); });
                        break;
                }

                char mbuf[32] = {0};
                sprintf(mbuf, "-%hu-%hu", x.get_ibeacon().value().get_major(), x.get_ibeacon().value().get_minor());
                uuid.append(mbuf);

                int8_t tx_power = x.get_ibeacon().value().get_signal_power();

                if (tx_power >= 100) {
                  tx_power = -69;
                }

                float dist = pow(10, (float)(tx_power - x.get_rssi()) / (10 * 2));

                if (dist < 50) {
                    ESP_LOGD("ble_adv", "Sending MQTT room update for '%s' (%s): %.03fm (%d rssi, %d sigpow)",
                             x.get_name().c_str(), uuid.c_str(), dist, x.get_rssi(), tx_power);

                    id(mqtt_client).publish_json(id(room_topic), [=](JsonObject root) {
                        root["id"] = uuid;
                        root["name"] = x.get_name();
                        root["distance"] = dist;
                        root["rssi"] = x.get_rssi();
                        root["tx_power"] = tx_power;
                    });
                } else {
                    ESP_LOGD("ble_adv", "Skipping MQTT room update for '%s' (%s): %.03fm (%d rssi, %d sigpow)",
                             x.get_name().c_str(), uuid.c_str(), dist, x.get_rssi(), tx_power);
                }
            }

ha_configuration.yaml

# Configure the sensors in Home Assistant as normal:
sensor:
  - platform: mqtt_room
    name: Test beacon 1
    device_id: aabbccdd-eeff-1122-3344-5566778899aa-0001-0002
    state_topic: room_presence

Hie, i got this error

/config/esphome/esp32-1etg.yaml: In lambda function:
/config/esphome/esp32-1etg.yaml:91:12: error: no matching function for call to 'esphome::mqtt::MQTTClientComponent::publish_json(std::__cxx11::basic_string&, setup()::<lambda(const esphome::esp32_ble_tracker::ESPBTDevice&)>::<lambda(ArduinoJson::JsonObject&)>)'
});
^
In file included from src/esphome/components/mqtt/custom_mqtt_device.h:7:0,
from src/esphome.h:29,
from src/main.cpp:3:
src/esphome/components/mqtt/mqtt_client.h:215:8: note: candidate: bool esphome::mqtt::MQTTClientComponent::publish_json(const string&, const json_build_t&, uint8_t, bool)
bool publish_json(const std::string &topic, const json::json_build_t &f, uint8_t qos = 0, bool retain = false);
^
src/esphome/components/mqtt/mqtt_client.h:215:8: note: no known conversion for argument 2 from 'setup()::<lambda(const esphome::esp32_ble_tracker::ESPBTDevice&)>::<lambda(ArduinoJson::JsonObject&)>' to 'const json_build_t& {aka const std::function<void(ArduinoJson6185_D1::ObjectRef)>&}'
*** [/data/esp32-1etg/.pioenvs/esp32-1etg/src/main.cpp.o] Error 1

Hi
I got this error too

/config/esphome/esp-32-tracker.yaml: In lambda function:
/config/esphome/esp-32-tracker.yaml:85:12: error: no matching function for call to 'esphome::mqtt::MQTTClientComponent::publish_json(std::__cxx11::basic_string&, setup()::<lambda(const esphome::esp32_ble_tracker::ESPBTDevice&)>::<lambda(ArduinoJson::JsonObject&)>)'
});
^
In file included from src/esphome/components/mqtt/custom_mqtt_device.h:7:0,
from src/esphome.h:28,
from src/main.cpp:3:
src/esphome/components/mqtt/mqtt_client.h:215:8: note: candidate: bool esphome::mqtt::MQTTClientComponent::publish_json(const string&, const json_build_t&, uint8_t, bool)
bool publish_json(const std::string &topic, const json::json_build_t &f, uint8_t qos = 0, bool retain = false);
^
src/esphome/components/mqtt/mqtt_client.h:215:8: note: no known conversion for argument 2 from 'setup()::<lambda(const esphome::esp32_ble_tracker::ESPBTDevice&)>::<lambda(ArduinoJson::JsonObject&)>' to 'const json_build_t& {aka const std::function<void(ArduinoJson6185_D1::ObjectRef)>&}'
Compiling /data/esp-32-tracker/.pioenvs/esp-32-tracker/libc74/ESPAsyncWebServer-esphome/WebRequest.cpp.o
*** [/data/esp-32-tracker/.pioenvs/esp-32-tracker/src/main.cpp.o] Error 1

I removed the ampersand before "root" in line 60 (following examples seen elsewhere, where publish_json has a very similar idiom but with no ampersand), and it compiled. Still having some difficulty getting it to actually detect devices, but that seems to be the error.

Ah, thanks! That saves me hunting it down!

I've just hit this issue after updating esphome.

If no devices are detected, I'll try to figure out what's changed and get it working again

Word of warning --- the current build of esphome has a busticated publish_json such that your code will compile but will publish a lot of empty messages to the given topic; see esphome issue #3112. Using build 2022.2.4 will work, though.

Word of warning --- the current build of esphome has a busticated publish_json such that your code will compile but will publish a lot of empty messages to the given topic; see esphome issue #3112. Using build 2022.2.4 will work, though.

How can I force it to use build 2022.2.4?

Word of warning --- the current build of esphome has a busticated publish_json such that your code will compile but will publish a lot of empty messages to the given topic; see esphome issue #3112. Using build 2022.2.4 will work, though.

How can I force it to use build 2022.2.4?

If you're only managing esphome through Home Assistant or similar, you can't easily downgrade; AFAICT Home Assistant provides no means to roll back to previous versions of add-ons. If you have a standalone esphome install, though, you can easily install a previous version (and if you have esphome devices you were managing with Home Assistant, you can upgrade them OTA from a standalone installation). How exactly to install a standalone version of esphome on a computer depends a bit on what tools you were using on the computer in the first place --- assuming you have Python (which is what esphome's desktop install uses under the hood), the command pip install esphome==2022.2.4 will do what you want (it's possible pip is actually pip3 on your system, though).

Or you can wait. Pull request #3289 resolves this issue. That one's not in the most recent build of esphome (2022.3.0), but I bet it'll be in the next.

i will have to wait

Word of warning --- the current build of esphome has a busticated publish_json such that your code will compile but will publish a lot of empty messages to the given topic; see esphome issue #3112. Using build 2022.2.4 will work, though.

How can I force it to use build 2022.2.4?

If you're only managing esphome through Home Assistant or similar, you can't easily downgrade; AFAICT Home Assistant provides no means to roll back to previous versions of add-ons. If you have a standalone esphome install, though, you can easily install a previous version (and if you have esphome devices you were managing with Home Assistant, you can upgrade them OTA from a standalone installation). How exactly to install a standalone version of esphome on a computer depends a bit on what tools you were using on the computer in the first place --- assuming you have Python (which is what esphome's desktop install uses under the hood), the command pip install esphome==2022.2.4 will do what you want (it's possible pip is actually pip3 on your system, though).

Or you can wait. Pull request #3289 resolves this issue. That one's not in the most recent build of esphome (2022.3.0), but I bet it'll be in the next.

Thank you very much for answering
i will have to wait

It looks like PR 3289 was included in the latest 2022.3.1 version that is now available

@kmdm - I am now getting a compile failure though, seems the gist still includes the ampersand on line 60

Is there a way we can modify this to include a maximum distance, so that anything over that distance doesn't get reported to HA. Similar to how Room Assistant works?

I know there's a power value, and a calculation to determine the distance so shouldn't be too difficult, however C++(?) isn't my strongest area. A simple if(distance < X) mqtt.publish i guess would do it (in C++)?

Something like this: https://gist.github.com/greghesp/fec67e356d7d761bc626891dbd0d5032
Not sure if the logic or variable types are correctly done though

@greghesp Ah, & removed!

I think you mean if (dist < max_distance) otherwise you're only submitting readings over the max_distance which seems to be the opposite of what you'd like!

Out of interest, what's the use case for filtering here instead of just letting HA decide which room the device is in?

@greghesp Ah, & removed!

I think you mean if (dist < max_distance) otherwise you're only submitting readings over the max_distance which seems to be the opposite of what you'd like!

Out of interest, what's the use case for filtering here instead of just letting HA decide which room the device is in?

Whoops! Bit sleepy still. I'm also not sure if 5 is a correct float.

As for the distance. I've not got presence sensors in every room, so I may go into 1 room and it reports to HA that I'm in room X when I'm not even in a room with a sensor. This way, HA would just report that I'm home, rather than in a room im not

The bottom of this post gives a good example:
https://www.reddit.com/r/homeassistant/comments/ma02pv/how_to_fine_tune_room_assistant_ble/?utm_medium=android_app&utm_source=share

@greghesp I've updated the gist to include a better calculation for distance using the signal power reported by the BLE beacon. I've added a dist filter but just hardcoded it at 50.

(In the HA companion app you can set what this value is)

The check for a tx_power of 100 is for some weird beacons I have which appear to set the transmit power to 100 so it's just badly fixed back to the default I was using before, it could be unnecessary for you but you can monitor the rssi/tx_power values on the esp32 logs and also the MQTT broker since I've added those values there as well (although any dist >= 50 won't be on MQTT obviously).

@kmdm What is the dist value measured in? Im assuming its not meters?

Thank you so much for sharing it.
modify the max_distance and remove almost all false positives. Is there any other parameter that you recommend me to change?

Thanks again

@kmdm Would it be possible to specify only certain UUIDs?

@mihsu81 you can do this on the HA side. It'll just ignore the rest that's sent