ESP32 & ESPHome Voice Assistant

esp32-voice.yaml

esphome:
  name: esp32-mic-speaker
  friendly_name: esp32-mic-speaker
  on_boot:
     - priority: -100
       then:
         - wait_until: api.connected
         - delay: 1s
         - if:
             condition:
               switch.is_on: use_wake_word
             then:
               - voice_assistant.start_continuous:

esp32:
  board: esp32dev
  framework:
    type: esp-idf
    version: recommended

# Enable logging
logger:

# Enable Home Assistant API
api:

ota:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esp32-Mic-Speaker"
    password: "9vYvAFzzPjuc"

i2s_audio:
  i2s_lrclk_pin: GPIO27
  i2s_bclk_pin: GPIO26

microphone:
  - platform: i2s_audio
    id: mic
    adc_type: external
    i2s_din_pin: GPIO13
    pdm: false

speaker:
  - platform: i2s_audio
    id: big_speaker
    dac_type: external
    i2s_dout_pin: GPIO25
    mode: mono

voice_assistant:
  microphone: mic
  use_wake_word: false
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  speaker: big_speaker
  id: assist

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(assist).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(assist).set_use_wake_word(false);

Regarding high pitch sounds during playback clicks are typically caused by signal integrity issues since these are relatively high speed signals. Especially if the distortion stops if you touch the high speed lines with your hand, move things. You should be able to so solve such issues by using better quality dupont lines with higher load spring mechanisms, soldering the dupont lines without the plastic cases, twisting the high speed lines together with gnd or even adding some grounded tinfoil around the signal wires. Another thing I noticed the INMP441 can pick up alot of noise if the PCB touches a surface that is vibrating. A good solution probably is to use some polyester fiber like in pillows to make sure the MEMS microphone does touch any hard surfaces in casing that you have.

Longer pauses like over 300ms are most likely caused by buffering issues, which is another story altogether.

I'm trying to build this project with a wemos d1 mini, max98357, ICS43434, and adafruit 1314 speaker. I'm having trouble getting the project to build when I have gotten it to build I get constant static/crackle on the speaker. Any help or advice would be greatly appreciated. Thanks!

I resolved this (thanks to another community member) by pulling the speaker pin low on boot using a simple output, haven't heard the crackle since.

Can I ask how? I find this helps but doesn't eliminate it...

esphome:
  name: ${device_name}
  friendly_name: ${friendly_name}
  on_boot:
    - priority: -100
      then:
        - output.turn_off: speaker_pin #suppress crackling during boot

speaker:
  - platform: i2s_audio
    id: spk
    dac_type: external
    i2s_dout_pin:
      number: ${din}
      allow_other_uses: true
    i2s_audio_id: i2s
    mode: mono

output:
  - platform: gpio
    pin: 
      number: ${din}
      allow_other_uses: true
    id: speaker_pin

Thanks!

Edit: I find I get crackling and static until the device gets to DETECTING_WAKE_WORD state.

Edit2: I actually found separating out lrclk and bclk resolved it for me.

i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: ${lrclk_mic}
    i2s_bclk_pin: ${bclk_mic}
  - id: i2s_out
    i2s_lrclk_pin: ${lrclk_amp}
    i2s_bclk_pin: ${bclk_amp}

I'm getting an error when installing the .yaml onto the ESP32 I have. I've pared it down to just the minimums to try and isolate the issue.
When I do a basic, bare-bones install of the ESP-WROOM-32 into Home Assistant using ESPHome, it works fine. I can see the wifi connecting on boot. But with the bare-bones .yaml, that's all it does.
Then I started adding some of the .yaml above in. I first tried changing the framework from Arduino to esp-idf by adding only this section:
esp32:
board: esp32dev
framework:
type: esp-idf
version: recommended
and trying to install. But I get this error that my sub-novice brain doesn't know how to troubleshoot:
INFO ESPHome 2024.11.1
INFO Reading configuration /config/esphome/voice-assistant.yaml...
INFO Generating C++ source...
INFO Compiling app...
Processing voice-assistant (board: esp32dev; framework: espidf; platform: platformio/espressif32@5.4.0)

HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash

• framework-espidf @ 3.40408.0 (4.4.8)
• tool-cmake @ 3.16.9
• tool-ninja @ 1.10.2
• toolchain-esp32ulp @ 2.35.0-20220830
• toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
  Reading CMake configuration...
  -- Component directory /data/cache/platformio/packages/framework-espidf/components/expat does not contain a CMakeLists.txt file. No component will be added
  -- Building ESP-IDF components for target esp32
  -- Configuring incomplete, errors occurred!
  See also "/data/build/voice-assistant/.pioenvs/voice-assistant/CMakeFiles/CMakeOutput.log".

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
CMake Error at /data/cache/platformio/packages/framework-espidf/tools/cmake/build.cmake:201 (message):
Failed to resolve component 'freertos'.
Call Stack (most recent call first):
/data/cache/platformio/packages/framework-espidf/tools/cmake/build.cmake:236 (__build_resolve_and_add_req)
/data/cache/platformio/packages/framework-espidf/tools/cmake/build.cmake:237 (__build_expand_requirements)
/data/cache/platformio/packages/framework-espidf/tools/cmake/build.cmake:518 (__build_expand_requirements)
/data/cache/platformio/packages/framework-espidf/tools/cmake/project.cmake:476 (idf_build_process)
CMakeLists.txt:3 (project)
========================= [FAILED] Took 12.33 seconds =========================