Dockerfile for occu coprocessor flash from hassos

Dockerfile

FROM ubuntu:21.04

WORKDIR /occu_tools

RUN apt-get update && apt-get -y install libusb-1.0-0-dev build-essential git wget && rm -rf /var/lib/apt/lists/*
RUN git clone git://git.zerfleddert.de/hmcfgusb && cd hmcfgusb/ && make
RUN wget -qO coprocessor_update_of_occu_3.55.5.eq3 https://raw.githubusercontent.com/eq-3/occu/3.55.5/firmware/HM-MOD-UART/coprocessor_update.eq3

CMD /bin/bash

How to reflash HM-MOD-RPI-PCB firmware from within HassOS.md

Instructions

This will guide you how to flash the homematic uart modules firmware within hassos.

1. Install the Terminal addon in Hassio and start it in privileged mode
2. Check homematic addon is stopped!
3. Copy above Dockerfile to /root of hassos
4. run following commands to update the firmware

docker build . -t occu_update_image --network=host
docker run -it --rm --privileged --network=host occu_update_image /bin/bash

Inside the container then run

hmcfgusb/flash-hmmoduart -U /dev/ttyAMA0 coprocessor_update_of_occu_3.55.5.eq3

You are done.

Inside the container you can wget any coprocessor firmware and flash that of course.

Hm let me test this again this evening, maybe i wrote the wrong command down. That it hangs on flashing is quite odd, maybe one has to consult the docs of the firmware fashing tool (https://git.zerfleddert.de/cgi-bin/gitweb.cgi/hmcfgusb). The whole setup here is just to get an environment to execute the tool(since hassos does not have any package manager etc.)
AFAIK if you have /dev/ttyAMA0 it is quite sure the serial interface to the module.
Note refashing the firmware only helps with these errors: https://github.com/home-assistant/addons/blob/1ba2b7ba80f1e1238e9ee52c765fd29720f14af3/homematic/DOCS.md#bidcos-errors--coming-from-raspmaticpivccu- when the module previously was used with pivccu or raspmatic.
I think such errors could also occur if the module itself is faulty or seriel interfaces conflict like its the case with rpi3 (especially the timeout error may also occur then)

The whole setup here is just to get an environment to execute the tool(since hassos does not have any package manager etc.)

This is the main reason why it would be great to get this working 👍 because there are some users trying to migrating from RaspberryMatic to HA and are struggling with the docker and CLI interfaces missing.
So this is a approach which would be just great, especially until RaspberryMatic is available as a Docker too and integrated in HA - which seems to be on the todo list of @jens-maus
https://github.com/jens-maus/RaspberryMatic/wiki/Installation-HomeAssistant

Anyhow, if you can give me some hints, I will test this first on the RP3 and later on the RP4 too.
I know that the module is working, because it was used with RaspberryMatic previously, I even tested it last week to verify this again 👍

So I would like to somehow join forces here, and perhaps this gonna work before RaspberryMatic is fully integrated in HA.

Some more information from my side:
The device ttyAMA0 is correct (in my case).
I tried to reflash the HM-MOD-RPI-PCB Firmware using these commands:

sudo su
apt-get update && apt-get -y install libusb-1.0-0-dev build-essential git
systemctl stop fhem
git clone git://git.zerfleddert.de/hmcfgusb
cd hmcfgusb/
make
wget https://raw.githubusercontent.com/eq-3/occu/ee68faf77e42ed5e3641790b43a710a3301cea7e/firmware/HM-MOD-UART/coprocessor_update.eq3
./flash-hmmoduart -U /dev/ttyAMA0 coprocessor_update.eq3

I used raspbian buster lite to flash the Firmware and before i started up everything i´ve made the following changes:

Inside "/boot/config.txt" i added:
enable_uart=1
dtoverlay=pi3-miniuart-bt
core_freq=250

Inside "boot/cmdline.txt" i removed:
console=serial0,115200

I replaced the URL from the wget with the URL from this Instruction above.

It flashes 43 blocks, then it says "Firmware update successfull!"

Before i reflashed the Firmware to the HM-MOD-RPI-PCB i tested it with Raspmatic. Everything works fine there.

When i try to use it via the Homematic Addon in Home Assistant, i get an Error Window with the title "BidCos-RF".

My Homematic Addon Configuration looks like this:
rf_enable: true
wired_enable: false
hmip_enable: false
rf:

• type: CCU2
  device: /dev/ttyAMA0
  wired:
• serial: xy
  key: abc
  ip: 192.168.0.0
  hmip:
• type: HMIP_CCU2
  device: /dev/ttyUSB0
  regahss_reset: false

Im using an Raspberry Pi 3B+ with the original Raspberry Power Supply. So it seems like there is no error with my Hardware.
It has to be an error with the Homematic Addon. I really hope we can fix this together, it would be very nice!

After playing around I figured out that for a Raspi B+ the AMA interface got moved to ttyS0
https://raspberrypi.stackexchange.com/questions/45570/how-do-i-make-serial-work-on-the-raspberry-pi3-or-later-model/45571#45571

So using ttyS0 instead of ttyAMA0 allowed me to flash the interface as in the documentation

But after this, the issue with the Homematic CCU Integration persists. Also if I change AMA to S0 in its configuration.

[16:22:54] INFO: Starting rfd...
2021/02/08 16:22:54.076 <Info> BidCoS-Service started
2021/02/08 16:22:54.077 <Info> XmlRpc Server is listening on TCP port 2001
2021/02/08 16:22:54.079 <Debug> CCU2CommController::init() - Legacy initialization.
2021/02/08 16:22:59.080 <Debug> () CCU2CommController::startCoprocessorApp(): Trying to start coprocessor application
2021/02/08 16:22:59.581  CCU2CommController::startCoprocessorAppThreadFunction(): Send start command
2021/02/08 16:22:59.581  CCU2CommController::SendSystemCommdand()
2021/02/08 16:22:59.581  CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 00 03 18 0a 
2021/02/08 16:23:00.581 <Debug> () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
2021/02/08 16:23:02.581  CCU2CommController::startCoprocessorAppThreadFunction(): Retrying to send start command
2021/02/08 16:23:02.581  CCU2CommController::startCoprocessorAppThreadFunction(): Send start command
2021/02/08 16:23:02.581  CCU2CommController::SendSystemCommdand()
2021/02/08 16:23:02.581  CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 01 03 9e 09 
2021/02/08 16:23:03.581 <Debug> () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
2021/02/08 16:23:04.081 <Fatal error> CCU2CommController::init(): Init failed. Cannot start coprocessor application.
2021/02/08 16:23:04.081  CCU2CommController::SendSystemCommdand()
2021/02/08 16:23:04.082  CCU2CommController::SendSystemCommdand() sending: fd 00 04 00 02 0a 00 3d 10 
2021/02/08 16:23:05.082 <Debug> () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
2021/02/08 16:23:05.082 <Error> () CCU2CommController::setCSMACAEnabled(): Error disabling CSMA/CA.
2021/02/08 16:23:05.083 <Error> No BidCoS-Interface available
2021/02/08 16:23:05.083 <Warning> Error initializing interfaces
2021-02-08 16:23:05,084 [ERROR] BidCosRpcDispatcher - Could not register BidCos-RF_java
2021-02-08 16:23:05,088 [ERROR] BidCosRpcDispatcher - Could not register BidCos-RF_java

After playing around I figured out that for a Raspi B+ the AMA interface got moved to ttyS0
https://raspberrypi.stackexchange.com/questions/45570/how-do-i-make-serial-work-on-the-raspberry-pi3-or-later-model/45571#45571

So using ttyS0 instead of ttyAMA0 allowed me to flash the interface as in the documentation

But after this, the issue with the Homematic CCU Integration persists. Also if I change AMA to S0 in its configuration.

[16:22:54] INFO: Starting rfd...
2021/02/08 16:22:54.076 <Info> BidCoS-Service started
2021/02/08 16:22:54.077 <Info> XmlRpc Server is listening on TCP port 2001
2021/02/08 16:22:54.079 <Debug> CCU2CommController::init() - Legacy initialization.
2021/02/08 16:22:59.080 <Debug> () CCU2CommController::startCoprocessorApp(): Trying to start coprocessor application
2021/02/08 16:22:59.581  CCU2CommController::startCoprocessorAppThreadFunction(): Send start command
2021/02/08 16:22:59.581  CCU2CommController::SendSystemCommdand()
2021/02/08 16:22:59.581  CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 00 03 18 0a 
2021/02/08 16:23:00.581 <Debug> () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
2021/02/08 16:23:02.581  CCU2CommController::startCoprocessorAppThreadFunction(): Retrying to send start command
2021/02/08 16:23:02.581  CCU2CommController::startCoprocessorAppThreadFunction(): Send start command
2021/02/08 16:23:02.581  CCU2CommController::SendSystemCommdand()
2021/02/08 16:23:02.581  CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 01 03 9e 09 
2021/02/08 16:23:03.581 <Debug> () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
2021/02/08 16:23:04.081 <Fatal error> CCU2CommController::init(): Init failed. Cannot start coprocessor application.
2021/02/08 16:23:04.081  CCU2CommController::SendSystemCommdand()
2021/02/08 16:23:04.082  CCU2CommController::SendSystemCommdand() sending: fd 00 04 00 02 0a 00 3d 10 
2021/02/08 16:23:05.082 <Debug> () CCU2CommController::waitForCoProcessorResponse(): Timeout while waiting for response.
2021/02/08 16:23:05.082 <Error> () CCU2CommController::setCSMACAEnabled(): Error disabling CSMA/CA.
2021/02/08 16:23:05.083 <Error> No BidCoS-Interface available
2021/02/08 16:23:05.083 <Warning> Error initializing interfaces
2021-02-08 16:23:05,084 [ERROR] BidCosRpcDispatcher - Could not register BidCos-RF_java
2021-02-08 16:23:05,088 [ERROR] BidCosRpcDispatcher - Could not register BidCos-RF_java

Oh thats interesting.
Your part from the Addon Logfile looks the same as mine.
So i hope we are on the right way to fix it and get it working! :)

I got a little further, after digging into some documentation.

This is what I added to config.txt

enable_uart=1
dtoverlay=pi3-miniuart-bt
dtoverlay=uart1
dtoverlay=disable-bt

Now the bluetooth interface seems to be disabled and the module can be addressed using /dev/ttyAMA0 the previous was with only this lines

enable_uart=1
dtoverlay=pi3-miniuart-bt

The problem is, even though the log now says

2021/02/08 19:29:07.826  RX SERIAL: fd 00 0c 00 00 00 43 6f 
2021/02/08 19:29:07.827  RX SERIAL: 5f 43 50 55 5f 42 4c 72 
2021/02/08 19:29:07.827  RX SERIAL: 51 
2021/02/08 19:29:07.827  RX FRAME: 00 00 00 43 6f 5f 43 50 55 5f 42 4c 
2021/02/08 19:29:07.827 <Debug> () CCU2CommController::handleIdentifyEvent(): Handling incoming identify event.
2021/02/08 19:29:07.827 <Debug> () CCU2CommController::handleIdentifyEvent(): Coprocessor entered bootloader. Starting application.
2021/02/08 19:29:07.828 <Debug> () CCU2CommController::startCoprocessorApp(): Trying to start coprocessor application
2021/02/08 19:29:07 [notice] 453#453: using the "epoll" event method
2021/02/08 19:29:07 [notice] 453#453: nginx/1.18.0
2021/02/08 19:29:07 [notice] 453#453: OS: Linux 5.4.79-v7
2021/02/08 19:29:07 [notice] 453#453: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/02/08 19:29:07 [notice] 453#453: start worker processes
2021/02/08 19:29:07 [notice] 453#453: start worker process 533
2021/02/08 19:29:08.329  CCU2CommController::startCoprocessorAppThreadFunction(): Send start command
2021/02/08 19:29:08.331  CCU2CommController::SendSystemCommdand()
2021/02/08 19:29:08.333  CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 00 03 18 0a 
2021/02/08 19:29:08.763  RX SERIAL: fd 00 04 00 00 04 01 99 
2021/02/08 19:29:08.763  RX SERIAL: 3d 
2021/02/08 19:29:08.764  RX FRAME: 00 00 04 01 
2021/02/08 19:29:08.776  RX SERIAL: fd 00 0d 00 00 00 43 6f 
2021/02/08 19:29:08.776  RX SERIAL: 5f 43 50 55 5f 41 70 70 
2021/02/08 19:29:08.777  RX SERIAL: d8 31 
2021/02/08 19:29:08.777  RX FRAME: 00 00 00 43 6f 5f 43 50 55 5f 41 70 70 
2021/02/08 19:29:08.777 <Debug> () CCU2CommController::handleIdentifyEvent(): Handling incoming identify event.
2021/02/08 19:29:08.778 <Debug> () CCU2CommController::handleIdentifyEvent(): Coprocessor entered application. Life is good.
2021/02/08 19:29:08.791  CCU2CommController::SendSystemCommdand()
2021/02/08 19:29:08.791  CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 01 02 1e 0c 
2021/02/08 19:29:08.793  RX SERIAL: fd 00 0a 00 01 04 02 01 
2021/02/08 19:29:08.794  RX SERIAL: 00 03 01 04 01 be 8a 
2021/02/08 19:29:08.795  RX FRAME: 00 01 04 02 01 00 03 01 04 01 
2021/02/08 19:29:08.795 <Info> CCU2CommController::init(): Coprocessor Bootloader Version is: 1.0.3
2021/02/08 19:29:08.795 <Info> CCU2CommController::init(): Coprocessor Firmware Version is: 1.4.1
2021/02/08 19:29:08.795  CCU2CommController::SendSystemCommdand()
2021/02/08 19:29:08.795  CCU2CommController::SendSystemCommdand() sending: fd 00 04 00 02 0a 00 3d 10 
2021/02/08 19:29:08.797  RX SERIAL: fd 00 04 00 02 04 01 19 
2021/02/08 19:29:08.797  RX SERIAL: 16 
2021/02/08 19:29:08.798  RX FRAME: 00 02 04 01 
2021/02/08 19:29:08.798 <Info> () CCU2CommController::setCSMACAEnabled(): CSMA/CA disabled.
2021/02/08 19:29:08.798  CCU2CommController::SendSystemCommdand()
2021/02/08 19:29:08.798  CCU2CommController::SendSystemCommdand() sending: fd 00 03 00 03 0b 92 39 
2021/02/08 19:29:08.800  RX SERIAL: fd 00 0e 00 03 04 02 50 
2021/02/08 19:29:08.801  RX SERIAL: 45 51 30 35 33 32 30 36 
2021/02/08 19:29:08.802  RX SERIAL: 33 b6 92 
2021/02/08 19:29:08.802  RX FRAME: 00 03 04 02 50 45 51 30 35 33 32 30 36 33 
2021/02/08 19:29:08.802  CCU2CommController::SendSystemCommdand()
2021/02/08 19:29:08.802  CCU2CommController::SendSystemCommdand() sending: fd 00 08 00 04 0e 60 21 82 f4 02 5e 88 
2021/02/08 19:29:08.805  RX SERIAL: fd 00 04 00 04 04 01 19 
2021/02/08 19:29:08.805  RX SERIAL: 6e 
2021/02/08 19:29:08.805  RX FRAME: 00 04 04 01 
2021/02/08 19:29:08.805 <Debug> Default interface is now PEQ0532063
2021/02/08 19:29:08.806 <Info> Current AES key=0, previous AES key=0
2021/02/08 19:29:08.806  CCU2BidcosRemoteInterface::SetAesKeyUser(): Trying to change user keys...
2021/02/08 19:29:08.808  RX SERIAL: fd 00 04 01 05 04 0b 0d 
2021/02/08 19:29:08.809  RX SERIAL: 46 
2021/02/08 19:29:08.809  RX FRAME: 01 05 04 0b 
2021/02/08 19:29:08.809 <Debug> (PEQ0532063) Response status: Input wrong.
2021/02/08 19:29:08.809  (PEQ0532063) CCU2CommController::handleIncomingSerialFrame(): Got an bidcos msg. Response waiting --> Handling as response...
2021/02/08 19:29:08.809  (PEQ0532063) CCU2CommController::handleIncomingSerialFrame(): Response not ok
2021/02/08 19:29:08.809  CCU2BidcosRemoteInterface::SetAesKeyUser(): Previous key empty. Skip setting previous key.
2021/02/08 19:29:08.809 <Debug> CCU2BidcosRemoteInterface::StartInterface(): addr=6a 5d 61 
2021/02/08 19:29:08.812  RX SERIAL: fd 00 04 01 06 04 01 0d 
2021/02/08 19:29:08.812  RX SERIAL: 46 
2021/02/08 19:29:08.812  RX FRAME: 01 06 04 01 
2021/02/08 19:29:08.812 <Debug> (PEQ0532063) Response status: OK.
2021/02/08 19:29:08.812  (PEQ0532063) CCU2CommController::handleIncomingSerialFrame(): Got an bidcos msg. Response waiting --> Handling as response...
2021/02/08 19:29:08.812  (PEQ0532063) CCU2CommController::handleIncomingSerialFrame(): Response ok
2021/02/08 19:29:08.841 <Debug> Device description rf_cfm.xml loaded (1)
2021/02/08 19:29:08.868 <Debug> Device description rf_dim_1t_644_le_v2_4.xml loaded (2)
2021/02/08 19:29:08.882 <Debug> Device description rf_sec_sfa.xml loaded (3)
2021/02/08 19:29:08.887 <Debug> Device description rf_tis_le_v1_0.xml loaded (4)
2021/02/08 19:29:08.904 <Debug> Device description rf_bl_644.xml loaded (5)
2021/02/08 19:29:08.929 <Debug> Device description rf_dim_1pwm_644.xml loaded (6)
2021/02/08 19:29:08.936 <Debug> Device description rf_sec_sco.xml loaded (7)

and life seems good ;) the WebUI is stuck at the loading screen forever.

I think it would make sense to get this communication over to the community
https://community.home-assistant.io/t/can-t-get-homematic-addon-working/275528