Skip to content

Instantly share code, notes, and snippets.

@Paraphraser
Last active May 2, 2024 04:02
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Paraphraser/cad3b0aa6428c58ee87bc835ac12ed37 to your computer and use it in GitHub Desktop.
Save Paraphraser/cad3b0aa6428c58ee87bc835ac12ed37 to your computer and use it in GitHub Desktop.
Compiling GoSungrow

Updating GoSungrow

This gist is intended to help you deal with the following error messages:

  • Error: appkey is incorrect 'er_invalid_appkey
  • Error: unknown error 'Request is not encrypted'

This gist has four parts. The instructions you should follow depend on what you want to do:

Part Explains how to
Part 1 recompile GoSungrow from its source code. This includes the patches that solve both errors.
Part 2 install a patched version of GoSungrow in Home Assistant.
Part 3 use a pre-compiled patched binary outside of Home Assistant. This lets you run GoSungrow from your Terminal command line.
Part 4 use a pre-built Docker image outside of Home Assistant.

The four parts are independent. In other words, you do not need to follow the steps in Part 1 and recompile GoSungrow yourself before you can use the patched version in Parts 2, 3 or 4.

about this gist

The reason this gist has a multi-part structure is historical.

The first version of this gist answered the question, "how do I recompile GoSungrow from its source code?" That material became Part 1.

Part 2 went through several revisions including using a Dockerfile plus the results of recompiling as per Part 1 to produce a new local image for Home Assistant. When triamazikamno provided patched Docker images, Part 2 was replaced with simpler (!) instructions.

Part 3 was added following a suggestion by xpufx which further leverages the work done by triamazikamno and avoids the need for recompiling GoSungrow from its source code.

Part 4 was added following a comment by Lmecano which identified a use-case for deploying the Docker images outside of the confines of Home Assistant.

Part 1 — Compiling GoSungrow

These instructions have been tested on macOS (Darwin) and Raspberry Pi OS (Debian). The Go compiler can be installed on Windows but I don't have any way of testing that.

Assumptions

  • git is installed.
  • wget is installed.

Prepare your system (Linux+macOS)

  1. Create the "go" sub-directory in your home directory:

    $ mkdir -p ~/go
  2. Add the following lines to your ~/.bashrc or ~/.profile as is your preference:

    #=======================================================================
    # Senses if the go compiler is installed
    #=======================================================================
    
    GO_BIN=/usr/local/go/bin
    if [ -x "$GO_BIN/go" ] ; then
       export GOPATH="$HOME/go"
       export PATH="$PATH:$GO_BIN:$GOPATH/bin"
    fi
    unset GO_BIN

    Tip:

    • If you copy/paste into a Windows text editor, make sure your editor saves with Unix ( : 0x0A) line endings, rather than Windows (␍␊ 0x0D 0x0A). Alternatively, use a tool like dos2unix to post-process the file.

Install/Update Go compiler

Linux (Raspberry Pi OS)

  1. Open your browser at the Go downloads page.

  2. Choose an appropriate image. Example:

    $ URL=https://go.dev/dl/go1.21.6.linux-arm64.tar.gz
    $ TARGZ=$(basename $URL)
  3. Also make a note of the SHA256 checksum and assign it to a variable. Example:

    $ HASH=e2e8aa88e1b5170a0d495d7d9c766af2b2b6c6925a8f8956d834ad6b4cacbd9a
  4. Download the image:

    $ wget $URL
  5. Verify the checksum:

    $ shasum -a 256 -c <<< "$HASH *$TARGZ"
    go1.21.6.linux-arm64.tar.gz: OK
  6. Install the compiler (replacing any older version):

    $ sudo rm -rf /usr/local/go
    $ sudo tar -C /usr/local -xzf $TARGZ

macOS

  1. Open your browser at the Go download and install page.
  2. Click the "Download" button.
  3. Run the installer package. This installs/updates as appropriate.

Tip:

  • If you have an older version of Go installed via HomeBrew, you can remove it with:

     $ brew uninstall go

    HomeBrew does not install the Go compiler in /usr/local/go/bin. You will need to adapt the login script commands if you want to use HomeBrew to install/update Go.

Windows

See download and install page.

Confirm compiler installation (Linux+macOS)

  1. Logout and login again so the new login script commands run.

  2. Confirm that GOPATH returns a sensible result:

    $ echo $GOPATH
    /home/pi/go
  3. Confirm that the compiler is present:

    • Linux (Raspberry Pi OS):

       $ go version
       go version go1.21.6 linux/arm64
    • macOS:

       $ go version
       go version go1.20.1 darwin/amd64

Compile GoSungrow

  1. Construct the following directory structure:

    $ mkdir -p ~/go-projects/MickMake
  2. Clone the GoUnify repository:

    $ cd ~/go-projects
    $ git clone https://github.com/MickMake/GoUnify.git
  3. Clone the GoSungrow repository:

    $ cd MickMake
    $ git clone https://github.com/MickMake/GoSungrow.git
  4. Move into the GoSungrow directory:

    $ cd GoSungrow
  5. Apply the patch to fix the er_invalid_appkey problem:

    $ git remote add -t encryption triamazikamno https://github.com/triamazikamno/GoSungrow.git
    $ git pull triamazikamno encryption
    $ git switch encryption

    Notes:

    1. You also need to update the APPKey and, optionally, make sure you are using the correct gateway host for your inverter. These steps are explained later at:

    2. When (if) MickMake/GoSungrow is updated to deal with this problem, you will want to undo the effect of this change and recompile from the master branch:

      $ git switch master
  6. Compile GoSungrow for your native architecture

    $ go mod tidy
    $ go build

Check your work

The next three steps tell you to run GoSungrow like this:

./GoSungrow

The ./ prefix means "run the just-recompiled binary from the working directory". Please do not make the mistake of omitting the ./ prefix because that risks executing an older version of GoSungrow that does not have any of the patches.

1. Confirm GoSungrow compiled

$ ./GoSungrow version
GoSungrow v3.0.7
Self-manage this executable.

Usage:
  GoSungrow version
  GoSungrow version [command]

Examples:
	GoSungrow version  


Available Commands:
  check                   Version	- Check and show any version updates.
  list                    Version	- List available versions.
  info                    Version	- Info on current version.
  latest                  Version	- Info on latest version.
  update                  Version	- Update version of this executable.

Flags: Use "GoSungrow help flags" for more info.

Additional help topics:

Use "GoSungrow version help [command]" for more information about a command.

2. Update the APPKey

If you followed the optional step to apply the patch to fix the er_invalid_appkey problem, you will also need to update the APPKey on any system where you intend to run GoSungrow. See APPKey configuration.

3. Set your gateway (optional)

Follow the instructions to set your iSolarCloud gateway configuration.

4. Confirm success

You can confirm everything is working by running:

$ ./GoSungrow api login
Email:	you@yourdomain.com
Create Date:	Thu Feb 09 13:14:55 CST 2023
Login Last Date:	2023-12-07 12:42:57
Login Last IP:
Login State:	1
User Account:	something
User Id:	999999
User Name:	someone
Is Online:	false
Token:	999999_99999999999999999999999999999999
Token File:	/home/pi/.GoSungrow/AppService_login.json

Cross-compiling

This section is optional. If you don't need to cross-compile, skip down to using the recompiled binary.

I want to run GoSungrow on a Raspberry Pi. Although I can install the Go compiler (as above) and compile GoSungrow on the Pi, it is much faster to do the work on macOS. All these commands are run on macOS:

  1. Cross-compile for 64-bit Raspberry Pi OS:

    $ GOOS=linux GOARCH=arm64 go build -o GoSungrow_linux_arm64
  2. Copy compiled binary to target Raspberry Pi system ("iot-hub"):

    $ scp GoSungrow_linux_arm64 iot-hub:./GoSungrow
    GoSungrow_linux_arm64    100%   90MB  90.0MB/s   00:01    
  3. Check result:

    $ ssh iot-hub file GoSungrow
    GoSungrow: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=T7VxiW_Bb7zpmTzK5Gjk/ek581bp4wUYxRJpI4LW7/67TEDExnbLJooHJ3cURP/8Pi_onP5w88QBDpLdkcP, with debug_info, not stripped
  4. Comparison with "native" compile on macOS:

    $ file GoSungrow
    GoSungrow: Mach-O 64-bit executable x86_64

Cross-compiling helper script

Copy the script below and paste into a file named build-all.sh:

#!/usr/bin/env bash

compile_for() {

   echo "Compiling for $1/$2"
   GOOS=$1 GOARCH=$2 go build -o GoSungrow-${1}-${2}

}

go clean

compile_for darwin amd64
compile_for darwin arm64
compile_for linux amd64
compile_for linux arm64

if [ -n "$(which lipo)" ] ; then
   echo "Constructing universal binary for macOS"
   lipo -create -output GoSungrow-mac GoSungrow-darwin-amd64 GoSungrow-darwin-arm64
fi

The build-all.sh script should be at the path:

~/go-projects/MickMake/GoSungrow/build-all.sh

Give the file execute permission and run it:

$ cd ~/go-projects/MickMake/GoSungrow
$ chmod +x build-all.sh
$ ./build-all.sh

The result of running the script is four binaries:

  • GoSungrow-darwin-amd64 for macOS on Intel chips
  • GoSungrow-darwin-arm64 for macOS on Apple silicon
  • GoSungrow-linux-amd64 for Linux on Intel chips
  • GoSungrow-linux-arm64 for Raspberry Pi

If the lipo tool is available, the script also assembles the first two binaries into:

  • GoSungrow-mac macOS "universal" binary

Using the recompiled binary

There are many ways to use the recompiled binary. About the simplest is to include it in a directory that is in your Unix PATH. The most common pattern for home directories is:

$ mkdir -p ~/.local/bin
$ cp ./GoSungrow ~/.local/bin/.

If the ~/.local/bin directory did not already exist, you may need to logout and login to give your .profile or .bashrc a chance to discover it and add it to your PATH.

Once the binary is in a directory that is in your PATH, you can execute it just by typing the command name (ie without the ./ prefix). For example:

$ GoSungrow api login

Part 2 — Using patched images in HomeAssistant

This is based on material I added to MickMake/GoSungrow Issue 101.

Note:

  • There seems to be a common misconception that Supervised Home Assistant does not use Docker images. You may not be aware of it but that doesn't mean Home Assistant isn't using Docker.

triamazikamno has provided patched images for GoSungrow on DockerHub. To use the correct image for your instance of Home Assistant, proceed as follows:

  1. The "Advanced SSH & Web Terminal" add-on must be installed:

    Must Be Advanced add-on

    Go to Settings » Add-ons » Advanced SSH & Web Terminal. In the:

    • "Configuration" tab:

      • you need to set a username and password. This example uses "hassio" as the username. It turns up in the ssh command below.
    • "Info" tab:

      • "Show in sidebar" should be turned ON.
      • "Protection mode" needs to be turned OFF.
      • if you changed any settings in either tab, click "RESTART". If you don't see "RESTART" then click "START".
  2. Do one of the following:

    • Either – from your support host (Linux, macOS, Windows), replace hassio with the username you set in the "Configuration" tab above and connect to your HA instance:

       $ ssh hassio@homeassistant.local

      The prompt you get is:

       ~ #
      

      The # indicates you are running as root and the ~ indicates that your working directory is root's home directory.

    • Or – from the HomeAssistant web GUI, click "Terminal". A terminal window opens and the prompt you get is:

       ~ $
      

      Although the Unix convention is that a $ prompt means "running unprivileged" while # means "running as root", the HA Terminal window is in fact running as root and the working directory is root's home directory.

    In what follows below, wherever you see a line starting with "#", it means "copy/paste everything except the # and press return". Any line that does not start with "#" indicates the expected response from the preceding command.

  3. You should be able to see the existing (broken) image:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG         IMAGE ID       CREATED        SIZE
    ba22da74/amd64-addon-gosungrow                    3.0.7       2f8714749ba2   3 months ago   161MB

    Your output may be slightly different but you should at least be able to identify a line with "gosungrow" and version 3.0.7.

    Note:

    • If you can't find at least one GoSungrow image then it means something else is wrong with your system. You won't be able to complete these instructions. I recommend deleting and reinstalling the GoSungrow add-on. That will install the broken version but, once that is in place, you should be able to complete these instructions. You may also find it helpful to run the following command:

       # docker system prune -f
  4. Construct the required variables:

    # old_image=$(docker images | grep gosungrow | awk '{print $1 ":" $2}')
    # echo $old_image
    ba22da74/amd64-addon-gosungrow:3.0.7
    
    # new_image=$(echo $old_image | awk -F/ '{print"triamazikamno/"$2}')
    # echo $new_image
    triamazikamno/amd64-addon-gosungrow:3.0.7

    You should get sensible responses to the echo commands. Your exact output may differ but you should be able to see that the earlier output from docker images has turned up in the first echo command, and that the first part of the image name (ie ba22da74) has been replaced with triamazikamno in the second echo command.

    Note:

    • If you don't get sensible responses to the echo commands then it means something is wrong. Go back and re-check your work.
  5. Retag the old image. This prevents it from being removed:

    # docker tag $old_image ${old_image}-backup

    Confirm that that has worked by re-running the images command:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG            IMAGE ID       CREATED        SIZE
    ba22da74/amd64-addon-gosungrow                    3.0.7          2f8714749ba2   3 months ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7-backup   2f8714749ba2   3 months ago   161MB

    You can see both repository+tag combinations point to the same ImageID.

    Note:

    • An ImageID is a hash of the image file so two repository+tag combinations pointing to the same ImageID means they are both pointing to the same image file on disk.
  6. Pull the replacement image from DockerHub:

    # docker pull $new_image
    3.0.7: Pulling from triamazikamno/amd64-addon-gosungrow
    659d66d51139: Already exists 
    7c0ba91aad39: Pull complete 
    fb2a01b55562: Pull complete 
    4425acca1925: Pull complete 
    d50c5eb93aa0: Pull complete 
    adde5526d152: Pull complete 
    Digest: sha256:216c20966785878ccae85b48e45f31fc5e38295f04589d0b2377a7c8b564c867
    Status: Downloaded newer image for triamazikamno/amd64-addon-gosungrow:3.0.7
    docker.io/triamazikamno/amd64-addon-gosungrow:3.0.7

    Again, the actual details may vary but images will confirm the result:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG            IMAGE ID       CREATED        SIZE
    triamazikamno/amd64-addon-gosungrow               3.0.7          f2cbc9418287   11 hours ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7          2f8714749ba2   3 months ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7-backup   2f8714749ba2   3 months ago   161MB

    Three repository+tag combinations but now we have two distinct images.

  7. Now we change the middle tag to point to the new image:

    # docker tag $new_image $old_image

    And confirm that again with images:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG            IMAGE ID       CREATED        SIZE
    ba22da74/amd64-addon-gosungrow                    3.0.7          f2cbc9418287   11 hours ago   161MB
    triamazikamno/amd64-addon-gosungrow               3.0.7          f2cbc9418287   11 hours ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7-backup   2f8714749ba2   3 months ago   161MB

    Note:

    • This step of using the old repository+tag combination to point to the new ImageID is what causes Home Assistant to load the new patched image rather than the old broken image.
  8. Go back to the Home Assistant GUI:

  9. If HA seems to freeze, try quitting your browser and reconnecting to the GUI. Worst case you may need to restart HA, and then start GoSungrow again.

Part 3 — Using a patched binary

triamazikamno has provided patched versions of GoSungrow on GitHub. If you want to obtain a patched version of GoSungrow for use outside of Home Assistant but you do not want to recompile GoSungrow yourself, proceed as follows:

  1. Use your browser to open the following URL:

  2. Scroll down to "Assets" and expand the disclosure triangle if necessary.

  3. Identify the asset which is appropriate to your platform. Examples:

    • macOS on Intel = GoSungrow-darwin_amd64.tar.gz
    • macOS on Apple silicon = GoSungrow-darwin_arm64.tar.gz
    • Linux on Intel = GoSungrow-linux_amd64.tar.gz
    • Linux on 64-bit Raspberry Pi = GoSungrow-linux_arm64.tar.gz
  4. Download the asset to your system. You can either do that from within your browser or you can copy the asset URL to your clipboard, then paste it into a wget command. Something like this would get the job done:

    $ mkdir ~/GoSungrow-patched
    $ cd ~/GoSungrow-patched
    $ wget https://github.com/triamazikamno/GoSungrow/releases/download/v3.0.7/GoSungrow-linux_arm64.tar.gz

    Just make sure you use the correct asset URL in the wget command.

  5. Unpack the .tar.gz (a tape archive – sometimes called a "tarball" – with gzip compression):

    • On macOS (assuming you asked your browser to do the download), the asset will be in your Downloads folder. Double-clicking the downloaded asset creates a folder of the same base name as the asset and extracts the asset's contents into that folder.

    • On Linux (assuming you have just done the wget as above and your working directory is still ~/GoSungrow-patched), you can extract the asset's contents into the working directory like this:

       $ tar -xzf *.tar.gz

    Either way, you wind up with a folder containing the GoSungrow binary plus some other odds and ends which you can ignore.

  6. Complete the following steps from Part 1:

Part 4 — Using a patched Docker image

triamazikamno has provided patched images for GoSungrow on DockerHub. Although the images were intended for use with Home Assistant, they are just Docker images and can also be used outside of Home Assistant.

Initial setup

  1. Create a directory to hold the GoSungrow configuration and its runtime artifacts. These instructions assume the following but the actual directory can have any name and be located be anywhere:

    $ mkdir -p $HOME/GoSungrow
  2. Use the following JSON as a template:

    {
      "sungrow_host": "«Host URL»",
      "sungrow_user": "«USERNAME»",
      "sungrow_password": "«PASSWORD»",
      "sungrow_appkey": "«APPKEY»",
      "mqtt_host": "localhost",
      "mqtt_port": "1883",
      "mqtt_user": "",
      "mqtt_password": "",
      "debug": false,
      "sungrow_timeout": 60
    }
  3. Edit the JSON:

    • "sungrow_host": see iSolarCloud gateway configuration and replace «Host URL» with the appropriate Host URL.
    • "sungrow_user" and "sungrow_password": replace both «USERNAME» and «PASSWORD» with your iSolarCloud credentials.
    • "sungrow_appkey": see APPKey configuration and replace «APPKEY» with the appropriate key.
    • "mqtt_host": the default value of "localhost" assumes the broker is running on the same host as the GoSungrow container but this field can contain a hostname, domain name or IP address.
    • "mqtt_port": the default port for MQTT is 1883 so only change this if you know your broker is running on a non-standard port.
    • "mqtt_user" and "mqtt_password": default to null strings, which assumes your broker does not check credentials. Set appropriate values if your broker authenticates messages.
    • "debug" and "sungrow_timeout": should be left as is.
  4. Save the edited JSON to the path:

    $HOME/GoSungrow/options.json
    

Running the container

  1. From the following list, choose the image which is appropriate for your host's architecture:

    Image Typical Platform
    triamazikamno/armhf-addon-gosungrow:3.0.7 Raspberry Pi 3
    triamazikamno/armv7-addon-gosungrow:3.0.7 Raspberry Pi 4 in 32-bit user mode (kernel mode is irrelevant)
    triamazikamno/aarch64-addon-gosungrow:3.0.7 Raspberry Pi 4/5 in full 64-bit mode
    triamazikamno/amd64-addon-gosungrow:3.0.7 Debian guest on Proxmox-VE running on Intel
  2. In the following command, replace «IMAGE» with the image you chose in the previous step:

    $ docker run -d --name gosungrow --net=host \
       -v "/etc/ssl/certs:/etc/ssl/certs:ro" \
       -v "$HOME/GoSungrow:/data" \
       «IMAGE»
  3. Execute the command. For example, on a Raspberry Pi 4 running full 64-bit Bullseye or Bookworm, the command would be:

    $ docker run -d --name gosungrow --net=host \
       -v "/etc/ssl/certs:/etc/ssl/certs:ro" \
       -v "$HOME/GoSungrow:/data" \
       triamazikamno/aarch64-addon-gosungrow:3.0.7

Notes:

  • --net=host is only needed to facilitate the use of "mqtt_host": "localhost" in the options.json file. If you run the GoSungrow container as part of a stack in which Mosquitto is also running, you can employ non-host mode and change "localhost" to "mosquitto".

  • The first -v maps the host's SSL certificates into the container. The container will inherit any changes made by routine updates to your operating system.

  • When you execute the docker run command on a non-Intel platform, you may see the following message:

     WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
    

    You can ignore that message. See GoSungrow Issue 32 for more information.

Configuration

iSolarCloud gateway configuration

Sungrow appears to operate the iSolarCloud servers shown in the table below. There may be more. If you discover another server then please add a comment to this gist and I will update the list.

Clicking a link in the "Web URL" column will take you to a web portal where you can login. The "Host URL" column is the URL you need to use for GoSungrow.

URLs marked with "†" are guesses based on the .eu pattern. Each domain name resolves in the Domain Name System but I have no way of testing whether any gateway other than the Australian host works in practice. If you use one of these servers and are able to confirm the Host URL is correct, please let me know in the comments below.

"Server" Web URL Host URL
Chinese www.isolarcloud.com.cn https://gateway.isolarcloud.com.cn 
European www.isolarcloud.eu https://gateway.isolarcloud.eu
International www.isolarcloud.com.hk https://gateway.isolarcloud.com.hk 
Australian au.isolarcloud.com https://augateway.isolarcloud.com

To apply a "Host URL":

  • If you are using the GoSungrow add-on in Home Assistant:

    1. Open the Home Assistant GUI.
    2. Click "Settings".
    3. Click "Add-ons".
    4. Click "GoSungrow".
    5. Click the "Configuration" tab.
    6. Enter the Host URL from the table into the sungrow_host field.
    7. Click "Save" then follow your nose.
  • If you are using the GoSungrow binary, run:

     $ ./GoSungrow config write --host "«Host URL»"
    

    Example:

     $ ./GoSungrow config write --host "https://augateway.isolarcloud.com"
    
  • If you are editing a JSON configuration file, replace the «Host URL» field with the value from the table:

     {
       "sungrow_host": "«Host URL»",
     }

    Example:

     {
       "sungrow_host": "https://augateway.isolarcloud.com",
     }

APPKey configuration

The situation with APPKeys is confusing. Some people report success with one key, other people another key. Below is a list of known keys. Try them in order until you find one that works:

  • B0455FBE7AA0328DB57B59AA729F05D8
  • ANDROIDE13EC118BD7892FE7AB5A3F20

If you discover new APPKeys, please let me know and I'll add them to this list.

To apply an "APPKey":

  • If you are using the GoSungrow add-on in Home Assistant:

    1. Open the Home Assistant GUI.
    2. Click "Settings".
    3. Click "Add-ons".
    4. Click "GoSungrow".
    5. Click the "Configuration" tab.
    6. Enter the APPKey from the table into the sungrow_appkey field.
    7. Click "Save" then follow your nose.
  • If you are using the GoSungrow binary, run:

     $ ./GoSungrow config write --appkey «APPKey»
    

    Example:

     $ ./GoSungrow config write --appkey B0455FBE7AA0328DB57B59AA729F05D8
    
  • If you are editing a JSON configuration file, replace the «APPKEY» field with the value from the table:

     {
       "sungrow_appkey": "«APPKEY»",
     }

    Example:

     {
       "sungrow_appkey": "B0455FBE7AA0328DB57B59AA729F05D8",
     }

Change History

  • 2024-05-02

    • Evidence now suggests there is no correlation between iSolarCloud servers and AppKeys. Text adjusted.
  • 2024-05-01

    • Consolidate all information about iSolarCloud gateways and APPKeys into a new section and cross-reference existing sections to the consolidated material.
  • 2024-03-15

    • Adds Part 4 to leverage existing updated DockerHub images for anyone who wants to run GoSungrow in a Docker container outside of Home Assistant.
  • 2024-03-13

    • Adds Part 3 to leverage existing recompiled binaries (which are inputs to the updated DockerHub images) for anyone who simply wants to obtain a patched version of GoSungrow without having to recompile it themselves.
  • 2024-02-28

    • Includes reminder to click SAVE when changing settings.
  • 2024-01-30

    • Adds reminder to set cloud gateway for non-AU systems.
  • 2024-01-22

    • Move go mod tidy before go build.
  • 2024-01-14

    • Emphasise need to use ./ prefix when testing the recompiled binary.
    • Add short section on using GoSungrow from ~/.local/bin.
  • 2023-12-24 - restructure to make clear the distinction between recompiling and using replacement images in Home Assistant.

  • 2023-12-06 - revise "hack" to clarify that the hack can be applied without opening a connection via SSH.

  • 2023-12-09 - revise "hack" to use images provided by triamazikamno on DockerHub.

  • 2023-12-07 - add steps to recompile from triamazikamno fork to resolve er_invalid_appkey problem:

  • 2023-11-29 - re-tested using go 1.21.4 on:

    • M2 MacBook Pro running macOS Ventura 13.6.2
    • Intel Mac Mini running Debian Bookworm guest on Proxmox VE
    • Raspberry Pi 4B running Raspberry Pi OS Bullseye
    • Raspberry Pi 4B running Raspberry Pi OS Bookworm
  • 2023-09-18 - satisfy GoUnify dependency.

@shamal69
Copy link

@Paraphraser Thank you so much for this from a fellow Aussie! :)

@Paraphraser
Copy link
Author

@danielsaraujo Hi

Unfortunately, I only have a single inverter so I don't have any experience with what looks "normal" for multiple inverters nor how to diagnose problems.

But in reading issues on the GoSungrow repo, I have gained the impression (possibly unreasonably) that the intersection set between "problems" and "multiple inverters" is non-empty.

Also, I don't actually use the GoSungrow add-on for Home Assistant in any practical sense so my experience debugging problems with the add-on is non-existent.

Now, having said that, if I had a problem such as you describe, I don't think I'd be trying to diagnose it from within Home Assistant.

What I would be doing is compiling GoSungrow for my regular desktop computer (macOS in my case) and running GoSungrow from the command line. At least on Unix, everything relevant turns up in ~/.GoSungrow/config.json and you can display it with:

$ GoSungrow config read

Assuming you're using the recompiled binaries and you've updated the API key (as per the first part of this gist) and you've set your credentials at some stage with:

$ GoSungrow config write --user=USERNAME --password=PASSWORD

and also perhaps set your local cloud server with something like:

$ GoSungrow config write --host=https://gateway.isolarcloud.com.hk/

then:

$ GoSungow api login

will tell you whether you're able to connect, and then you can start exploring with:

$ GoSungrow show ps list
$ GoSungrow show ps detail

along with the other command-line examples in the main MickMake readme.

If you get sensible results at the command line then you know the problem is something about the HA environment and, in the classic "are you sure it's plugged in?" sense, I'd probably blow the add-on away, drop into an "advanced ssh" session and run:

# docker system prune -f

to try to clear-up any dangling bits and pieces, then re-install the add-on, followed by repeating the steps in part 2 of the gist.

After that ... dunno, really. Sorry.

@Hazkarax
Copy link

Hazkarax commented Feb 4, 2024

Hi and thanks for this fix @Paraphraser. I get stuck on part 4 trying to confirm success. I get ERROR: unknown error '' when I try to do the api login. I am not running it in a container. Perhaps something went wrong in the first steps as I only updated Go using brew before creating the directory in my home folder.

Any ideas? Kind of stuck at the moment

@Paraphraser
Copy link
Author

At macOS I've said to install the Go compiler using the installer package rather than HomeBrew, with a tip to remove the HomeBrew package if it's there already.

Although I can't remember the details, I originally used HomeBrew to install Go but something subsequently persuaded me that the official installer packer was the better choice. That's why these instructions say to do it that way.

I can't say for certain if this will solve the problem but it's a place to start.

@SuperNestendo
Copy link

Thanks for testing / distributing all this information!

Unfortunately, I had the Error: appkey is incorrect 'er_invalid_appkey issue with an install from Add On Store but after applying the fix I now get the Error: unknown error 'Request is not encrypted'. I tried the recompile instructions but my Linux / Home Assistant knowledge was too low. Anything else you could recommend? Happy to provide any more info.

@Paraphraser
Copy link
Author

@SuperNestendo unfortunately, the only way to fix Home Assistant is to follow the instructions in Part 2 of the Gist.

The "er_invalid_appkey" message is fixed by setting the correct key in the add-on configuration (step 8 of part 2). If you only change the key but don't follow all the earlier steps in Part 2 then that's what causes "Request is not encrypted".

To be clear, you do not need to recompile GoSungrow in order to get the Home Assistant add-on going again. The work of recompiling and preparing a new image has been done for you. The steps in Part 2 explain how to download that image and swap it into place for Home Assistant to use.

Part 1 of the gist (how to recompile) is only needed if you want to use GoSungrow outside of the Home Assistant environment.

@millesm
Copy link

millesm commented Feb 12, 2024

worked perfectly 1st go on RasberryPi HA install.

Thanks heaps

@SuperNestendo
Copy link

@SuperNestendo unfortunately, the only way to fix Home Assistant is to follow the instructions in Part 2 of the Gist.

The "er_invalid_appkey" message is fixed by setting the correct key in the add-on configuration (step 8 of part 2). If you only change the key but don't follow all the earlier steps in Part 2 then that's what causes "Request is not encrypted".

Thanks for the info. I followed part 2 and got the same results but still ended with the Error: unknown error 'Request is not encrypted'.

image

@Paraphraser
Copy link
Author

Paraphraser commented Feb 16, 2024

IMG_2488

Please study the pattern in mine.

Line 2 is the image I downloaded from DockerHub by following the instructions in the gist. The triamazikamno image is the recompiled version with the fix for the problems discussed in Issue 101. It is the image that I want Home Assistant to use.

The problem is that there's no easy way of telling Home Assistant to use that image directly. It has to be done indirectly.

What Home Assistant is looking for is the image named ba22da74/amd64-addon-gosungrow:3.0.7. The name is the concatenation of the Repository and Tag columns. You'll see that in line 1.

Focus on the Image IDs for lines 1 and 2. Those are the same ID (f2cbc9418287). Docker images aren't defined by their names. They are defined by their IDs. The first two lines in my screen shot are pointing to the same file on disk (like aliases).

Line 3 has a different Image ID. This is the old malfunctioning image. It has been kept around on the off chance it might be needed if we ever have to undo all this in order to get Home Assistant to recognise and download an updated version supplied by MickMake.

Now, compare what you see in my screen shot with yours.

Ignore the fact that your image IDs are different (they differ according to architecture). Home Assistant is still looking for the image named ba22da74/amd64-addon-gosungrow:3.0.7 but it has the same ID as the "-backup". In other words, your lines 1 and 2 are both pointing to the old broken image. The fixed triamazikamno image isn't getting a chance to run.

If you are able to look back over the history of the commands you executed, I think it's reasonably likely you'll find that you made a mistake at some point when following the instructions in the gist.

I said "reasonably likely" because I can't safely assert that you must have made a mistake. That's because I'm already aware of another problem with the aarch64 images (they have incorrect architecture values internally). It's always possible the gist instructions are wrong, on aarch64 platforms, because of that other problem. Only you will be able to confirm that and, then, only if you can go back over your history and compare it with the gist. Of course, if you can do that and you do find a bug in the gist as it applies to aarch64 then please tell me and I'll update the gist accordingly.

Anyway, with that possibility in mind, and rather than me say something along the lines of, "just start from scratch and be sure to follow the gist precisely" and risk you coming back and saying "it still doesn't work", I think the simplest thing to do now is to just fix the problem in situ. Like this:

$ docker tag 005ee47d3a6a ba22da74/aarch64-addon-gosungrow:3.0.7

What that will do is move the ba22da74/aarch64-addon-gosungrow:3.0.7 name so that it points to the 005ee47d3a6a image. Your pattern will change so lines 1 and 3 point to the triamazikamno image, leaving the -backup as the old image.

Then, go into HA and start the container (or stop/start) and see what happens.

@Paraphraser
Copy link
Author

@SuperNestendo did my explanation yesterday solve your problem or are you still having trouble?

@SuperNestendo
Copy link

@SuperNestendo did my explanation yesterday solve your problem or are you still having trouble?

YES!!!! Thank you! That solution makes complete sense, not sure how I missed it in the setup. Working now! So exciting!

@shamal69
Copy link

@Paraphraser - thank you so much for sorting this out for everyone. I am just wondering if there is any parameter to change the polling interval? At the moment, mine (I assume default) is set to 5 mins. I would ideally like to have 2 min resolution on data - ie 30 vs 12 data points per hour.
Again, thanks for the effort!

@roshanlasantha
Copy link

Hi @Paraphraser,

I'm new to the HA. I followed the instructions and managed to get it connected to the API but for some reason when querying the API. Any pointers to solve this issue?

Following is my log file.
Token File: /data/.GoSungrow/AppService_login.json
[07:07:35] INFO: Syncing data from gateway https://augateway.isolarcloud.com ...
2024/03/07 07:07:35 INFO: Connecting to MQTT HASSIO Service...
2024/03/07 07:07:35 INFO: Connecting to SunGrow...
2024/03/07 07:07:35 INFO: Found SunGrow 2 devices
2024/03/07 07:07:35 INFO: Caching Sungrow metadata...
2024/03/07 07:07:35 INFO: Cached 392 Sungrow data points...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0xc8 pc=0x451414]

goroutine 1 [running]:
github.com/MickMake/GoSungrow/iSolarCloud/AppService/queryDeviceList.(*EndPoint).SetPvPoints(, {{, , }}, {0x4000509080, {0x400087b887, 0x7}, {0xc17277cdecb6140d, 0x20713140, 0x443f8e0}, ...})
/Users/isit/git/GoSungrow/iSolarCloud/AppService/queryDeviceList/data.go:419 +0x624
github.com/MickMake/GoSungrow/iSolarCloud/AppService/queryDeviceList.(*EndPoint).GetEnergyStorageSystem(
, {0x4000509080, {0x400087b887, 0x7}, {0xc17277cdecb6140d, 0x20713140, 0x443f8e0}, {{0x4000b7eb00, 0x2, 0x2}}, ...})
/Users/isit/git/GoSungrow/iSolarCloud/AppService/queryDeviceList/data.go:299 +0x2f8
github.com/MickMake/GoSungrow/iSolarCloud/AppService/queryDeviceList.(*EndPoint).GetData(
)
/Users/isit/git/GoSungrow/iSolarCloud/AppService/queryDeviceList/data.go:259 +0x94
github.com/MickMake/GoSungrow/iSolarCloud/AppService/queryDeviceList.EndPoint.GetEndPointData(...)
/Users/isit/git/GoSungrow/iSolarCloud/AppService/queryDeviceList/struct.go:367
github.com/MickMake/GoSungrow/iSolarCloud.(*SunGrowData).CallEndpoint(, {, _}, {{0x40006003f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}, ...})
/Users/isit/git/GoSungrow/iSolarCloud/data.go:160 +0x30c
github.com/MickMake/GoSungrow/iSolarCloud.(*SunGrowData).getDataSinglePsIdRequired(0x40006e3450, {0x27cd1f8, 0x4000431880})
/Users/isit/git/GoSungrow/iSolarCloud/data.go:283 +0x294
github.com/MickMake/GoSungrow/iSolarCloud.(*SunGrowData).GetDataSingle(0x40006e3450, {0x400014b040, 0xf})
/Users/isit/git/GoSungrow/iSolarCloud/data.go:238 +0x124
github.com/MickMake/GoSungrow/iSolarCloud.(*SunGrowData).GetData(0x40006e3450)
/Users/isit/git/GoSungrow/iSolarCloud/data.go:209 +0x154
github.com/MickMake/GoSungrow/cmd.(*CmdMqtt).Cron(0x40000aa8f0)
/Users/isit/git/GoSungrow/cmd/cmd_mqtt.go:375 +0x1fc
github.com/MickMake/GoSungrow/cmd.(*CmdMqtt).CmdMqttRun(0x40000aa8f0, 0x0?, {0x0?, 0x0?, 0x0?})
/Users/isit/git/GoSungrow/cmd/cmd_mqtt.go:273 +0x70
github.com/spf13/cobra.(*Command).execute(0x4000093b00, {0x400003a1a0, 0x0, 0x0})
/Users/isit/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x5c4
github.com/spf13/cobra.(*Command).ExecuteC(0x40004f4000)
/Users/isit/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x340
github.com/spf13/cobra.(*Command).Execute(...)
/Users/isit/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/MickMake/GoUnify/Unify.(*Commands).Execute(...)
/Users/isit/go/pkg/mod/github.com/!mick!make/!go!unify@v1.0.3-0.20230904042338-0db745f1bada/Unify/struct.go:277
github.com/MickMake/GoUnify/Unify.(*Unify).Execute(0x4000533400)
/Users/isit/go/pkg/mod/github.com/!mick!make/!go!unify@v1.0.3-0.20230904042338-0db745f1bada/Unify/struct.go:216 +0x318
github.com/MickMake/GoSungrow/cmd.Execute(...)
/Users/isit/git/GoSungrow/cmd/commands.go:94
main.main()
/Users/isit/git/GoSungrow/main.go:11 +0x68
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

@Paraphraser
Copy link
Author

@roshanlasantha My first thought was to wonder if this was a side-effect of the HA 2024.03 update that has just been released. I upgraded my own HA but my GoSungrow add-on continued to work normally and didn't produce the same error as you are seeing.

I have never written more than a few lines of Go and I've never even looked at the GoSungrow source code but my 50-odd years' experience examining crash logs from other programming languages leads me to conclude that what you are seeing is likely the result of some unexpected data being supplied by iSolarCloud to the GoSungrow add-on. As to what the underlying cause of that might be, I can't speculate.

You might be able to get a little further if you follow the first part of this gist and compile the binary separately, then follow the instructions in the main README on the GoSungrow repo to define your credentials, set the updated API key, run GoSungrow api login and then experiment with the other commands in the main README.

If you can get replicate the problem outside of the HA environment then I'd suggest opening a new issue on the GoSungrow repo with as much detail as you can provide. Open issues are a good way of figuring out if it's just you or if other people are having similar problems. When the problem with the change of API key and encryption requirement broke GoSungrow, other members of the community with the relevant skills were able to figure out the cause, prepare fixes and generate the revised Docker images for the HA add-on. Who knows - they might be able to help you too.

My contribution was limited to writing this gist which took advantage of their excellent work

To be honest, I keep wondering whether GoSungrow has much of a future. According to MickMake's GitHub profile page, his last activity on GitHub was December 19th 2023, with his last activity related to GoSungrow on September 4th 2023. Best case, he's occupied doing other things and will circle back eventually. Worst case, the project is abandoned. While I hate sounding like some prophet of doom, the more time that passes, the more I suspect the latter.

@xpufx
Copy link

xpufx commented Mar 12, 2024

I am wondering why things are so complicated here. My use case is NO home assistant.

Download the release from https://github.com/triamazikamno/GoSungrow/releases/tag/v3.0.7 which seems to include the patch?. (I can api login)

./GoSungrow config write --appkey=B0455FBE7AA0328DB57B59AA729F05D8
Done.

What am I missing?

@Paraphraser
Copy link
Author

You are not missing anything, other than, perhaps, a bit of history.

This gist started out as just the material in Part 1. It was in answer to the question "how can I recompile GoSungrow myself?"

Part 2 was added after the AppKey problem occurred but before triamazikamno did the work to construct new Docker images and push those to DockerHub.

In its original form, Part 2 explained how to use a Dockerfile, plus the malfunctioning GoSungrow Docker image, plus the recompiled binary from Part 1, to generate a local image which could then be used by Home Assistant.

Then triamazikamno did the work and it made sense to rewrite Part 2 to take advantage of it.

However, the original question remains valid, as does the entirety of Part 1 of the gist.

It just never occurred to me to write a Part 3 to explain how to use triamazikamno assets. That's probably because I assumed that anyone who wasn't interested in Home Assistand and simply wanted a local binary (which includes me) would rather compile the thing themselves.

@Paraphraser
Copy link
Author

@xpufx following on from my earlier reply, I have taken your question as an excellent suggestion and added a Part 3 to the gist.

To be honest, I keep hoping MickMake will resurface, provide a version 3.0.8, and I can then explain how to undo any "damage" done to HA installations which are using the "hack" in Part 2, after which we can all go back to having a quiet life.

@xpufx
Copy link

xpufx commented Mar 13, 2024

I keep hoping MickMake will resurface, provide a version 3.0.8,

Same here. Thanks for providing me context and keeping the gist updated.

@Lmecano
Copy link

Lmecano commented Mar 13, 2024

It's a wonderful step by step, thank you !
I still have this issue, but may be the Sungrow API is dead.... ?
[15:03:57] INFO: Login to iSolarCloud using gateway https://gateway.isolarcloud.eu ... Error: Sungrow API EndPoint not yet implemented Sungrow API EndPoint not yet implemented

@Paraphraser
Copy link
Author

@Lmecano If this gist has given you the impression that I have any deep knowledge about anything to do with Sungrow infrastructure then I'm sorry but that's not the case. Such "knowledge" as I have comes from starting with the README on the GoSungrow repo, exploration, frustration and guesswork. Even then I only ever went as far as I needed to go to get the data I wanted.

I don't use the Home Assistant version. I use the compiled binary in a script which runs once a day to collect "yesterday's" data. My goal was to replicate a time-series that began when I got my first solar system, which had a SolaX inverter. My script fetches a grand total of 12 metrics from iSolarCloud. That's all I need to do all the calculations I want. I really only look at it once a day to see how yesterday went, and then once a week to do an expected bill calculation. Sure, early on I'd use the iPad app to watch the thing in real time but that got real old, real fast. In reality my main objective is just to avoid ever being in the situation where the inverter stopped working, I didn't realise, and then I got a surprise bill. Nothing more than that. An eyeball once a day is sufficient.

If my script ever broke with a message like you are seeing then I would have no idea what to do next.

But.

My understanding is that, in this context, "API" (Application Programming Interface) refers to the communications protocol and the syntax of the request/response messages transported between client (your device) and server (iSolarCloud), while "endpoint" refers to a specific data element which usually looks something like this:

1234567_1_1_1.p1

I believe (I do not know) that that decomposes as follows:

  • 1234567 is the "psid" (maybe "personal site identifier" or some such);

  • 1_1_1 seems to be a "device ID". It means my Sungrow SG5.0RS inverter, as distinct from, say, 7_1_1 which seems to mean the inline meter; and

  • .p1 appears to be the data store for Daily Yield.

Thus:

$ GoSungrow-mac show point data 202403130000 202403132359 5 1234567_1_1_1.p1 >results.txt

In words, "fetch Daily Yield at 5-minute intervals for yesterday's date". There are 288 5-minute periods in 24 hours so I expect the output file to contain at least that many lines:

$ wc -l results.txt 
     294 results.txt

The other 294-288=6 lines are header/trailer. The observation for the 5-minute period beginning midday yesterday:

$ grep "12:00:00" results.txt 
┃ 2024-03-13 12:00:00    ┃ 1234567_1_1_1    ┃ 9800                ┃

or 9.8kWh to that point.

So, to the extent that my earlier definition of the "Sungrow API" holds, I can communicate with the (Australian) server and retrieve expected end-points, so I don't think anything is "broken", at least not in any systematic sense.

But that still doesn't explain why you are getting that message.

I wondered if the explanation might be as simple as using the wrong server. However, when I try switching to the EU server, I get different error messages:

$ GoSungrow-mac api login
Error: cannot login

$ GoSungrow-mac show point data 202403130000 202403132359 5 1234567_1_1_1.p1
Error: Account does not exist '账号不存在'

Logically, if you aren't seeing those messages, the EU server must be correct for you. What happens when you run:

$ GoSungrow show ps list

I get my expected three devices:

┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Ps Key           ┃ Ps Id   ┃ Device Type ┃ Device Code ┃ Channel Id ┃ Serial #    ┃ Factory Name ┃ Device Model   ┃
┣━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┫
┃ 1234567_1_1_1    │ 1234567 │ 1           │ 1           │ 1          │ B2222334445 │ SUNGROW      │ SG5.0RS        ┃
┃ 1234567_22_247_1 │ 1234567 │ 22          │ 247         │ 1          │ B2222334445 │ SUNGROW      │ WiNet-S        ┃
┃ 1234567_7_1_1    │ 1234567 │ 7           │ 1           │ 1          │ B2222334445 │ SUNGROW      │ SG Smart Meter ┃
┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛

If you can get that far then my guess is that the error message means what I would interpret it as saying were I to receive it myself, which is that, given a request for:

psid_deviceid.pnnnn

the pnnnn data store (aka bucket aka metric) doesn't exist for your inverter.

As to why? Beats me. Sorry.

@Lmecano
Copy link

Lmecano commented Mar 14, 2024

Thanks for the feedback, you are still more knowledgeable than I do :)
I have succeeded to run a bash of the image with
docker run -it <image-id> /bin/bash

And modified manually all the config file and tried to manual login with
GoSungrow api login
But I just got an "Error : empty file"

I'm wondering if I should not compile everything as you did in Part 1, because when I run the docker image, I got the following warning :
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

@Paraphraser
Copy link
Author

@Lmecano I have extended the gist to include a new Part 4 which explains how to take one of the Docker images built as Home Assistant add-ons and repurpose it as a local container using docker run.

The new material also explains why the "WARNING" message can be ignored.

Something I suspect you might not be fully aware of is what happens when you run a container like this:

$ docker run -it <image-id> /bin/bash

That tells Docker to:

  1. Pull the image (if it is not already downloaded);
  2. Instantiate the image as a running container; and
  3. Override the container's normal startup sequence by telling it to execute /bin/bash.

All you really get from doing that is the base operating system (Alpine Linux).

In the case of the GoSungrow container, the normal startup sequence runs a script called run.sh (it's inside the container). That script takes the options.json file and uses it to create a config.json (the thing you see when you run GoSungrow config read), and then launches GoSungrow with the arguments mqtt run.

Overriding that means options.json is never read and config.json is never created. That's why GoSungrow api login moans about an empty file.

If you want to open a shell into the GoSungrow container, the correct way is to get it going as is explained in Part 4 then, once it is running, you can interact with it via:

$ docker exec -it gosungrow bash

That opens a shell into the initialised container. Then you can run GoSungrow commands alongside the mqtt run command and it will all just work.

There's really no way around this because the setup work done by run.sh is the foundation for everything else. Plus, the only reason docker exec is able to open a shell is because the mqtt run is in an infinite loop keeping the container running.

If you simply want to experiment with GoSungrow where you have full control over what it does and when it does it, you have to install the binary locally. That's what Part 3 is for.

I hope I've covered-off everything you raised in your post.

@Lmecano
Copy link

Lmecano commented Mar 15, 2024

Hello, thanks again !
I have followed some docker tutorial to be more familiar with the vocabulary, and with your explanations, I'm starting to got it !
I have prepared the container to run as you describe, but when time has come, I got the following permission error :
d50c6b123e2ab99216b5d785095cbff30d8c2a16ed06394c472893b12403d870 docker: Error response from daemon: error while creating mount source path '/root/GoSungrow': mkdir /root/GoSungrow: read-only file system.
The container doesn't appear in the list of running container with docker ps

Even a chmod -R 777 $HOME/GoSungrow doesn't help.
From what I have found on internet, could be an issue with the docker version (I'm using an RPI 3, may be that's why....)

I have installed Docker Desktop on my window, I will try all of that on another HomeAutomation build
EDIT : I have installed Debian on a VM and compiled the soft. The connection to API is fine and it looks like I can see my Sungrow system \o/
The question would be now.... why the homeassistant version doesn't catch the API endpoint

@Paraphraser
Copy link
Author

@Lmecano

Error response from daemon: error while creating mount source path '/root/GoSungrow': mkdir /root/GoSungrow: read-only file system

That error makes absolutely no sense to me.

Let's go back a few hundred steps.

If you start with a blank SD card, use Raspberry Pi Imager to transfer Raspberry Pi OS onto that media, insert that media into your Pi (3, 4, 5, Zero2W, whatever) and apply power, you get a privileged initial user account and the root account is disabled.

Historically, that initial user account is called "pi" but you can now set it to be any name (although I still recommend going with "pi" and it's what I'm going to assume in this discussion).

What I mean by "privileged" is that "pi" has the ability to run sudo. On Raspberry Pi OS, "pi" also has the ability to run sudo without being prompted to enter a password.

Other OS install procedures have different approaches. For example, if you install Debian either natively or as a Proxmox guest, the installer prompts you to enter a root password. If you supply a password, the root account is enabled and the initial user account isn't privileged. However, if you leave the root password blank then the initial account is privileged (but a password is needed to run sudo).

The take home message from all of this is that there are so many variations on the theme that, without knowing exactly how you set up your Pi, I can really only discuss this in general terms.

In Part 4 of this gist, the instruction is that you should create the GoSungrow folder before executing the docker run command.

Assuming you did that, if you are logged-in as root then running:

# docker run … -v "$HOME/GoSungrow:/data"

should see $HOME expand to /root and it should not be necessary for docker to attempt to create /root/GoSungrow because it should already exist.

However, if you did not create /root/GoSungrow before running the docker command then it is still running as root and should have no trouble creating /root/GoSungrow itself.

Conversely, if you are logged in as "pi" then running:

$ docker run … -v "$HOME/GoSungrow:/data"

should see $HOME expand to /home/pi and it should not be necessary for docker to attempt to create /home/pi/GoSungrow because it should already exist.

However, if you did not create /home/pi/GoSungrow before running the docker command then it is still running as "pi" and should have no trouble creating /home/pi/GoSungrow itself, and should not be trying to create /root/GoSungrow in the first place.

Even if you are logged in as "pi" but run:

$ sudo docker run … -v "$HOME/GoSungrow:/data"

the expansion of $HOME occurs before the sudo so the command that is actually passed to sudo is:

docker run … -v "/home/pi/GoSungrow:/data"

and, again, /home/pi/GoSungrow should exist already so there's no reason for docker to try to create it.

The only thing that seems to make sense is this kind of pattern:

$ mkdir -p $HOME/GoSungrow
$ sudo -s
# docker run … -v "$HOME/GoSungrow:/data" 

In that situation, $HOME expands to /home/pi for the mkdir command, but expands to /root within the sudo shell context.

If that (or something like it) is what you have been doing then I further surmise that you feel the need to use sudo to run docker commands, likely because they don't seem to work without sudo.

Am I anywhere close to guessing correctly about this?

If I am then the basic problem might be that you have missed a step somewhere along the way when installing Docker, which is to give yourself membership of the "docker" group. Do this test (while logged-in as "pi" and not while inside a sudo shell):

$ groups

Do you see "docker" in the list? If not, that's a problem. You can fix it by doing:

$ sudo /usr/sbin/usermod -G docker -a $USER

and then you must logout and login again. After that, "docker" should be in the groups list and you shouldn't have to use sudo to run docker commands.

Perhaps also read what-is-sudo for more hints on when sudo is or isn't appropriate.

The real problem with over-use of sudo is that it can easily do an awful lot of damage to your system which is hard to discover and fix. The damage lurks under the hood causing permission errors, which lead you to using sudo and causing further damage. It's a vicious cycle.

From what I have found on internet, could be an issue with the docker version (I'm using an RPI 3, may be that's why....)

This is a definitely possibility. I keep running across tutorials and YouTube videos which boil down to using apt to install docker and docker-compose. That's a recipe for a mess. What usually happens is you get an obsolete docker-compose with dependencies which force the downgrade of docker, and then both become pinned and can never be upgraded.

Look, if you're up for it, perhaps take a look at PiBuilder.

What PiBuilder is designed to do is construct a rock solid server-class platform for IOTstack.

IOTstack is not a "system". It's a set of conventions for allowing arbitrary collections of Docker containers to be run together as a "stack". The classical form of that stack is MING (Mosquitto, InfluxDB, Node-RED and Grafana). Although GoSungrow is not included as one of IOTstack's templates, it's easy enough to add if that is what you wanted to do (I've done it). Plus, the problem of setting up Mosquitto is solved because that is part of IOTstack.

The material point, though, is that IOTstack needs Docker so PiBuilder installs Docker properly, which includes using the official "convenience script" and setting up the group membership as above. So, even if IOTstack doesn't sound like something you want or need, using PiBuilder ensures the Docker side of things will have been done right.

Also, if you would prefer to do the same thing on a VM, you might get some useful hints from running-on-proxmox-debian.

The container doesn't appear in the list of running container with docker ps

No. A container has to at least start before it appears in the process list. Not being able to create the folder prevents the container from starting.

The question would be now.... why the homeassistant version doesn't catch the API endpoint

I don't understand your question. Do you mean:

  1. The HA version does report "EndPoint not yet implemented" but the VM version doesn't get that error; or
  2. The HA version doesn't report "EndPoint not yet implemented" but the VM version does get that error?

Either way, my answer is the same. I don't know. I don't have any deep understanding about how Sungrow inverters log metrics, nor how the iSolarCloud API works, nor how GoSungrow works. All I have to go on is my own standalone use of GoSungrow and I don't see any errors like this in what I'm using it for.

@Lmecano
Copy link

Lmecano commented Mar 20, 2024

It makes no sense to me either :D
For some context, I have installed HomeAssistant through RPImager on a RPI3 model B.
Some information :

  • Before running the "run" command, the folder $HOME/GoSungrow was created.
  • The whoiam command identify me as root.
  • I have never needed to use the sudo command in HA
  • Even sudo docker run .... leads to the same error message
  • Even creating the folder in /home/pi/GoSungrow leads to the same error message
  • I did not installed docker, it's included in HA

In parallel, on my personal computer, I have installed a VirtualMachine of Debian to compile the code (Part1 of your tutorial)
=> Doing this way work ! I can see my Sungrow material and some information.

In summary : I can't make this addon work with HA. But it works installed directly on Debian.
May be I could compile the addon in HA, as it is a Debian-based OS.... but it looses all the advantages to have a simple embedded addon, I think.

I think I will forget about getting the data from the Sungrow API (doesn't look reliable) and just do my own electronic board to measure the information I need.

Thank you for the support <3

@Paraphraser
Copy link
Author

@Lmecano - well that makes things a lot clearer. If you'd given me that context at the start, I wouldn't have guessed (incorrectly; several times) about what you were trying to do.

If you're running Home Assistant, regardless of platform, the only part of this gist you need to concern yourself with is Part 2.

You don't need the $HOME/GoSungrow folder - at all - irrespective of whether $HOME expands to /root or /home/«someUserNameOfYourChoiceHere».

You don't need to be using sudo - for anything.

You don't need to be doing docker run - with or without sudo.

Yes, the whoami command will return root because that's what you get when you use the Advanced SSH and Web Terminal. Even if you SSH (rather than run Terminal from the sidebar), whatever user you define (the default is hassio) still becomes root on login.

And, yes, you don't need to be installing Docker because HA comes with Docker because Docker is how HA works.

For the record, I have been running HA as a Proxmox-VE guest on an old Intel MacMini. That's mainly because I don't actually use HA for anything productive so I don't want to dedicate a Pi to HA.

But, today, I worked through the same install process as you, albeit on a Pi 4B rather than a Pi 3B, because I no longer have a 3B (mine died about a year ago). I used Raspberry Pi Imager to download the HA image and burn it to SD. I booted the Pi 4, installed Advanced SSH and Web Terminal, Mosquitto broker and GoSungrow. I configured all three from scratch. I went through the steps in Part 2. And GoSungrow just worked. The last part of the log:

Is Online:	false
Token:	374058_053ea977157c4d0ba2afa14b170aa5eb
Token File:	/data/.GoSungrow/AppService_login.json
[09:58:32] INFO: Syncing data from gateway https://augateway.isolarcloud.com ...
2024/03/21 09:58:32 INFO: Connecting to MQTT HASSIO Service...
2024/03/21 09:58:32 INFO: Connecting to SunGrow...
2024/03/21 09:58:32 INFO: Found SunGrow 3 devices
2024/03/21 09:58:32 INFO: Caching Sungrow metadata...
2024/03/21 09:58:33 INFO: Cached 963 Sungrow data points...
2024/03/21 09:58:33 INFO: Syncing 148 entries with HASSIO from getPsList.
CUCU?CU?CUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCU?CUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCU?CUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCUCU?CUCUCUCUCUCUCUCUCUCUCUCUCUCUCU?CUCUCUCUCUCUCUCUCUCUCUCU?CUCUCUCUCUCUCUCUCU
2024/03/21 09:58:34 INFO: Syncing 205 entries with HASSIO from getPsDetail.
????CUCUCUCU?CUCUCUCU?CUCUCU?CU?CUCUCUCUCU??CU?CUCUCUCUCUCUCUCUCUCU?CUCUCUCUCU?CU?CUCUCUCU?CU???CU?CUCUCUCU?CUCU?CUCUCUCUCU??CUCUCUCU?CU?CUCUCUCU?CUCU?CU?CU?CU?CUCUCUCUCU?CUCU???CUCUCUCUCUCUCUCU?CUCUCUCUCUCUCUCUCUCUCU?CU?CUCUCUCUCUCUCUCUCU?CUCU?CUCU???CUCU?CUCUCUCUCUCUCU?CU?CUCUCUCU??CU?CUCUCUCUCUCU?CUCUCUCUCU?CUCUCU?CUCU?CUCU?CUCUCUCUCUCU??CUCUCUCUCUCU
2024/03/21 09:58:34 INFO: Syncing 461 entries with HASSIO from queryDeviceList.
CU-CUCUCUCU--CU-CU--CU-CUCU---CUCU-CU--CU-CU-CU-CU-CUCUCUCUCU
2024/03/21 09:58:34 INFO: Starting ticker...
2024/03/21 09:58:34 INFO: Fetch Schedule: 5m
2024/03/21 09:58:34 INFO: Sleep Delay:    40s
2024/03/21 10:04:34 INFO: Syncing 461 entries with HASSIO from queryDeviceList.
-UU----U-UU-U--UU--UUU--UU-UUUU--UU-CUUU
2024/03/21 10:04:35 INFO: Syncing 148 entries with HASSIO from getPsList.
UUUUUUCUUUUU?UUUUUUUUUUUUUUUUUU?UUUUUUUUUCUUUUUUUU?UUUUUUU??UUUUUUUUUUUUUU?UUUUUUUUUUUUUUUUUUUUUUUUUCUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUCUUUUUUUUUUUU?UUCUU
2024/03/21 10:04:35 INFO: Syncing 205 entries with HASSIO from getPsDetail.
U?UU?U?UUU???U?U?UU?UU??UU??UUU??UUUCUUUU?U?UUCUUCUUUUU?U?UUCUUU?UU?UUUU?UUUUU?UUUUUUUUU?U?U?UU?U??UUUU??UUUUUU?UUUUUUU?CUCUU?UUU?U?UUUUU?U?UUU?U??UU?U??UUUU?UUUUUUU?UUUU?UUUUCUUCUUU?U??U?UUUUUUUUUUUUUUUU??UUUUU??
2024/03/21 10:10:24 INFO: Syncing 148 entries with HASSIO from getPsList.
UU?UCUUCUUU?UUUUUUUUUUUUUUUUCUUUUUUUUUUUUCUUUUUUUUUUU?UUUUUUUUUUUUUUUU??UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU?UUUUUUUUUUUUUU?UUUUUUUUUCUUUUUUUUUUUUUUUUUUUUUU
2024/03/21 10:10:24 INFO: Syncing 205 entries with HASSIO from getPsDetail.
UUUU?UUUUUUUU?UUU?U?UU??UUUUUUUUUUU??UUUUUUCU?U??CU?UUUUUUUUUU?UU?U?UUUUUU??U?UUUUCUUU??UUCUUU??CUUCUU?UUU?UU?UCU?U?UUUUUCU??UUUCUU??UU??UUUUUU?UU?CU?UUU?????CU??U?U?UU?UCU?UCU?UUCUUU?UUCUU?UUUUUUUUUUU?UUCUUUUU?UU?UUU?UU?
2024/03/21 10:10:24 INFO: Syncing 461 entries with HASSIO from queryDeviceList.
-UUU--UU---UUU--UUUU---UCUCU--U-UUU-U-UU-

Now, if I want to use the already-running GoSungrow container (the actual running add-on) to experiment with other GoSungrow commands, I do this:

  1. Click Terminal in the side-bar.

  2. Type the command:

    docker ps
    
  3. One of the entries in the response is:

    CONTAINER ID   IMAGE                                  COMMAND                  CREATED          STATUS          PORTS                                                                                                                          NAMES
    04e0c040d4ab   3fa94738/aarch64-addon-gosungrow:3.0.7 "/init /usr/local/bi…"   16 minutes ago   Up 16 minutes                                                                                                                                  addon_3fa94738_gosungrow
    
  4. I can use either the CONTAINER ID of 04e0c040d4ab or the NAME addon_3fa94738_gosungrow in the next command. I'm going to use the ID because it's shorter:

    docker exec -it 04e0c040d4ab bash
    
  5. The prompt changes to bash-5.1# to indicate I'm running a shell inside the container. Now I can run other GoSungrow commands:

    GoSungrow --config /data/.GoSungrow/config.json show ps list
    

    and the response is:

    ┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
    ┃ Ps Key           ┃ Ps Id   ┃ Device Type ┃ Device Code ┃ Channel Id ┃ Serial #    ┃ Factory Name ┃ Device Model   ┃
    ┣━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┫
    ┃ 9999999_1_1_1    │ 9999999 │ 1           │ 1           │ 1          │ B2272674442 │ SUNGROW      │ SG5.0RS        ┃
    ┃ 9999999_22_247_1 │ 9999999 │ 22          │ 247         │ 1          │ B2272674442 │ SUNGROW      │ WiNet-S        ┃
    ┃ 9999999_7_1_1    │ 9999999 │ 7           │ 1           │ 1          │ B2272674442 │ SUNGROW      │ SG Smart Meter ┃
    ┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛
    

Meanwhile, the command run by the container (GoSungrow mqtt run) when it starts continues to run in parallel with all this activity.

When I'm done, a couple of Control-D to exit.

@Lmecano
Copy link

Lmecano commented Mar 22, 2024

Hello Paraphraser,

Good to know, may be it's because it's a RPI 3, I don't know

Thank you for the test :)

@guyserotonin
Copy link

Thank you so much @Paraphraser !
Part 2 worked a treat mate.

@igoratencompass
Copy link

@duncanjmck I'm getting the same error ERROR: unknown error 'Repeated request' both in HA and when running GoSungrow binary. Did you get to the bottom of it?

@Paraphraser
Copy link
Author

Long discussion at MickMake issue 111. No actual resolution as yet. But the more people who read/contribute, the greater the chance we will have a collective "ahah" moment.

I have never seen this issue so it's hard for me to throw my usual tools at the problem (eg tcpdump; modified image with additional instrumentation, etc).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment