-Parsing SML Energy Meter Values for Loxone.md

Parsing SML Energy Meter Values for Loxone Miniservers

This is a custom PicoC program to parse the SML energy readings in a tibber+Loxone setup.

Our system runs a tibber Pulse, connected to a EMH ED300L energy meter. We run this code on a Loxone Miniserver, which can run custom PicoC code.

We use the TCP stream method to read the current SML data packet from the tibber bridge

Getting started

On macOS: brew install picoc

Note that the picoc version used in homebrew has some weird quirks. For some tests, I've used this:

git clone git@gitlab.com:zsaleeba/picoc.git
cd picoc
make
ln -s $(pwd)/picoc /usr/local/bin/picoc

This has other quirks, but at least I've been able to fix the code.

On other systems: Install PicoC as you would install any other software.

Now, you can parse the example message:

./run.sh

Running tests

This runs the example message and checks that the dummy setoutput method was called correctly.

./test.sh

Finding new addresses

You'll have to do this process whenever you encounter an energy meter with a different number/structure of reported values (f.ex. single-direction meter or dual for grid feed-in and consumption etc).

1. Run curl to download an example SML message from your energy meter:
   Be sure to replace the username, password, and IP address.

   curl "http://admin:1234-ABCD@10.0.1.134/data.json?node_id=1" -D - > sml_message.bin

2. Run this to convert the file to a hex dump into example.h.

   xxd -i sml_message.bin | awk '/,$/{printf $0;next} {print}' >> example.h

3. Open example.h in a text editor, and make sure the format matches.
   • Remove/comment out the old example_message constants
   • Rename the variables to example_message and example_message_len
   • Add the length in bytes (from example_message_len) into the [] brackets after example_message.
4. Run picoc -s reader.c to run the extraction once so you can read out the addresses.
   • Press Ctrl+C to abort the loop
5. Search for these snippets in the output:

   -> output 2 is at offset 0x00b3, len 5
   -> output 1 is at offset 0x00c8, len 5
   -> output 0 is at offset 0x011c, len 4

   They might not be sorted.
6. Open quick.c, and replace the constants output_num, output_offsets, and output_lengths with the values you've just learned (after sorting them).

Running on Loxone

1. Update configuration in quick.c:
   • HTTP_IP is the IP of your tibber bridge.
   • Set output_num, output_offsets, and output_lengths to the message offsets you desire (See Finding new addresses above.)
     Note that you can't use newlines in arrays.
2. Copy the script to a Loxone custom script block.
3. Set inputs as directed in the top of quick.c
   • Text input 1: base64-coded basicauth username:password (without newline) for your tibber bridge:

     echo -n "admin:1234-ABCD" | base64
     # output: YWRtaW46MTIzNC1BQkNE

   • Numeric input 1: Interval (in seconds) to wait between updates
4. Capture outputs:
   • Add a tracker for text output 1.
     This will contain errors (as long as they persist)
   • Add virtual outputs for numeric outputs 1-n.
     These will contain your desired readings.

Running Examples

To help development and understand the SML format, there's some helper files:

• sml_message.bin which contains a raw SML binary message
• run_perl.sh which runs smldump.pl on that message
• run_python.sh which runs smllib on that message
  To run this, you first need to run pip install smllib

Protocol Description

First byte is Type & length. Length -1 in most cases.

Types (first nibble & 0x70):

• 0 bytearray
• 5 signed
• 6 unsigned
• 7 list (length not -1)

0x1b, 0x1b, 0x1b, 0x1b,  SML Magic Number
0x01, 0x01, 0x01, 0x01,  Version 1
0x76,  List with 6 entries - SmlMessage
  1 0x07 (transaction id): 0x00, 0x13, 0x00, 0x16, 0xc7, 0xa9,
  2 0x62: 0x00 Unsigned 1 byte (k & 0xf - 1 -> 1 byte)
  3 0x62: 0x00 Unsigned 1 byte
  4 0x72,   List with 2 entries
      1 0x63: 0x01, 0x01 Unsigned 2 bytes
      2 0x76: list 6 entries
          1 0x01: 0 bytes
          2 0x01: 0 bytes
          3 0x07: 6 bytes: 0x00, 0x13, 0x04, 0xf0, 0x97, 0xe3
          4 0x0b: 10 bytes bytes: <snip>
          5 0x01: 0 bytes,
          6 0x01: 0 bytes,
  5 0x63: unsigned 2: 0x98, 0x8b
  6 0x00 ?
0x76: list 6 - SmlMessage
  1 0x07: 6 bytes 0x00, 0x13, 0x00, 0x16, 0xc7, 0xaa
  2 0x62 unsigned 1b: 0x00
  3 0x62: 0x00
  4 0x72: list 2  - SmlGetListResponse
      1 0x63: 0x07, 0x01 unsigned 2
      2 0x77 list 7:
          1 0x01
          2 0x0b: 10 bytes: <snip>
          3 0x07: 6 bytes: 0x01, 0x00, 0x62, 0x0a, 0xff, 0xff
          4 0x72: list 2
              1 0x62: 0x01
              2 0x65: 0x04, 0xf0, 0x73, 0x85   current sensor time
            0x77 list 7
              1 0x77 list 7
                1 0x07: 6b 0x81, 0x81, 0xc7, 0x82, 0x03, 0xff  OBIS "Hersteller-Identifikation"
                2 0x01
                3 0x01
                4 0x01
                5 0x01
                6 0x04: 0x45, 0x4d, 0x48 ("EMH")
                7 0x01
            0x77: list 7
              1 0x07: 0x01, 0x00, 0x00, 0x00, 0x09, 0xff OBIS "Server-Id / Geraeteeinzelidentifikation"
              2 0x01
              3 0x01
              4 0x01
              5 0x01
              6 0x0b: <snip>
              7 0x01
            0x77: list 7
              1 0x07: 0x01, 0x00, 0x01, 0x08, 0x00, 0xff OBIS "Zaehlwerk neg. Wirkenergie (Einspeisung), tariflos"
              2 0x64: 0x00, 0x01, 0x82
              3 0x01
              4 0x62: 0x1e -> 30 -> Wh
              5 0x52: 0xff -> scale E-1
              6 0x56, 0x00, 0x00, 0xe6, 0x3b, 0x5f -> 1508847.9 Wh
              7 0x01
              ... The rest is up to the reader

Some example OBIS codes

(These are taken from smldump.pl)

• 0x00, 0x00, 0x60, 0x01, 0xFF, 0xFF: Seriennummer
• 0x01, 0x00, 0x00, 0x00, 0x09, 0xFF: Server-Id / Geraeteeinzelidentifikation
• 0x01, 0x00, 0x01, 0x08, 0x00, 0xFF: Zaehlwerk pos. Wirkenergie (Bezug), tariflos
• 0x01, 0x00, 0x01, 0x08, 0x01, 0xFF: Zaehlwerk pos. Wirkenergie (Bezug), Tarif 1
• 0x01, 0x00, 0x01, 0x08, 0x02, 0xFF: Zaehlwerk pos. Wirkenergie (Bezug), Tarif 2
• 0x01, 0x00, 0x02, 0x08, 0x00, 0xFF: Zaehlwerk neg. Wirkenergie (Einspeisung), tariflos
• 0x01, 0x00, 0x02, 0x08, 0x01, 0xFF: Zaehlwerk neg. Wirkenergie (Einspeisung), Tarif 1 (Bezug)
• 0x01, 0x00, 0x02, 0x08, 0x02, 0xFF: Zaehlwerk neg. Wirkenergie (Einspeisung), Tarif 2 (Einspeisung)
• 0x01, 0x00, 0x0F, 0x07, 0x00, 0xFF: Betrag der aktuellen Wirkleistung
• 0x01, 0x00, 0x10, 0x07, 0x00, 0xFF: Aktuelle Wirkleistung gesamt
• 0x01, 0x00, 0x24, 0x07, 0x00, 0xFF: Aktuelle Wirkleistung L1
• 0x01, 0x00, 0x38, 0x07, 0x00, 0xFF: Aktuelle Wirkleistung L2
• 0x01, 0x00, 0x4c, 0x07, 0x00, 0xFF: Aktuelle Wirkleistung L3
• 0x81, 0x81, 0xC7, 0x82, 0x03, 0xFF: Hersteller-Identifikation
• 0x81, 0x81, 0xC7, 0x82, 0x05, 0xFF: Public Key

Example output

➜ ./run.sh
OPEN /dev/tcp/10.0.1.134/80
WRITE GET /data.json?node_id=1 HTTP/1.0
Authorization: Basic <snip>

 (80)
READ(128, 5000) -> 128 (408 remaining)
READ(128, 5000) -> 128 (280 remaining)
READ(128, 5000) -> 128 (152 remaining)
READ(128, 5000) -> 24 (24 remaining)
READ(128, 5000) -> 0 (0 remaining)
total bytes: 408
Headers are correct.
76: list 6:
 07: <snip> bytes (6)
 62: 00 unsigned 1: 0
 62: 00 unsigned 1: 0
 72: list 2:
   63: 01 01 unsigned 2: 257
   76: list 6:
     01: bytes (0)
     01: bytes (0)
     07: <snip> bytes (6)
     0b: <snip> bytes (10)
     01: bytes (0)
     01: bytes (0)
 63: 9d 18 unsigned 2: 40216
 00: bytes (-1)
76: list 6:
 07: <snip> bytes (6)
 62: 00 unsigned 1: 0
 62: 00 unsigned 1: 0
 72: list 2:
   63: 07 01 unsigned 2: 1793
   77: list 7:
     01: bytes (0)
     0b: <snip> bytes (10)
     07: <snip> bytes (6)
     72: list 2:
       62: 01 unsigned 1: 1
       65: 04 f4 5c 4d unsigned 4: 83123277
     77: list 7:
       77: list 7:
         07: 81 81 c7 82 03 ff bytes (6) <- OBIS
         01: bytes (0)
         01: bytes (0)
         01: bytes (0) <- unit
         01: bytes (0) <- scale (0)
         04: 45 4d 48 bytes (3) scale 0.000000  <- value = 0.000000 ?
         01: bytes (0)
       77: list 7:
         07: 01 00 00 00 09 ff bytes (6) <- OBIS
         01: bytes (0)
         01: bytes (0)
         01: bytes (0) <- unit
         01: bytes (0) <- scale (0)
         0b: <snip> bytes (10) scale 0.000000  <- value = 0.000000 ?
         01: bytes (0)
       77: list 7:
         07: 01 00 01 08 00 ff bytes (6) <- OBIS
         64: 00 01 82 unsigned 3: 386
         01: bytes (0)
         62: 1e unsigned 1: 30 <- unit
         52: ff signed 1: -1 <- scale (-1)
         56: 00 00 e8 3a 6d signed 5: 15219309 scale -1.000000  <- value = 1521930.900000 Wh
         01: bytes (0)
       77: list 7:
         07: 01 00 01 08 01 ff bytes (6) <- OBIS
         01: bytes (0)
         01: bytes (0)
         62: 1e unsigned 1: 30 <- unit
         52: ff signed 1: -1 <- scale (-1)
         56: 00 00 e8 3a 6d signed 5: 15219309 scale -1.000000  <- value = 1521930.900000 Wh -> output 1
SETTING OUTPUT 1 to 1521930.900000

         01: bytes (0)
       77: list 7:
         07: 01 00 01 08 02 ff bytes (6) <- OBIS
         01: bytes (0)
         01: bytes (0)
         62: 1e unsigned 1: 30 <- unit
         52: ff signed 1: -1 <- scale (-1)
         56: 00 00 00 00 00 signed 5: 0 scale -1.000000  <- value = 0.000000 Wh -> output 2
SETTING OUTPUT 2 to 0.000000

         01: bytes (0)
       77: list 7:
         07: 01 00 10 07 00 ff bytes (6) <- OBIS
         01: bytes (0)
         01: bytes (0)
         62: 1b unsigned 1: 27 <- unit
         52: ff signed 1: -1 <- scale (-1)
         55: 00 00 03 ee signed 4: 1006 scale -1.000000  <- value = 100.600000 W -> output 0
SETTING OUTPUT 0 to 100.600000

         01: bytes (0)
       77: list 7:
         07: 81 81 c7 82 05 ff bytes (6) <- OBIS
         01: bytes (0)
         01: bytes (0)
         01: bytes (0) <- unit
         01: bytes (0) <- scale (0)
         83+02: ba df 00 d0 00 00 00 00 00 00 00 00 00 00 00 00 bytes (48) scale 0.000000  <- value = 0.000000 ?
         01: bytes (0)
     01: bytes (0)
     01: bytes (0)
 63: 88 49 unsigned 2: 34889
 00: bytes (-1)
76: list 6:
 07: <snip> bytes (6)
 62: 00 unsigned 1: 0
 62: 00 unsigned 1: 0
 72: list 2:
   63: 02 01 unsigned 2: 513
   71: list 1:
     01: bytes (0)
 63: a9 b9 unsigned 2: 43449
 00: bytes (-1)
00: bytes (-1)
00: bytes (-1)
end of message
crc actual: 0x78a0, swap endian: 0xffffa078, expected: 0xffffa078
bytes (-1)

Notice the stray output of bytes (-1) at the end -- only the PicoC version in the homebrew repos does that. gitlab.com/zsaleeba/picoc and clang don't.

PicoC quirks

Here's a list of quirks and undocumented behavior I've found while developing this program:

On the Loxone Miniserver especially:

• It's slooooow
  • Don't try to run CRC on it or other fancy stuff
  • One round of reader.c takes about 15s on my test device, and only 2-3s of that is the HTTP request
• stream_read() takes a timeout value, and will take exactly that long to complete.
  • So prefer using shorter timeouts, and loop a while until you have the data you want.
• PicoC is running in script mode, so your main() will not automatically be executed unless you add a line main(); at the bottom (a.k.a. picoc -s)
• You can use printf() to debug
  • Output is written to /log/def.log on the SD-Card
  • You can read the file in your browser using http://miniserver/dev/fsget/log/def.log or via (S)FTP
  • Output lines are terminated automatically
    • If you add \n in your call, you'll cause an empty line
    • You can't printf lines piece-by-piece because of this
• The program is only run once
  • If it calls exit(), or the script ends, it will stop running until you restart the Miniserver (f.ex. by updating the config)
  • If you want to react to input, you'll have to run a loop and use sleep() or sleeps()
• If your program causes an error (segfault, PicoC errors, etc), the Miniserver will disable the script and reboot
  • All in-memory info will be lost - and you possibly won't get logs telling you what went wrong
• The PicoC version on the Miniserver:
  • Crashes when you try to use function-scoped static variables -- use global variables instead
  • Will throw syntax errors when trying to do ternary operations (f.ex. a>b ? a : b)
  • Doesn't support #include
• exit() doesn't take an argument
  • but on regular PicoC, it needs a return code like exit(1)
• sleep(int time) on Loxone takes milliseconds
  • on latest PicoC & regular C it takes seconds
  • There's sleeps(int time) on Loxone if you need it

General PicoC quirks:

• You can't concatenate string constants by breaking them into multiple pieces like this: "a " "string"
• Note that PicoC exerts weird behavior when using if (DEBUG) { in some places
  • Examples are suddenly throwing type errors in if-conditions involving struct member accesses
  • we use if (DEBUG) { instead to avoid these
• #define works like a function call
  • It can only return an expression
  • It can't be used for type definitions
• typedef struct doesn't work
• Arrays have to have their size hardcoded
  • f.ex.: char* bytes[4] = {1, 2, 3, 4}
• Arrays have to be defined in a single line, no linebreaks or \ allowed

Links

• Wikipedia: Smart Message Language (German)
• Loxone PicoC documentation
• Perl SMLDump script
• Python SmlLib
• PicoC interpreter
• OpenSML
• EMH ED300L Manual (German)
• tibber Pulse (German)
• Loxone Miniserver (German)
• Information about tibber Bridge API (German)
• Some more info

License Notes

• PicoC underlies the BSD-3-Clause license.
• smldump.pl and Python SmlLib are both provided under the GNU GPLv3
• The CRC checking code in reader.c is taken from OpenSML, licensed under the MIT license.
• This picoc program, documentation and auxiliary files are provided under the BSD 3-Clause license.

.gitignore

a.out
.DS_Store
*.dSYM/
.vscode
/picoc/

emh.py

#!/usr/bin/env python3

import sys
from smllib import SmlStreamReader

stream = SmlStreamReader()
data = sys.stdin.buffer.read()
# stream.add(b'BytesFromSerialPort')
stream.add(data)
sml_frame = stream.get_frame()
if sml_frame is None:
    print('Bytes missing')

# Shortcut to extract all values without parsing the whole frame
obis_values = sml_frame.get_obis()

# return all values but slower
parsed_msgs = sml_frame.parse_frame()
for msg in parsed_msgs:
    # prints a nice overview over the received values
    print(msg.format_msg())

example.h

// curl "http://admin:1234-ABCD@10.0.1.134/data.json?node_id=1" -D - > sml_message.bin
// xxd -i sml_message.bin | awk '/,$/{printf $0;next} {print}'


// example message for single-direction meter:
// unsigned char example_message[408] = {
//  0x48, 0x54, 0x54, 0x50, 0x2f, 0x31, 0x2e, 0x31, 0x20, 0x32, 0x30, 0x30, 0x20, 0x4f, 0x4b, 0x0d, 0x0a, 0x43, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, 0x2d, 0x54, 0x79, 0x70, 0x65, 0x3a, 0x20, 0x74, 0x65, 0x78, 0x74, 0x2f, 0x74, 0x65, 0x78, 0x74, 0x0d, 0x0a, 0x43, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, 0x2d, 0x4c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x3a, 0x20, 0x33, 0x32, 0x34, 0x0d, 0x0a, 0x63, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x3a, 0x20, 0x63, 0x6c, 0x6f, 0x73, 0x65, 0x0d, 0x0a, 0x0d, 0x0a, 0x1b, 0x1b, 0x1b, 0x1b, 0x01, 0x01, 0x01, 0x01, 0x76, 0x07, 0x00, 0x13, 0xca, 0xfe, 0xba, 0xbe, 0x62, 0x00, 0x62, 0x00, 0x72, 0x63, 0x01, 0x01, 0x76, 0x01, 0x01, 0x07, 0x00, 0x13, 0xca, 0xfe, 0xba, 0xbe, 0x0b, 0xca, 0xfe, 0xba, 0xbe, 0x48, 0x00, 0xca, 0xfe, 0xba, 0xbe, 0x01, 0x01, 0x63, 0x9d, 0x18, 0x00, 0x76, 0x07, 0x00, 0x13, 0x00, 0x1d, 0x29, 0x90, 0x62, 0x00, 0x62, 0x00, 0x72, 0x63, 0x07, 0x01, 0x77, 0x01, 0x0b, 0xca, 0xfe, 0xba, 0xbe, 0x48, 0x00, 0xca, 0xfe, 0xba, 0xbe, 0x07, 0x01, 0x00, 0xca, 0xfe, 0xba, 0xbe, 0x72, 0x62, 0x01, 0x65, 0x04, 0xf4, 0x5c, 0x4d, 0x77, 0x77, 0x07, 0x81, 0x81, 0xc7, 0x82, 0x03, 0xff, 0x01, 0x01, 0x01, 0x01, 0x04, 0x45, 0x4d, 0x48, 0x01, 0x77, 0x07, 0x01, 0x00, 0x00, 0x00, 0x09, 0xff, 0x01, 0x01, 0x01, 0x01, 0x0b, 0xca, 0xfe, 0xba, 0xbe, 0x48, 0x00, 0xca, 0xfe, 0xba, 0xbe, 0x01, 0x77, 0x07, 0x01, 0x00, 0x01, 0x08, 0x00, 0xff, 0x64, 0x00, 0x01, 0x82, 0x01, 0x62, 0x1e, 0x52, 0xff, 0x56, 0x00, 0x00, 0xe8, 0x3a, 0x6d, 0x01, 0x77, 0x07, 0x01, 0x00, 0x01, 0x08, 0x01, 0xff, 0x01, 0x01, 0x62, 0x1e, 0x52, 0xff, 0x56, 0x00, 0x00, 0xe8, 0x3a, 0x6d, 0x01, 0x77, 0x07, 0x01, 0x00, 0x01, 0x08, 0x02, 0xff, 0x01, 0x01, 0x62, 0x1e, 0x52, 0xff, 0x56, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x77, 0x07, 0x01, 0x00, 0x10, 0x07, 0x00, 0xff, 0x01, 0x01, 0x62, 0x1b, 0x52, 0xff, 0x55, 0x00, 0x00, 0x03, 0xee, 0x01, 0x77, 0x07, 0x81, 0x81, 0xc7, 0x82, 0x05, 0xff, 0x01, 0x01, 0x01, 0x01, 0x83, 0x02, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe, 0xba, 0xbe, 0x01, 0x01, 0x01, 0x63, 0x88, 0x49, 0x00, 0x76, 0x07, 0x00, 0x13, 0xca, 0xfe, 0xba, 0xbe, 0x62, 0x00, 0x62, 0x00, 0x72, 0x63, 0x02, 0x01, 0x71, 0x01, 0x63, 0xa9, 0xb9, 0x00, 0x00, 0x00, 0x1b, 0x1b, 0x1b, 0x1b, 0x1a, 0x02, 0x25, 0x42
// };
// unsigned int example_message_len = 408;

// example for a dual-direction meter:
unsigned char example_message[472] = {
  0x48, 0x54, 0x54, 0x50, 0x2f, 0x31, 0x2e, 0x31, 0x20, 0x32, 0x30, 0x30,  0x20, 0x4f, 0x4b, 0x0d, 0x0a, 0x43, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74,  0x2d, 0x54, 0x79, 0x70, 0x65, 0x3a, 0x20, 0x74, 0x65, 0x78, 0x74, 0x2f,  0x74, 0x65, 0x78, 0x74, 0x0d, 0x0a, 0x43, 0x6f, 0x6e, 0x74, 0x65, 0x6e,  0x74, 0x2d, 0x4c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x3a, 0x20, 0x33, 0x38,  0x38, 0x0d, 0x0a, 0x63, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x69, 0x6f,  0x6e, 0x3a, 0x20, 0x63, 0x6c, 0x6f, 0x73, 0x65, 0x0d, 0x0a, 0x0d, 0x0a,  0x1b, 0x1b, 0x1b, 0x1b, 0x01, 0x01, 0x01, 0x01, 0x76, 0x07, 0x00, 0x0c,  0x02, 0x78, 0x96, 0x79, 0x62, 0x00, 0x62, 0x00, 0x72, 0x63, 0x01, 0x01,  0x76, 0x01, 0x01, 0x07, 0x00, 0x0c, 0x06, 0xdb, 0xdc, 0xd3, 0x0b, 0x09,  0x01, 0x45, 0x4d, 0x48, 0x00, 0x00, 0x90, 0xa4, 0xc8, 0x01, 0x01, 0x63,  0x16, 0x6a, 0x00, 0x76, 0x07, 0x00, 0x0c, 0x02, 0x78, 0x96, 0x7a, 0x62,  0x00, 0x62, 0x00, 0x72, 0x63, 0x07, 0x01, 0x77, 0x01, 0x0b, 0x09, 0x01,  0x45, 0x4d, 0x48, 0x00, 0x00, 0x90, 0xa4, 0xc8, 0x07, 0x01, 0x00, 0x62,  0x0a, 0xff, 0xff, 0x72, 0x62, 0x01, 0x65, 0x06, 0xdb, 0x04, 0xda, 0x7a,  0x77, 0x07, 0x81, 0x81, 0xc7, 0x82, 0x03, 0xff, 0x01, 0x01, 0x01, 0x01,  0x04, 0x45, 0x4d, 0x48, 0x01, 0x77, 0x07, 0x01, 0x00, 0x00, 0x00, 0x09,  0xff, 0x01, 0x01, 0x01, 0x01, 0x0b, 0x09, 0x01, 0x45, 0x4d, 0x48, 0x00,  0x00, 0x90, 0xa4, 0xc8, 0x01, 0x77, 0x07, 0x01, 0x00, 0x01, 0x08, 0x00,  0xff, 0x64, 0x01, 0x01, 0xa0, 0x01, 0x62, 0x1e, 0x52, 0xff, 0x56, 0x00,  0x1b, 0x9f, 0x9e, 0x12, 0x01, 0x77, 0x07, 0x01, 0x00, 0x02, 0x08, 0x00,  0xff, 0x64, 0x01, 0x01, 0xa0, 0x01, 0x62, 0x1e, 0x52, 0xff, 0x56, 0x00,  0x01, 0x32, 0xe8, 0x98, 0x01, 0x77, 0x07, 0x01, 0x00, 0x01, 0x08, 0x01,  0xff, 0x01, 0x01, 0x62, 0x1e, 0x52, 0xff, 0x56, 0x00, 0x1b, 0x9f, 0x9e,  0x12, 0x01, 0x77, 0x07, 0x01, 0x00, 0x02, 0x08, 0x01, 0xff, 0x01, 0x01,  0x62, 0x1e, 0x52, 0xff, 0x56, 0x00, 0x01, 0x32, 0xe8, 0x98, 0x01, 0x77,  0x07, 0x01, 0x00, 0x01, 0x08, 0x02, 0xff, 0x01, 0x01, 0x62, 0x1e, 0x52,  0xff, 0x56, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x77, 0x07, 0x01, 0x00,  0x02, 0x08, 0x02, 0xff, 0x01, 0x01, 0x62, 0x1e, 0x52, 0xff, 0x56, 0x00,  0x00, 0x00, 0x00, 0x00, 0x01, 0x77, 0x07, 0x01, 0x00, 0x10, 0x07, 0x00,  0xff, 0x01, 0x01, 0x62, 0x1b, 0x52, 0xff, 0x55, 0xff, 0xff, 0xff, 0x4f,  0x01, 0x77, 0x07, 0x81, 0x81, 0xc7, 0x82, 0x05, 0xff, 0x01, 0x01, 0x01,  0x01, 0x83, 0x02, 0xb9, 0x1f, 0xba, 0xfc, 0x05, 0xc3, 0x02, 0x87, 0x4c,  0xa8, 0x80, 0x82, 0x20, 0x42, 0x3d, 0x72, 0x1c, 0x8d, 0x76, 0xd3, 0x00,  0x9a, 0x90, 0x51, 0x5b, 0x28, 0x0d, 0x0f, 0x6b, 0x74, 0x4f, 0x0d, 0xa7,  0xb7, 0x6f, 0x60, 0xb2, 0xb4, 0xbc, 0x2f, 0x27, 0xb6, 0x9a, 0x32, 0x2d,  0x0f, 0xe9, 0x31, 0x01, 0x01, 0x01, 0x63, 0x18, 0x07, 0x00, 0x76, 0x07,  0x00, 0x0c, 0x02, 0x78, 0x96, 0x7d, 0x62, 0x00, 0x62, 0x00, 0x72, 0x63,  0x02, 0x01, 0x71, 0x01, 0x63, 0x51, 0x6e, 0x00, 0x1b, 0x1b, 0x1b, 0x1b,  0x1a, 0x00, 0x30, 0x8c
};
unsigned int example_message_len = 472;

polyfill.h

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <unistd.h>

#include "example.h"

struct _STREAM {
    int read_pos;
};

struct _STREAM* stream_create(char* filename, int read, int append) {
    static struct _STREAM stream;
    stream.read_pos = 0;
    if (DEBUG) {
        printf("OPEN %s\n", filename);
    }
    return &stream;
}

int stream_write(struct _STREAM* stream, void *ptr, int size) {
    if (DEBUG) {
        printf("WRITE %s (%d)\n", (char*) ptr, size);
    }
    return size;
}
int stream_read(struct _STREAM* stream, void* ptr, int size, int timeout){
    int out = 0;
    int remaining = example_message_len - stream->read_pos;
    if (remaining > 0) {
        out = min(size, remaining);
        memcpy(ptr, &example_message[stream->read_pos], out);
    }
    stream->read_pos += out;
    if (DEBUG) {
        printf("READ(%d, %d) -> %d (%d remaining)\n", size, timeout, out, remaining);
    }
    return out;
}
void stream_flush(struct _STREAM* stream) {}
void stream_close(struct _STREAM* stream) {}

void setoutput(int output, float value){
    // Dummy
    printf("\nSETTING OUTPUT %d to %f\n", output, value);
}

char* getinputtext(int num) {
    switch (num) {
    case 0:
        return "basic-auth-base64";
    default:
        printf("input %d not set\n", num);
        exit(1);
    }
}

float getinput(int num) {
    switch (num) {
    case 0:
        return 0.005; // Computer sleep() takes seconds, Loxone takes ms
    default:
        printf("input %d not set\n", num);
        exit(1);
    }
}

void setoutputtext(int output, char* str) {
    printf("SETTING TEXT OUTPUT %d to '%s'\n", output, str);
}

quick.c

/*
Inputs:
- Text 1: Base64-kodiertes Basic-Auth Username & Passwort.
  In den meisten Kommandozeilen kann man das so generieren:
    echo -n "admin:1234-ABCD" | base64
- Input 1: Wartezeit (in Sekunden) zwischen zwei Anfragen.

Outputs:
- Text 1: evtl. Fehler, falls vorhanden.
  Sobald die Abfrage wieder funktioniert, wird das Feld wieder leer.

- Output 1-n: Ausgelesene Werte.
  Können über folgende Variablen unten eingestellt werden:
    - output_num: Anzahl der Outputs
    - output_offsets: Position der Werte in SML-Nachricht. Kann durch Ausführen von beigelegtem Tool `reader.c` herausgefunden werden.
    - output_lengths: Ebenso
    - Scale: Skalierung der Werte. Bei den meisten Zählern 0.1.
*/

#define HTTP_PATH "/data.json?node_id=1"
#define HTTP_IP "10.0.1.134"

#define READ_TIMEOUT_MS 100
#define BUFF_SIZE 4096

int output_num = 3;
int output_offsets[3] = {0x11c, 0xc8, 0xb3};
int output_lengths[3] = {4, 5, 5};
double scale = 0.1;

// DEBUG 1: Relativ viel Log-output
// DEBUG 0: kein Log-output
#define DEBUG 1

// CRC 1: Übertragungsfehler erkennen, könnte Performance einschränken (nachmessen!)
// CRC 0: Keine Fehlererkennung, könnte aber ab und zu Müll-Werte ausgeben.
#define CRC 1

/* Ende der Einstellungen */

int min(int a, int b) {
    if (a < b) {
        return a;
    } else {
        return b;
    }
}

// Polyfills & includes for "local mode"
#ifndef setoutput
#define local_mode 1
#include "polyfill.h"
#endif

#ifdef local_mode
struct _STREAM* http_stream = 0;
#else
STREAM* http_stream = 0;
#endif

struct buffer {
    int len;
    int pos;
    char* data;
};

void cleanup() {
    if (http_stream != 0) {
        stream_close(http_stream);
        http_stream = 0;
    }
}

#define ERR_LEN 256;
char error_str[256];

void error(char* msg) {
    setoutputtext(0, msg);
#ifdef __clang__
    cleanup();
    exit(1);
#endif
}
void no_error() {
    setoutputtext(0, "");
}

long to_int(char* data, int offset, int len) {
    long value = 0;
    for (int i=0; i<len; i++) { 
        value = (value << 8) + (unsigned char) data[offset+i];
    }
    return value;
}

int read4(struct buffer* buf) {
    int value = 0;
    value += buf->data[buf->pos++] << 24;
    value += buf->data[buf->pos++] << 16;
    value += buf->data[buf->pos++] << 8;
    value += buf->data[buf->pos++];
    return value;
}

int read_magic(struct buffer* buf) {
    int magic = read4(buf);
    if (magic != 0x1b1b1b1b) {
        sprintf(error_str, "Got 0x%08x:, expected magic", magic);
        error(error_str);
        return 1;
    }
    return 0;
}
int read_headers(struct buffer* buf) {
    if (read_magic(buf) != 0) {
        return -1;
    }
    int version = read4(buf);
    if (version != 0x01010101) {
        sprintf(error_str, "Got 0x%08x: ", version);
        error(error_str);
        return 1;
    }
    return 0;
}

/*
 * CRC16 implementation From OpenSML
 * Copyright(c) 2017 I. Fischer (ingo@fischer-ka.de)
 * MIT Licensed
 */
unsigned short crc_table[256] = {0x0000, 0x1189, 0x2312, 0x329b, 0x4624, 0x57ad, 0x6536, 0x74bf, 0x8c48, 0x9dc1,
    0xaf5a, 0xbed3, 0xca6c, 0xdbe5, 0xe97e, 0xf8f7, 0x1081, 0x0108, 0x3393, 0x221a, 0x56a5, 0x472c, 0x75b7,
    0x643e, 0x9cc9, 0x8d40, 0xbfdb, 0xae52, 0xdaed, 0xcb64, 0xf9ff, 0xe876, 0x2102, 0x308b, 0x0210, 0x1399,
    0x6726, 0x76af, 0x4434, 0x55bd, 0xad4a, 0xbcc3, 0x8e58, 0x9fd1, 0xeb6e, 0xfae7, 0xc87c, 0xd9f5, 0x3183,
    0x200a, 0x1291, 0x0318, 0x77a7, 0x662e, 0x54b5, 0x453c, 0xbdcb, 0xac42, 0x9ed9, 0x8f50, 0xfbef, 0xea66,
    0xd8fd, 0xc974, 0x4204, 0x538d, 0x6116, 0x709f, 0x0420, 0x15a9, 0x2732, 0x36bb, 0xce4c, 0xdfc5, 0xed5e,
    0xfcd7, 0x8868, 0x99e1, 0xab7a, 0xbaf3, 0x5285, 0x430c, 0x7197, 0x601e, 0x14a1, 0x0528, 0x37b3, 0x263a,
    0xdecd, 0xcf44, 0xfddf, 0xec56, 0x98e9, 0x8960, 0xbbfb, 0xaa72, 0x6306, 0x728f, 0x4014, 0x519d, 0x2522,
    0x34ab, 0x0630, 0x17b9, 0xef4e, 0xfec7, 0xcc5c, 0xddd5, 0xa96a, 0xb8e3, 0x8a78, 0x9bf1, 0x7387, 0x620e,
    0x5095, 0x411c, 0x35a3, 0x242a, 0x16b1, 0x0738, 0xffcf, 0xee46, 0xdcdd, 0xcd54, 0xb9eb, 0xa862, 0x9af9,
    0x8b70, 0x8408, 0x9581, 0xa71a, 0xb693, 0xc22c, 0xd3a5, 0xe13e, 0xf0b7, 0x0840, 0x19c9, 0x2b52, 0x3adb,
    0x4e64, 0x5fed, 0x6d76, 0x7cff, 0x9489, 0x8500, 0xb79b, 0xa612, 0xd2ad, 0xc324, 0xf1bf, 0xe036, 0x18c1,
    0x0948, 0x3bd3, 0x2a5a, 0x5ee5, 0x4f6c, 0x7df7, 0x6c7e, 0xa50a, 0xb483, 0x8618, 0x9791, 0xe32e, 0xf2a7,
    0xc03c, 0xd1b5, 0x2942, 0x38cb, 0x0a50, 0x1bd9, 0x6f66, 0x7eef, 0x4c74, 0x5dfd, 0xb58b, 0xa402, 0x9699,
    0x8710, 0xf3af, 0xe226, 0xd0bd, 0xc134, 0x39c3, 0x284a, 0x1ad1, 0x0b58, 0x7fe7, 0x6e6e, 0x5cf5, 0x4d7c,
    0xc60c, 0xd785, 0xe51e, 0xf497, 0x8028, 0x91a1, 0xa33a, 0xb2b3, 0x4a44, 0x5bcd, 0x6956, 0x78df, 0x0c60,
    0x1de9, 0x2f72, 0x3efb, 0xd68d, 0xc704, 0xf59f, 0xe416, 0x90a9, 0x8120, 0xb3bb, 0xa232, 0x5ac5, 0x4b4c,
    0x79d7, 0x685e, 0x1ce1, 0x0d68, 0x3ff3, 0x2e7a, 0xe70e, 0xf687, 0xc41c, 0xd595, 0xa12a, 0xb0a3, 0x8238,
    0x93b1, 0x6b46, 0x7acf, 0x4854, 0x59dd, 0x2d62, 0x3ceb, 0x0e70, 0x1ff9, 0xf78f, 0xe606, 0xd49d, 0xc514,
    0xb1ab, 0xa022, 0x92b9, 0x8330, 0x7bc7, 0x6a4e, 0x58d5, 0x495c, 0x3de3, 0x2c6a, 0x1ef1, 0x0f78};

int crc16(char* buffer, int len) {
    int crc = 0xffff;
    for (int i = 0; i < len; i++) {
        crc = (crc >> 8) ^ (crc_table[(crc ^ buffer[i]) & 0xff]);
    }
    crc ^= 0xffff;
    return crc;
}
// end of CRC implementation

void handle_crc(struct buffer* buf) {
    // Find second SML header
    for (;buf->pos+4 < buf->len; buf->pos++) {
        if (memcmp(&buf->data[buf->pos], "\x1b\x1b\x1b\x1b", 4) == 0) {
            if (DEBUG) {
                printf("Found CRC check at offset %d\n", buf->pos);
            }
            break;
        }
    }
    if (buf->pos+4 >= buf->len) {
        sprintf(error_str, "Could not find CRC header");
        error(error_str);
        return 1;
    }

    int end_of_main_message = buf->pos + 4 + 2; // CRC Header is also checksummed
    read_magic(buf);
    char _1a = buf->data[buf->pos++];
    if (_1a != 0x1a) {
        printf("Got 0x%x, wanted 0x1a\n", _1a);
        error("Invalid CRC header");
    }
    buf->pos++; // skip "number of bytes added" field
    short expected_crc = (buf->data[buf->pos++] << 8) + (buf->data[buf->pos++] & 0xff);

    short actual_crc = crc16(buf->data, end_of_main_message);
    short actual_rotated = ((actual_crc & 0xff) << 8) + ((actual_crc & 0xff00) >> 8);
    
    if (actual_crc != expected_crc && actual_rotated != expected_crc) {
        printf("crc actual: 0x%0x, swap endian: 0x%04x, expected: 0x%04x\n", actual_crc, actual_rotated, expected_crc);
        error("CRC mismatch");
    } else if (DEBUG) {
        printf("crc actual: 0x%04x, swap endian: 0x%04x, expected: 0x%04x\n", actual_crc, actual_rotated, expected_crc);
    }
}

int parse(struct buffer* buf) {
    if (read_headers(buf) != 0) {
        return 1;
    }
    
    for (int out=0; out<output_num; out++) {
        int offset = output_offsets[out];
        int len = output_lengths[out];
        long value = to_int(buf->data, offset, len);
        setoutput(out, (double)value * scale);
    }
    if (CRC) {
        if (DEBUG) {
            printf("Starting CRC check...\n");
        }
        handle_crc(buf);
        if (DEBUG) {
            printf("CRC done.\n");
        }
    }
    return 0;
}

int skip_headers(struct buffer* buf) {
    // cut HTTP header: look for \r\n\r\n
    for (int offset = 0; offset < buf->len; offset++) {
        if (memcmp(&buf->data[offset], "\r\n\r\n", 4) == 0) {
            return offset+4;
        }
    }
    sprintf(error_str, "Could not find end of HTTP headers");
    error(error_str);
    return -1;
}

char* http_basicauth;

int run_fetch() {
    if (DEBUG) {
        printf("start.");
    }
    char filename[256]; filename[0] = 0;
    strcat(filename, "/dev/tcp/");
    strcat(filename, HTTP_IP);
    strcat(filename, "/80");
    http_stream = stream_create(filename, 0, 0);
    if (DEBUG) {
        printf("stream create.");
    }
    char command[256]; command[0] = 0;
    strcat(command, "GET ");
    strcat(command, HTTP_PATH);
    strcat(command, " HTTP/1.0\r\nAuthorization: Basic ");
    strcat(command, http_basicauth);
    strcat(command, "\r\n\r\n");

    int written = stream_write(http_stream, command, strlen(command));
    if (written != strlen(command)) {
        sprintf(error_str, "HTTP request failed: should have written %ld bytes, wrote %d: ", strlen(command), written);
        error(error_str);
        return 1;
    }
    stream_flush(http_stream);
    if (DEBUG) {
        printf("req sent.");
    }

    struct buffer buf;
    char read_buffer[BUFF_SIZE];
    buf.pos = 0;
    buf.len = 0;
    buf.data = read_buffer;

    int loop = 1;
    do {
        int read = stream_read(http_stream, &buf.data[buf.len], BUFF_SIZE - buf.len, READ_TIMEOUT_MS);
        buf.len += read;

        //TODO check for "Content-Length:", parse, and read that number of bytes after header.
        loop = (read > 0);
        loop &= (buf.len < BUFF_SIZE);
    } while (loop);
    if (DEBUG) {
        printf("total bytes: %d", buf.len);
    }
    cleanup();
    int offset = skip_headers(&buf);
    buf.data+=offset;
    if (DEBUG) {
        printf("starting parse.");
    }
    if (parse(&buf) == 0) {
        no_error();
    }
    return 0;
}

int main() {
    http_basicauth = getinputtext(0);

    float repeat_time = getinput(0);
    while (1) {
        if (DEBUG) {
            printf("running fetch");
        }
        run_fetch();
        if (DEBUG) {
            printf("done. sleeping %fs", repeat_time);
        }
        sleep((int)(repeat_time * 1000));
    }
}

#ifndef __clang__
// Loxone PicoC is running in script mode
main();
#endif

reader.c

// Otherwise, configure your HTTP endpoint:
#define HTTP_PATH "/data.json?node_id=1"
#define HTTP_IP "10.0.1.134"
char* http_basicauth;
#define READ_TIMEOUT_MS 5000

// set to 0 to silence all output
#define DEBUG 0

// helpers
int min(int a, int b) {
    // Loxone PicoC can't do ternary operators
    if (a < b) {
		return a;
	} else {
		return b;
	}
}

// Polyfills & includes for "local mode"
#ifndef setoutput
#define local_mode 1
#include "polyfill.h"
#endif

// need to change this to `STREAM*` for loxone
#ifdef local_mode
struct _STREAM* http_stream = 0;
#else
STREAM* http_stream = 0;
#endif

int output_num = 3;
char output_obis[3*6] = {/*Leistung*/ 0x01, 0x00, 0x10, 0x07, 0x00, 0xFF, /*Zähler Bezug*/ 0x01, 0x00, 0x01, 0x08, 0x01, 0xFF, /*Zähler Einspeisung*/ 0x01, 0x00, 0x02, 0x08, 0x00, 0xFF};

char* units[63] = { "?", "a", "mo", "wk", "d", "h", "min", "s", "º", "ºC", "geld", "m", "m/s", "m³", "m³ (corr)", "m³/h", "m³/h (corr)", "m³/d", "m³/d (corr)", "l", "kg", "N", "Nm", "Pa", "bar", "J", "J/h", /*27*/ "W", "VA", "var", /*30*/"Wh", "VAh", "varh", "A", "C", "V", "V/m", "F", "Ω", "Ωm²/m", "Wb", "T", "A/m", "H", "Hz", "1/(Wh)", "1/(varh)", "1/(VAh)", "V²h", "A²h", "kg/s", "S, mho", "K", "1/(V²h)", "1/(A²h)", "1/m³", "%", "Ah", "Wh/m³", "J/m³", "Mol %", "g/m³", "Pa s"};

struct buffer {
    int len;
    int pos;
    char* data;
};

// max len for SML bytearrays seems to be 16 bytes
struct array {
    int len;
    char data[16];
};

void debug_array(struct array* arr, int newline) {
    if (!DEBUG) { return; }

    for (int i=0; i < min(16, arr->len); i++) {
        printf("%02x ", arr->data[i]);
    }
    if (newline == 1) {
        printf("\n");
    }
}
void debug_bytes(char* array, int length) {
    if (!DEBUG) { return; }
    for (int i=0; i<length; i++) {
        printf(" %02x", array[i]);
    }
    printf("\n");
}
void print_space(int level) {
    if (!DEBUG) { return; }
    if (level >= 0) {
        printf("%*c", (level+1)*2, ' ');
    }
}

void cleanup() {
    if (http_stream != 0) {
        stream_close(http_stream);
        http_stream = 0;
    }
}

void error(char* msg) {
    printf("Error: %s\n", msg);
    // set outputs
    for (int i=0; i<output_num; i++) {
        setoutput(i, 0.0);
    }
    cleanup();
#ifdef __clang__
    exit(1);
#else
    exit();
#endif
}

int read4(struct buffer* buf) {
    int value = 0;
    value += buf->data[buf->pos++] << 24;
    value += buf->data[buf->pos++] << 16;
    value += buf->data[buf->pos++] << 8;
    value += buf->data[buf->pos++];
    return value;
}
long to_int(struct array* arr) {
    long value = 0;
    for (int i=0; i<arr->len; i++) { 
        value = (value << 8) + (unsigned char) arr->data[i];
    }
    return value;
}

struct array read_array(struct buffer* buf, int len) {
    struct array out;
    memset(out.data, 0, 16);
    out.len = len;
    if (len > 0) {
        if (len > 16) {
            // unsupported... read nothing but advance pointer
            out.data[0] = 0xba;
            out.data[1] = 0xdf;
            out.data[2] = 0x00;
            out.data[3] = 0xd0;
        } else {
            memcpy(out.data, buf->data+buf->pos, len);
        }
        buf->pos += len;
    }
    return out;
}

void read_magic(struct buffer* buf) {
    int magic = read4(buf);
    if (magic != 0x1b1b1b1b) {
        printf("Got 0x%08x: ", magic);
        error("Expected magic");
    }
}
void read_headers(struct buffer* buf) {
    read_magic(buf);
    int version = read4(buf);
    if (version != 0x01010101) {
        printf("Got 0x%08x: ", version);
        error("Expected version");
    }
}

/*
 * CRC16 implementation From OpenSML
 * Copyright(c) 2017 I. Fischer (ingo@fischer-ka.de)
 * MIT Licensed
 */
unsigned short crc_table[256] = {0x0000, 0x1189, 0x2312, 0x329b, 0x4624, 0x57ad, 0x6536, 0x74bf, 0x8c48, 0x9dc1,
    0xaf5a, 0xbed3, 0xca6c, 0xdbe5, 0xe97e, 0xf8f7, 0x1081, 0x0108, 0x3393, 0x221a, 0x56a5, 0x472c, 0x75b7,
    0x643e, 0x9cc9, 0x8d40, 0xbfdb, 0xae52, 0xdaed, 0xcb64, 0xf9ff, 0xe876, 0x2102, 0x308b, 0x0210, 0x1399,
    0x6726, 0x76af, 0x4434, 0x55bd, 0xad4a, 0xbcc3, 0x8e58, 0x9fd1, 0xeb6e, 0xfae7, 0xc87c, 0xd9f5, 0x3183,
    0x200a, 0x1291, 0x0318, 0x77a7, 0x662e, 0x54b5, 0x453c, 0xbdcb, 0xac42, 0x9ed9, 0x8f50, 0xfbef, 0xea66,
    0xd8fd, 0xc974, 0x4204, 0x538d, 0x6116, 0x709f, 0x0420, 0x15a9, 0x2732, 0x36bb, 0xce4c, 0xdfc5, 0xed5e,
    0xfcd7, 0x8868, 0x99e1, 0xab7a, 0xbaf3, 0x5285, 0x430c, 0x7197, 0x601e, 0x14a1, 0x0528, 0x37b3, 0x263a,
    0xdecd, 0xcf44, 0xfddf, 0xec56, 0x98e9, 0x8960, 0xbbfb, 0xaa72, 0x6306, 0x728f, 0x4014, 0x519d, 0x2522,
    0x34ab, 0x0630, 0x17b9, 0xef4e, 0xfec7, 0xcc5c, 0xddd5, 0xa96a, 0xb8e3, 0x8a78, 0x9bf1, 0x7387, 0x620e,
    0x5095, 0x411c, 0x35a3, 0x242a, 0x16b1, 0x0738, 0xffcf, 0xee46, 0xdcdd, 0xcd54, 0xb9eb, 0xa862, 0x9af9,
    0x8b70, 0x8408, 0x9581, 0xa71a, 0xb693, 0xc22c, 0xd3a5, 0xe13e, 0xf0b7, 0x0840, 0x19c9, 0x2b52, 0x3adb,
    0x4e64, 0x5fed, 0x6d76, 0x7cff, 0x9489, 0x8500, 0xb79b, 0xa612, 0xd2ad, 0xc324, 0xf1bf, 0xe036, 0x18c1,
    0x0948, 0x3bd3, 0x2a5a, 0x5ee5, 0x4f6c, 0x7df7, 0x6c7e, 0xa50a, 0xb483, 0x8618, 0x9791, 0xe32e, 0xf2a7,
    0xc03c, 0xd1b5, 0x2942, 0x38cb, 0x0a50, 0x1bd9, 0x6f66, 0x7eef, 0x4c74, 0x5dfd, 0xb58b, 0xa402, 0x9699,
    0x8710, 0xf3af, 0xe226, 0xd0bd, 0xc134, 0x39c3, 0x284a, 0x1ad1, 0x0b58, 0x7fe7, 0x6e6e, 0x5cf5, 0x4d7c,
    0xc60c, 0xd785, 0xe51e, 0xf497, 0x8028, 0x91a1, 0xa33a, 0xb2b3, 0x4a44, 0x5bcd, 0x6956, 0x78df, 0x0c60,
    0x1de9, 0x2f72, 0x3efb, 0xd68d, 0xc704, 0xf59f, 0xe416, 0x90a9, 0x8120, 0xb3bb, 0xa232, 0x5ac5, 0x4b4c,
    0x79d7, 0x685e, 0x1ce1, 0x0d68, 0x3ff3, 0x2e7a, 0xe70e, 0xf687, 0xc41c, 0xd595, 0xa12a, 0xb0a3, 0x8238,
    0x93b1, 0x6b46, 0x7acf, 0x4854, 0x59dd, 0x2d62, 0x3ceb, 0x0e70, 0x1ff9, 0xf78f, 0xe606, 0xd49d, 0xc514,
    0xb1ab, 0xa022, 0x92b9, 0x8330, 0x7bc7, 0x6a4e, 0x58d5, 0x495c, 0x3de3, 0x2c6a, 0x1ef1, 0x0f78};

int crc16(char* buffer, int len) {
    int crc = 0xffff;
    for (int i = 0; i < len; i++) {
        crc = (crc >> 8) ^ (crc_table[(crc ^ buffer[i]) & 0xff]);
    }
    crc ^= 0xffff;
    return crc;
}
// end of CRC implementation

void handle_eof(struct buffer* buf) {
    if (DEBUG) {
        printf("end of message\n");
    }
    buf->pos--;
    int end_of_main_message = buf->pos + 4 + 2; // CRC Header is also checksummed

    read_magic(buf);
    struct array crc_msg = read_array(buf, 4);
    char _1a = crc_msg.data[0];
    if (_1a != 0x1a) {
        printf("Got 0x%x, wanted 0x1a\n", _1a);
        error("Invalid CRC header");
    }
    short expected_crc = (crc_msg.data[2] << 8) + (crc_msg.data[3] & 0xff);

    short actual_crc = crc16(buf->data, end_of_main_message);
    short actual_rotated = ((actual_crc & 0xff) << 8) + ((actual_crc & 0xff00) >> 8);
    
    if (actual_crc != expected_crc && actual_rotated != expected_crc) {
        printf("crc actual: 0x%0x, swap endian: 0x%04x, expected: 0x%04x\n", actual_crc, actual_rotated, expected_crc);
        error("CRC mismatch");
    } else if (DEBUG) {
        printf("crc actual: 0x%04x, swap endian: 0x%04x, expected: 0x%04x\n", actual_crc, actual_rotated, expected_crc);
    }
}

int read_len(int typ, struct buffer* buf) {
    int len = (typ & 0x0f) - 1;
    if ((typ & 0x80) == 0x80) {
        // "extended length"
        // length: low nibble of typ + low nibble of next byte (-2)
        int ext = buf->data[buf->pos++];
        len = ((typ & 0x0f) << 4) + ((ext & 0x0f) - 2);
        if (DEBUG) {
            printf("%02x+%02x: ", typ, ext);
        }
    } else if (DEBUG) {
        printf("%02x: ", typ);
    }
    return len;
}

// Collect values from previous iteration
char scale = 0;
unsigned char unit = 0;
char obis[6] = {0,0,0,0,0,0};
int value_beginning = 0;
int value_length = 0;

void detect_outputs(int len, struct array* val, long vali, int level, int* level_remaining) {
    if (level_remaining[level] == 2) {
        scale = (char) vali & 0xff;
        if (DEBUG) {
            printf(" <- scale (%hhd)", scale);
        }
    } else if (level_remaining[level] == 3) {
        if (DEBUG) {
            printf(" <- unit");
        }
        unit = vali;
    } else if (level_remaining[level] == 6 && len == 6) {
        if (DEBUG) {
            printf(" <- OBIS");
        }
        memcpy(obis, val->data, len);
    } else if (level_remaining[level] == 1) {
        float scalef = scale;
        double value = 0.0;
        if (scale > 127) {
            // Integer overflow doesn't work??
            // (or chars are 2 bytes)
            scalef = scale - 256;
        }
        if (DEBUG) {
            printf(" scale %f ", scalef);
        }
        value = (double)vali * pow(10, scalef);
        char* unitStr = units[unit];
        if (DEBUG) {
            printf(" <- value = %f %s", value, unitStr);
        }
        for (int out=0; out<output_num; out++) {
            char* output_obi = &output_obis[out*6];
            int cmp = memcmp(obis, output_obi, 6);
            if (cmp == 0) {
                printf(" -> output %d", out);
                printf(" is at offset 0x%04x, len %d\n", value_beginning, value_length);
                setoutput(out, value);
            }
        }
    }
}

void parse(struct buffer* buf) {
    read_headers(buf);
    int level = -1;
    int level_remaining[16] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}; // max recursion level 16
    int level_length[16] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
    if (DEBUG) {
        printf("Headers are correct.\n");
    }
    while (buf->pos < buf->len) {
        // if (DEBUG) {
        //     printf("Parsing... at pos %d / %d\n", buf->pos, buf->len);
        // }
        int typ = buf->data[buf->pos++];
        if (level == -1 && typ == 0x1b) {
            handle_eof(buf);
            return;
        }

        struct array val;
        long vali = 0;

        print_space(level);
        int len = read_len(typ, buf);

        if ((typ & 0x70) == 0x70) { // sub-array
            len +=1;
            level++;
            level_remaining[level] = len;
            level_length[level] = len;
            if (DEBUG) {
                printf("list %d:\n", len);
            }
        } else {
            value_beginning = buf->pos;
            value_length = len;
            val = read_array(buf, len);
            debug_array(&val, 0);

            switch (typ & 0x70) {
            case 0x50: // signed
                vali = to_int(&val);
                if (DEBUG) {
                    if (len == 1) {
                        printf("signed %d: %hhd", len, (char)vali);
                    } else {
                        printf("signed %d: %ld", len, vali);
                    }
                }
                break;
            case 0x60: // unsigned
                vali = to_int(&val);
                if (DEBUG) {
                    printf("unsigned %d: %lu", len, vali);
                }
                break;
            case 0x00: // bytearray
                if (DEBUG) {
                    printf("bytes (%d)", len);
                }
                break;
            default:
                error("Unknown type");
            }
            // detect outputs
            // check if we're on the correct level
            if ((level == 4) && (level_length[level] == 7)) {
                detect_outputs(len, &val, vali, level, level_remaining);
            }
            if (DEBUG) {
                printf("\n");
            }
        }
        while (level >= 0) {
            if (level_remaining[level] <= 0) {
                level--;
            } else {
                level_remaining[level]--;
                break;
            }
        }
    }
}

int skip_headers(struct buffer* buf) {
    // cut HTTP header: look for \r\n\r\n
    for (int offset = 0; offset < buf->len; offset++) {
        if (memcmp(&buf->data[offset], "\r\n\r\n", 4) == 0) {
            return offset+4;
        }
    }
    error("Could not find end of HTTP headers");
    return 0;
}

#define BUFF_SIZE 4096
#define RD_BLOCK_SIZE 128

int run_fetch() {
    if (DEBUG) {
        printf("start.\n");
    }
    char filename[256]; filename[0] = 0;
    strcat(filename, "/dev/tcp/");
    strcat(filename, HTTP_IP);
    strcat(filename, "/80");
    http_stream = stream_create(filename, 0, 0);
    if (DEBUG) {
        printf("stream create.\n");
    }
    char command[256]; command[0] = 0;
    strcat(command, "GET ");
    strcat(command, HTTP_PATH);
    strcat(command, " HTTP/1.0\r\nAuthorization: Basic ");
    strcat(command, http_basicauth);
    strcat(command, "\r\n\r\n");

    int written = stream_write(http_stream, command, strlen(command));
    if (written != strlen(command)) {
        printf("Should have written %ld bytes, wrote %d: ", strlen(command), written);
        error("HTTP request failed");
        return 1;
    }
    stream_flush(http_stream); // flush output buffer
    if (DEBUG) {
        printf("req sent.\n");
    }

    struct buffer buf;
    char read_buffer[BUFF_SIZE];
    buf.pos = 0;
    buf.len = 0;
    buf.data = read_buffer;

    int loop = 1;
    do {
        int read = stream_read(http_stream, &buf.data[buf.len], min(RD_BLOCK_SIZE, BUFF_SIZE - buf.len), READ_TIMEOUT_MS);
        buf.len += read;

        loop = (read > 0);
        loop &= (buf.len < BUFF_SIZE);
    } while (loop);
    if (DEBUG) {
        printf("total bytes: %d\n", buf.len);
    }
    cleanup();
    int offset = skip_headers(&buf);
    buf.data+=offset;
    if (DEBUG) {
        printf("starting parse.\n");
        //debug_bytes(buf.data, buf.len);
    }
    parse(&buf);

    cleanup();
    return 0;
}

int main() {
    http_basicauth = getinputtext(0);

    float repeat_time = getinput(0);
    while (1) {
        printf("running fetch");
        run_fetch();
        printf("done. sleeping %fs", repeat_time);
        sleep((int)(repeat_time * 1000));
    }
}

#ifndef __clang__
// Loxone PicoC is running in script mode
main();
#endif

run.sh

#!/bin/bash

picoc -s reader.c

run_clang.sh

#!/bin/bash

set -euo pipefail

clang -Wall -g reader.c
lldb -o run -- ./a.out
rm -rf a.out*

run_perl.sh

#!/bin/bash
./smldump.pl <(sed '1,/^\r$/d' sml_message.bin)

run_python.sh

#!/bin/bash

sed '1,/^\r$/d' sml_message.bin | ./emh.py

sml_message.bin

smldump.pl

#!/usr/bin/perl
#
# smldump.pl
#
# Dump structure from a binary SML (Smart Message Language) file.
# SML is used in various smart metering systems in germany.
#
# QUICK AND DIRTY HACK STYLE
#
# (C) 2019 Hajo Noerenberg
#
# http://www.noerenberg.de/
# https://github.com/hn/smldump
#
# https://de.wikipedia.org/wiki/Smart_Message_Language
# https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Publikationen/TechnischeRichtlinien/TR03109/TR-03109-1_Anlage_Feinspezifikation_Drahtgebundene_LMN-Schnittstelle_Teilb.pdf?__blob=publicationFile&v=1
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3.0 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/gpl-3.0.txt>.
#

use strict;

my %obis = (
    "\x00\x00\x60\x01\xFF\xFF" => "Seriennummer",
    "\x01\x00\x00\x00\x09\xFF" => "Server-Id / Geraeteeinzelidentifikation",
    "\x01\x00\x01\x08\x00\xFF" => "Zaehlwerk pos. Wirkenergie (Bezug), tariflos",
    "\x01\x00\x01\x08\x01\xFF" => "Zaehlwerk pos. Wirkenergie (Bezug), Tarif 1",
    "\x01\x00\x01\x08\x02\xFF" => "Zaehlwerk pos. Wirkenergie (Bezug), Tarif 2",
    "\x01\x00\x02\x08\x00\xFF" => "Zaehlwerk neg. Wirkenergie (Einspeisung), tariflos",
    "\x01\x00\x02\x08\x01\xFF" => "Zaehlwerk neg. Wirkenergie (Einspeisung), Tarif 1",
    "\x01\x00\x02\x08\x02\xFF" => "Zaehlwerk neg. Wirkenergie (Einspeisung), Tarif 2",
    "\x01\x00\x0F\x07\x00\xFF" => "Betrag der aktuellen Wirkleistung",
    "\x01\x00\x10\x07\x00\xFF" => "Aktuelle Wirkleistung gesamt",
    "\x01\x00\x24\x07\x00\xFF" => "Aktuelle Wirkleistung L1",
    "\x01\x00\x38\x07\x00\xFF" => "Aktuelle Wirkleistung L2",
    "\x01\x00\x4c\x07\x00\xFF" => "Aktuelle Wirkleistung L3",
    "\x81\x81\xC7\x82\x03\xFF" => "Hersteller-Identifikation",
    "\x81\x81\xC7\x82\x05\xFF" => "Public Key",
);

my $buf;

open( SML, "<", $ARGV[0] ) || die( "Unable to open input file: " . $! );
binmode(SML);

read( SML, $buf, 4 );
printf( "%*v2.2X\n", " ", $buf );
die if ( $buf ne "\x1b\x1b\x1b\x1b" );		# Start escape sequence

read( SML, $buf, 4 );
printf( "%*v2.2X\n\n", " ", $buf );
die if ( $buf ne "\x01\x01\x01\x01" );		# SML Version 01

my @level = (0);

while ( read( SML, $buf, 1 ) ) {

    pop(@level) while ( @level && $level[-1] == 0 );
    last if ( !@level && ( $buf eq "\x1B" ) );	# First byte of end escape sequence
    $level[-1]-- if (@level);

    print " " x ( 4 * @level );
    printf( "%02X", ord($buf) );
    my $al = ord( $buf & "\x0F" ) - 1;

    if ( ( $buf & "\x80" ) eq "\x80" ) {       # Extended length
        read( SML, my $albuf, 1 );
        printf( "+%02X", ord($albuf) );
        die("Unsupported") if ( ord( $albuf & "\x70" ) );
        $al = ord( $buf & "\x0F" ) << 4 + ord( $albuf & "\x0F" ) - 2;
    }

    print ": ";

    if ( ( $buf & "\x70" ) eq "\x70" ) {       # List of
        push( @level, ord( $buf & "\x0F" ) );
    }
    elsif ( ( $buf & "\x7F" ) eq "\x00" ) {    # EndOfSmlMSg
        print "\n";
    }
    elsif ( ( $buf & "\x70" ) eq "\x00" ) {    # Octet String
        read( SML, $buf, $al );
        printf( "%*v2.2X", " ", $buf );
        print "\t# OBIS " . $obis{$buf} if ( $obis{$buf} );
    }
    elsif ( ( $buf & "\x70" ) eq "\x50" ) {    # IntegerX
        read( SML, $buf, $al );
        printf( "%*v2.2X", " ", $buf );
    }
    elsif ( ( $buf & "\x70" ) eq "\x60" ) {    # UnsignedX
        read( SML, $buf, $al );
        printf( "%*v2.2X", " ", $buf );
    }
    else {
        die("Unsupported");
    }

    print "\n";
}

printf( "%*v2.2X ", " ", $buf );
read( SML, $buf, 3 );
printf( "%*v2.2X\n", " ", $buf );
die if ( $buf ne "\x1b\x1b\x1b" );		# last three bytes of end escape sequence

read( SML, $buf, 4 );
printf( "%*v2.2X\n", " ", $buf );		# CRC-16/X-25

close(SML);

test.sh

#!/bin/bash
set -euo pipefail

expected='SETTING OUTPUT 1 to 1521930.900000
SETTING OUTPUT 2 to 0.000000
SETTING OUTPUT 0 to 100.600000'

set +e
actual="$(./run.sh)"
ret=$?
set -e

if difference="$(diff -u <(echo "${expected}") <(grep "SETTING OUTPUT" <<< "${actual}"))"; then
    echo "OK"
else
    cat <<EOF
--- Full output: ---
${actual}
--- diff: ---
${difference}
FAIL
EOF
    exit 1
fi

if [[ "${ret}" != 0 ]]; then
    echo "run.sh exited with ${ret}"
    exit 1
fi