UDP sender and receiver in CircuitPython

udp_recv_code.py

# udp_recv_code.py -- receive UDP messages from any receiver, can be another CircuitPython device
# 24 Aug 2022 - @todbot / Tod Kurt
# cribbing from code at https://github.com/adafruit/circuitpython/blob/main/tests/circuitpython-manual/socketpool/datagram/ntp.py

import time, wifi, socketpool
from secrets import secrets
print("Connecting to WiFi...")
wifi.radio.connect(ssid=secrets['ssid'],password=secrets['password'])
print("my IP addr:", wifi.radio.ipv4_address)
pool = socketpool.SocketPool(wifi.radio)

udp_host = str(wifi.radio.ipv4_address) # my LAN IP as a string
udp_port = 5005             # a number of your choosing, should be 1024-65000
udp_buffer = bytearray(64)  # stores our incoming packet

sock = pool.socket(pool.AF_INET, pool.SOCK_DGRAM) # UDP socket
sock.bind((udp_host, udp_port))  # say we want to listen on this host,port

print("waiting for packets on",udp_host, udp_port)
while True:
    size, addr = sock.recvfrom_into(udp_buffer)
    msg = udp_buffer.decode('utf-8')  # assume a string, so convert from bytearray
    print(f"Received message from {addr[0]}:", msg)

udp_send_code.py

# udp_send_code.py -- send UDP messages to specified receiver, can be another CircuitPython device
# 24 Aug 2022 - @todbot / Tod Kurt
# cribbing from code at https://github.com/adafruit/circuitpython/blob/main/tests/circuitpython-manual/socketpool/datagram/ntp.py

import time, wifi, socketpool
from secrets import secrets
print("Connecting to WiFi...")
wifi.radio.connect(ssid=secrets['ssid'],password=secrets['password'])
print("my IP addr:", wifi.radio.ipv4_address)
pool = socketpool.SocketPool(wifi.radio)

udp_host = "192.168.1.123"  # LAN IP of UDP receiver
udp_port = 5005             # must match receiver!

my_message = "hi there from CircuitPython!"

sock = pool.socket(pool.AF_INET, pool.SOCK_DGRAM) # UDP, and we'l reuse it each time

num = 0
while True:
    # stick a nubmer on the end of the message to show progress and conver to bytearray
    udp_message = bytes(f"{my_message} {num}", 'utf-8')
    num += 1
    print(f"Sending to {udp_host}:{udp_port} message:", udp_message)
    sock.sendto(udp_message, (udp_host,udp_port) )  # send UDP packet to udp_host:port
    time.sleep(1)

To send or receive from Linux / MacOS, you can use netcat (nc) with commands like:

• Sending UDP to a device:

echo "hello from my Mac"  | nc -vv -c -u 192.168.1.123 5005

• Receiving UDP from a device:

nc -vv -u -l -p 5005

Note: if on MacOS, use brew install netcat to install proper GNU netcat and not the Apple-specific one. (and even then it seems to not work, the Python code below does work though)

In desktop Python that runs on your Mac, Linux, Windows box, you can send with code like this:

# https://lan.developer.lifx.com/docs/packet-contents
import time, socket

udp_host = "192.168.1.123"  # LAN IP of receiver
udp_port = 5005
message = "hi there from Python!"

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP

num = 0
while True:
    udp_message = bytes( f"{message} {num}", 'utf-8')
    num += 1
    print(time.monotonic(), f"sending to {udp_host}:{udp_port} message:", udp_message)
    sock.sendto(udp_message, (udp_host,udp_port) )
    time.sleep(1)

And to receive UDP on desktop Python, that looks like:

import time, socket

udp_host = "192.168.42.133" # my LAN IP
udp_port = 5005

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP

sock.bind((udp_host,udp_port))

print("waiting for packets on",udp_host,udp_port)
while True:
     data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
     print("received message: %s" % data)

Hi todbot. Thank you for your examples. I am porting to CircuitPython on an Adafruit Feather ESP32-S2 with TFT a script that ran OK (some years ago) on Python using a Raspberry Pi. My script hangs on the line with the sock.recvfrom() command. See my post on: Adafruit Forums. Any idea how I can get the script running?

Have you verified you're actually sending UDP packets currently?

I've not used recv_into() but I think all the receive functions will block if it's not received all its bytes and when setblocking() and/or settimeout() have been set.

What's you're entire code, for both the sender and the receiver?

Another source of examples that may be useful: https://github.com/anecdata/Socket/tree/main/examples

Thank you very much for your rapid replies. My script is intended for just receiving broadcasted UDP datagrams, sent by X-Plane 12 flight simulator. I have no 'Sender' part. I will go to experiment with setblocking() and settimeout(). I did not use them until now in this (ported) script.
I come back a.s.a.p.

Here is the part of the script that is 'hanging':

     # Function created by Charlylima. Modified by @PaulsPt
     def FindIp(self):
        global pool
        
        # Find the IP of XPlane Host in the Local Area Network.
        # It takes the first one it can find.
        
        TAG = tag_adjust("xp.FindIp(): ")
        self.BeaconData = {}

        # open socket for multicast group.

        try:
            if pool is None:
                print(TAG+'pool is None. Going to create it', file=sys.stderr)
                pool = make_pool()
            if pool is not None:
                if not my_debug:
                    print(TAG+'type(pool)= {}'.format(type(pool)), file=sys.stderr)
            else: 
                raise ValueError(f"pool must be not None. Got {pool}")
            
            self.my_DataRef_sock = pool.socket(pool.AF_INET, pool.SOCK_DGRAM) # SocketPool has no attribute IPPROTO_UDP !!!
            self.my_DataRef_sock.settimeout(5)
            self.my_DataRef_sock.setblocking(True)
            
            if not my_debug:
                print(TAG+'type(self.my_DataRef_sock)= {}'.format(type(self.my_DataRef_sock)), file=sys.stderr)
                
            #self.my_DataRef_sock.setsockopt(pool.SOL_SOCKET, pool.SO_REUSEADDR, 1)
            self.my_DataRef_sock.bind((self.MCAST_GRP, self.MCAST_PORT))
            
            #self.my_DataRef_sock.connect((self.MCAST_GRP, self.MCAST_PORT))
            
            if not my_debug:
                print(TAG+'type(self.BeaconData)= {}'.format(type(self.BeaconData)), file=sys.stderr)
                print(TAG+'self.BeaconData= {}'.format(self.BeaconData), file=sys.stderr)
        except ValueError as e:
            print(TAG+'Error: {}'.format(e), file=sys.stderr)
            raise
        except Exception as e:
            print(TAG+'Error: {}'.format(e), file=sys.stderr)
            raise
        
        # frame_fmt = "4sl"
        packet_size = 61 # dec 61 = hex 0x3D -- dec 181 = hex 0xB5      struct.calcsize(frame_fmt)
        print(TAG+'packet_size= {}'.format(packet_size), file=sys.stderr)
        packet = bytearray(packet_size)  # stores our incoming packet
        
        if not my_debug:
            # print(TAG+'packet= {}'.format(packet), file=sys.stderr)
            print(TAG+'waiting for beacon packets, group {}, port {}'.format(self.MCAST_GRP, self.MCAST_PORT), file=sys.stderr)
        
        le_BeaconData = len(self.BeaconData)
        
        while le_BeaconData == 0:
            # receive data
            try:
                # From Wireshark capture:
                # Frame 499: 61 bytes on wire (488 bits), 61 bytes captured (488 bits) on interface \Device\NPF_Loopback, id 0 Null/Loopback

                print(TAG+'we passed here. Line {}'.format(269), file=sys.stderr)

                #size = self.my_DataRef_sock.recv_into(packet)
                
                size, addr = self.my_DataRef_sock.recvfrom_into(packet)  # <<<=== HERE THE SCRIPT HANGS !

                print(TAG+'nr bytes received= {}'.format(size), file=sys.stderr)
                
                print(TAG+'Received packet (raw) {}'.format(packet), file=sys.stderr)
                # msg = packet.decode('utf-8')  # assume a string, so convert from bytearray
                print(TAG+'Received packet from {}, packet {}, size {}'.format(addr[0], packet, size), file=sys.stderr)

                #packet, sender = self.my_DataRef_sock.recvfrom(packet_size)  # was (15000)
                #print(TAG+'packet= \'{}\', sender= {}'.format(packet, sender[0]), file=sys.stderr)
                # decode data
                # * Header
                
                # Strip unused bytes:
                msg = packet[32:] # slice off bytes 0 - 31. If packet_size = 61 then the sliced size will be 29 (0x3D - 0x20 = 0x1D = dec 21 )
                header = msg[0:4]
                data = msg[5:]
                
                if not my_debug:
                    print(TAG+'Entering...', file=sys.stderr)
                    print('packet header = {}'.format(header), file=sys.stderr)
                    #print('packet received = {}'.format(packet), file=sys.stderr)
                    print('packet data part = {}'.format(data), file=sys.stderr)

                if header == b'DATA': # 2 lines added by Paulsk. The DATA packet we handle in the XPlaneUdpDatagram Class object.
                    pass
                elif header == b'BECN':
                    # blink the LED
                    blink()
                    # * Data
                    # data = msg[5:]  # was: packet[5:21]
                    # struct becn_struct
                    # {
                    # 	uchar beacon_major_version;		// 1 at the time of X-Plane 10.40
                    # 	uchar beacon_minor_version;		// 1 at the time of X-Plane 10.40
                    # 	xint application_host_id;		// 1 for X-Plane, 2 for PlaneMaker
                    # 	xint version_number;			// 104014 for X-Plane 10.40b14 - 113201 for X-Plane 11.32
                    # 	uint role;				        // 1 for master, 2 for extern visual, 3 for IOS
                    # 	ushort port;				    // port number X-Plane is listening on
                    # 	xchr	computer_name[strDIM];  // the hostname of the computer
                    # };
                    beacon_major_version = 0
                    beacon_minor_version = 0
                    application_host_id = 0
                    xplane_version_number = 0
                    role = 0
                    port = 0
                    (
                      beacon_major_version,  # 1 at the time of X-Plane 10.40
                      beacon_minor_version,  # 1 at the time of X-Plane 10.40, 2 at the time of X-Plane 11
                      application_host_id,   # 1 for X-Plane, 2 for PlaneMaker
                      xplane_version_number, # 104014 for X-Plane 10.40b14 - 113201 for X-Plane 11.32
                      role,                  # 1 for master, 2 for extern visual, 3 for IOS
                      port,                  # port number X-Plane is listening on
                    ) = struct.unpack("<BBiiIH", data)


                    if my_debug:
                        print('beacon_major_version = {}'.format(beacon_major_version), file=sys.stderr)
                        print('beacon_minor_version = {}'.format(beacon_minor_version), file=sys.stderr)
                        print('application_host_id = {}'.format(application_host_id), file=sys.stderr)

                    # Originally beacon_minor_version was checked for a value of 1 but investigation by Paulsk revealed that X-Plane 11 returns a value of  2
                    computer_name = packet[21:-1]   # packet[21:-1]
                    if beacon_major_version == 1 \
                       and beacon_minor_version == 2 \
                       and application_host_id == 1:
                        self.BeaconData["IP"] = sender[0]
                        self.BeaconData["Port"] = port
                        self.BeaconData["hostname"] = computer_name.decode()
                        self.BeaconData["XPlaneVersion"] = xplane_version_number
                        self.BeaconData["role"] = role

                        if not my_debug:
                            print('\n'+TAG+'-- Beacon UDP packet received:', file=sys.stderr)
                            print('Host IP         = {}'.format(self.BeaconData["IP"]), file=sys.stderr)
                            print('Port            = {}'.format(self.BeaconData["Port"]), file=sys.stderr)
                            print('Hostname        = {}'.format(self.BeaconData["hostname"]), file=sys.stderr)
                            print('X-Plane version = {}'.format(self.BeaconData["XPlaneVersion"]), file=sys.stderr)
                            print('Role            = {}'.format(self.BeaconData["role"]), file=sys.stderr)
                    
                    le_BeaconData = len(self.BeaconData)

                else:
                    print(TAG+'-- Unknown packet from {]'.format(sender[0]), file=sys.stderr)
                    print('{} bytes'.format(str(len(packet))), file=sys.stderr)
                    print(packet, file=sys.stderr)
                    print(binascii.hexlify(packet), file=sys.stderr)

            except self.my_DataRef_sock.timeout:
                print(TAG+'UDP rx socket timed out', file=sys.stderr)
                raise XPlaneIpNotFound()
            except OSError as e:
                if e.errno == 11:
                    print(TAG+'Resource temporarily unavailable (EAGAIN)', file=sys.stderr)
                else:
                    print(TAG+'OSError {}'.format(e), file=sys.stderr) # [Errno 11] EAGAIN
            except Exception as e:
                print(TAG+'Error: {}'.format(e), file=sys.stderr)
                raise
            
        self.my_DataRef_sock.setblocking(False)
        self.my_DataRef_sock.close()
        return self.BeaconData
`

That's quite a lot of complicated code. Since your problem appears very low level, I would recommend starting with a very simple program that just receives the packets from the sender, to verify the sender is actually sending packets (i.e. verify there's no firewall or router config that's preventing multicast from getting to your device)

OK, I'll will use a simple script to test. I already set port triggers in the router of my ISP. In the MS Windows 11 desktop PC running the X-Plane flight simulator I created UDP input/output rules for the ports needed in the Defender Firewall.

I used your receiver example (only modified to use settings.toml instead of the former secrets.py.
Script used:

# udp_recv_code.py -- receive UDP messages from any receiver, can be another CircuitPython device
# 24 Aug 2022 - @todbot / Tod Kurt
# cribbing from code at https://github.com/adafruit/circuitpython/blob/main/tests/circuitpython-manual/socketpool/datagram/ntp.py

import time, wifi, socketpool, os
print("Connecting to WiFi...")
wifi.radio.connect(ssid=os.getenv("CIRCUITPY_WIFI_SSID"), password=os.getenv("CIRCUITPY_WIFI_PASSWORD"))
print("my IP addr:", wifi.radio.ipv4_address)
pool = socketpool.SocketPool(wifi.radio)

# I, @PaulskPt, used for udp_host erroneously: os.getenv("MULTICAST_GROUP")
udp_host = str(wifi.radio.ipv4_address) # my LAN IP as a string  
udp_port = int(os.getenv("MULTICAST_PORT"))  # a number of your choosing, should be 1024-65000
udp_buffer = bytearray(64)  # stores our incoming packet

sock = pool.socket(pool.AF_INET, pool.SOCK_DGRAM) # UDP socket
sock.bind((udp_host, udp_port))  # say we want to listen on this host,port

print("waiting for packets on",udp_host, udp_port)
while True:
    size, addr = sock.recvfrom_into(udp_buffer)
    msg = udp_buffer.decode('utf-8')  # assume a string, so convert from bytearray
    print(f"Received message from {addr[0]}:", msg)

Resulting in the following output:

Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XGPS2,-85.700840,38.354860,156.1102,147.9949,0.0001,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0
Received message from 192.168.1.96: XATT2,148.0,5.9,0.5,-0.0000,0.0000,-0.0000,0.0,-0.0,-0.0,-0.01,0

Btw: the data is from a situation when the airplane was on the ground, parked. Engine running.

There is no packet reception when I use port 49707 or 49003 instead of port 49002.
Port 49707 is used for UDP datagrams (BECN and DATA packets) to Multicast group '169.255.1.1'. The DATA packets contain
data elements that I selected inside the X-Plane 12 setup

Update:
I receive the DATA packets when I enter in X-Plane settings for 'General Data Output' IP Address, the IP address: 192.168.1.110, which is the IP Address of the CPY device running the script, and Port 49707. But I get a 'UnicodeError' because the received UDP packets are packed.
When I modify the line 'msg = udp_buffer.decode('utf-8')' into: 'msg = udp.buffer' then the following output is received (only one packet copied below):

Received message from 192.168.1.96: bytearray(b'DATA*\x03\x00\x00\x00\x00\x00\x00\x00:\xb9\x989\xa9\xdf\x999\xd5E\x979\x00\xc0y\xc4\x00\x00\x00\x00\x1d\x13\xb19\xe5\x14\xae9\x11\x00\x00\x00e\xc6\xbc@8\xb5\t?\x1e\xff\x13C\x00\xc0y\xc4\xcb\x10\x19')

So, I think my problem is solved!
Again thank you for your rapid responses!

Thanks, this is very helpful!

By the way, line 22 of udp_recv_code.py

msg = udp_buffer.decode('utf-8')

should be

msg = udp_buffer[:size].decode('utf-8')

or otherwise you might print unwanted things:)

@aceg00 thank you for your response and advice.

@todbot You should consider updating this with the above correction from @aceg00 . The current code is a real foot gun. Many thanks for this gist nonetheless though!