ldk_networking_proxy.md

Horizontally scalable networking layer with LDK

I am investigating the best way to implement a stateless networking layer daemon for an LDK-based node. It seems at a high-level the two main options are to either do it at the byte level or event/msg level. The byte level has the benefit mentioned in the docs that it does not require the node to trust the networking daemon. The event/msg level solution has the benefit that the decryption and deserialization of the raw bytes can happen on the scalable proxy, further reducing the load on the singleton node.

For the purposes of this doc I am only focused on the event/msg level solution because I am trying to off-load as much work as possible to a daemon that can be scaled horizontally.

LDK Networking Proxy Daemon

• Runs a standard PeerManager with lightning-net-tokio.
• The MessageHandler's forward all msg's to the remote node via rpc for handling.
• The node maintains a map of which proxy each peer is connected to.
• The node has standard MessageHandler's that get called in response to rpc from the proxies.

This works fine for bytes inbound to the node but I'm trying to find the best way to get the bytes out.

Outbound Byte Options

1) Event/Msgs-Only with Peer Filtering

The proxy MessageHandler's get_and_clear_pending_msg_events remotely calls the node to get pending message send events.

The challenge with this approach is that the current implementation of MessageSendEventsProvider clears and returns events for all peers. Would need an approach that filters these events for the peers connected to the proxy that is calling into get_and_clear_pending_msg_events. The best solution requires either updating the current trait method to support an optional filter or more likely adding a second method that can be used when filtering by a set of peers is required.

The benefits of this solution is that it requires no changes to lightning-net-tokio and the node does not require a PeerManager at all. It's definitely the most straightforward solution assuming this filtering would be accepted by LDK.

2) Raw-Bytes with Process Events

The proxy MessageHandler's get_and_clear_pending_msg_events always just returns an empty Vec and the process_events call that happens on the proxy's PeerManager never actually processes any events. I can still use lightning-net-tokio as-is for the proxy PeerManager.

I will need to use a PeerManager on the node with a SocketDescriptor implementation that can make remote rpc calls to the correct proxy for send_data and socket_disconnected. The proxy can forward those calls to the actual peer connection.

One open question with this approach is when do I call process_events on the node's PeerManager.

2a) It would be nice if there was a PeerManager trait or at a minimum some kind of "ProcessEventsHandler" trait/callback that lightning-net-tokio could be updated to use so that the proxy could provide a wrapped PeerManager and forward all calls to process_events to the node.

2b) If implementing such a trait to be used by lightning-net-tokio is not something that would be accepted by LDK, I could alternatively just call process_events on the node whenever there is a remote call from the proxy to handle any events and/or on a timer. Doesn't feel as clean but would avoid needing any changes to LDK.

The benefit of this solution is that in theory it can be implemented without any changes to LDK but comes with some risk from potential small changes to the overall timing and flow of when process_events is called.

Questions to LDK

1. Am I missing a reasonable option?
2. Would adding a way to filter cleared pending msgs by peer to the MessageSendEventsProvider trait be a change LDK would be open to?
3. If no to #2, would adding a new "PeerManager" trait or at a minimum some kind of "ProcessEventsHandler" trait/callback for lightning-net-tokio be an acceptable change?
4. Would calling process_events on the singleton node's PeerManager after every received msg be a reasonable and safe thing to do? Is there a better strategy if I go this route?