Skip to content

Instantly share code, notes, and snippets.

@elsirion
Last active Jan 3, 2023
Embed
What would you like to do?
Core Lightning Rust Pay Plugin Bounty

Alternative Core Lightning pay plugin

tl;dr 6M sat bounty for an alternative Core Lightning pay plugin written in Rust using LDK.

The problem

I run a CLN node that I mainly want to use for my private payments. Unfortunately CLN's routing algorithm is not as intelligent/optimal in practice as it could be imo.

In my opinion the problems are:

  • Excessive splitting: Payments that could be done in one go are split from the beginning, leading to
  • Frequent temporarily stuck payments: They aren't really stuck, but take a few seconds, blocking further route finding attempts. My theory is that excessive splitting increases the probability to have a slow node in one of the paths and we need to way for it to undo the HTLC.
  • No memory of learned balances from previous payments

I could try to fix these in CLN, but I'm not too eager to brush up on my C and it might just get me stuck in a slightly better local optimum.

Towards a solution

To fix these problems I'd like to have an easily hackable, alternative pay plugin. It should be written in Rust and based on LDK, which seems to be the most versatile lightning library right now and will allow rapid iteration on routing algorithms. It's recommended to work together with the CLN project to improve the new Rust plugin crate along the way. The core feature set of the pay plugin should be:

  • Pickhardt payments or similarly optimized split payments, but with configurable cost for splitting, i.e. using 2 instead of 1 channel has a higher cost
  • Local ranking of nodes: I want to be able to express preference for certain nodes which I know are professionally maintained. Probing of ping times should also be used to build the overall rating to prefer low-latency network connections and fix stuck payments.
  • Local channel ranking/balance estimation: Channel balance bounds learned from previous routing attempts and possibly some light probing in the background should be used to improve route finding. This information needs to decay/learned bounds needs to be relaxed over time, reflecting the possibility that other ayments have changed the channel balances.

I imagine this to only be the starting point for more experimentation with better routing algorithms and hope that some of the work will be upstreamed to the rust-bitcoin and LDK open source ecosystem.

Bounty

So far 6 million satoshi have been pledged by:

The bounty can be redeemed by the first person or group that builds an open source CLN pay plugin with the features described above. The bounty expires 2023-12-31.

If anyone else wants to see this happen, please reach out and I will add your pledge here. For now this is a honour-bound system, but I'm willing to set up a multi sig with reputable community members (especially if more people want to contribute).

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I, elsirion, will pay 5,000,000sat to the first person or group 
that builds an open source CLN pay plugin with the features 
described above. The bounty expires 2023-12-31.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEs83/b21LK+nqiwAgswBeVxqjFNoFAmOsWyQACgkQswBeVxqj
FNrw/A//QL5vGBU8xTMvgRgAQD4cuGV1aqSL1YnOncobFWoEKr7mEz7O5T/CGrmP
bNVh82TDgaJBVfEsivRo9sOZZqCnQNI9lVQoyYKdsBlTu7ezEwAOp7sw7Aqq8Qmu
vkWjR9uK/x9qfNpDn3dEDog/nd/P8Ifo/cScwHlYCeDNJu+bNsPJS5GORpr3yTp2
YZRx6Tfmd35nhlM7wHS0kDENBsrbQDe+5aLF/2irHXZwAOQG+/Iz9MdyJAqklGk3
utIy+JF47fGmqNHHzoLStmLhBhvPU52jo/F29KUdXab4lfQ1xie5f7KyMvF9WR6+
D6WwHoEfsXajAxk6XV37GpmEnu4iXh1ScIYJNuY78CELcOyxjlAMHTZMzT48H7/7
XgwfkgyVGMK6DWCFT64Wd+QOtVo5huWJ5f43mAQjzKuJNbX+WrcGNahV2jvbXgga
croLv9qke6MFQTv010wt+037EbcL32a06XlSDYXcruDo7XF4V7z/fGu9TcBi/ThL
1Oo9FrHfIeSgxdRDakOJUSKISH6P2g8P7KPFVdOwOkGMYwAl4nQYzODLsW/oTSnI
+L92A4edK4vCNntVZoV3jg0r8pXbT4j+dFLEMPAYOVGTDTfQbA6PU9K7UTI9km/t
BxZgH8wsYJnXAo8hjyJ4ws3RI1lKxEQonJRxTPHroc+sgCnnXyc=
=tau8
-----END PGP SIGNATURE-----
@fiatjaf
Copy link

fiatjaf commented Dec 30, 2022

Can I add a million sats there to the bounty?

@elsirion
Copy link
Author

elsirion commented Dec 30, 2022

With pleasure! Will edit later and also post on NVK's bounty page.

@delta1
Copy link

delta1 commented Dec 30, 2022

I’m quite keen to dig into this, will post an update after this silly season

@elsirion
Copy link
Author

@fiatjaf Thx again! Updated 🎉

@evd0kim
Copy link

evd0kim commented Jan 2, 2023

Is using LDK really strong requirement? I don't see it really fitting into CLN plugin. The only common thing there is Rust.

@elsirion
Copy link
Author

elsirion commented Jan 2, 2023

I don't see it [LDK] really fitting into CLN plugin

@evd0kim It's mostly about not building multiple incompatible router/node scoring/channel scoring/network topology implementations in rust, LDK seems like a good shelling point for a common API so that implementations can be shared and reused across the ecosystem. https://twitter.com/EricSirion/status/1609176230816845825

If LDK's goals turn out too incompatible I'd be open to dropping it, but after talking to @moneyball and looking at their docs I'm quite confident LDK makes sense here.

@evd0kim
Copy link

evd0kim commented Jan 2, 2023

@elsirion well, ok. Let's see if anybody picks it up. After looking into reference Pickhardt payments implementation and LDK code, while knowing CLN plugin API I am not confident it is real. Although some sort of frankenstein is always possible. There is even a combination of Scala Immortan and CLN for Hosted channels protocol.

@moneyball
Copy link

@evd0kim LDK already implements most of what is described here, so I think most of the work is just doing the integration. I'm looking forward to seeing how easy/hard it is to integrate.

@moneyball
Copy link

@evd0kim what are your concerns about it not being real / not working?

I asked LDK devs and they think it should work. It might require a few changes to LDK but should be simple fixes that we'd happily do right away.

@evd0kim
Copy link

evd0kim commented Jan 2, 2023

Inner workings of lightningd aren't accessible for plugins. For that reason there are only two ways to read CLN graph either by requesting via RPC or reading serialized file from disk.

I see from Pikhardt reference code that the plugin must be using graph anyway so plugin will tax node with RAM and CPU. RPC request will be more expensive and stupid.

I don't see it reasonable to wait or ask for changes in LDK for the sake of unification with CLN it just looks strange to me I am sure nobody will really will keep this integration in shape. On the other hand looking for specialized graph library in Rust for this task and for implementation as CLN plugin may lead to useful outcomes like building other plugins using graph related heuristics.

@moneyball
Copy link

Graph data can come from RGS? https://lightningdevkit.org/blog/announcing-rapid-gossip-sync/

No need to worry about any changes needed for LDK. We're happy to make them as long as they make sense.

@evd0kim
Copy link

evd0kim commented Jan 2, 2023

Yes, they can apparently. I am confused pretty much though because it looks like separate protocol. However, my point is that bounty is quite specific I will just wait for somebody to implement it so I will know something new about LDK and Rust.

@elsirion
Copy link
Author

elsirion commented Jan 3, 2023

plugin will tax node with RAM and CPU. RPC request will be more expensive and stupid.

Somehow I don't really think this is a huge problem, just not elegant. Even Raspberry Pis come with a lot of memory these days and if you only sync the graph via RPC every minute or so (or using a subscription to gossip events) that should not be too heavy on the node.

Alternatively RGS would work too, although it adds an external dependency afaik, ideally the plugin could run standalone imo.

@fiatjaf
Copy link

fiatjaf commented Jan 3, 2023

Reading data from CLN's gossip.store is probably the sanest way to do this. That's what I assumed would be done.

Ideally this should have a clean separation between the data collection and route calculation / payment attempting parts, because then it would make it easy and magical to add data from other sources that aren't the public gossip (I'm thinking about Public Hosted Channels, Mercury Statechannels and other things that are not acceptable in the public gossip).

@JefriNewbie-hash
Copy link

anyone picks it up. Having looked at the Pickhardt reference payment implementation and the LDK code, I'm not sure if it's true, even though I know the CLN plugin API. Although some sort of Frankenstein is always a possibility. The Hosted Channels protocol even has a combination of Scala Immortan and CLN.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment