24 May 2021 #lightning-dev

24-may-2021-lightning-dev.txt

<niftynei> hello hello
<niftynei> for all the early birds who want to get a look at today's agenda https://github.com/lightningnetwork/lightning-rfc/issues/874
* rusty (~rusty@pdpc/supporter/bronze/rusty) has joined
<BlueMatt> here today or libera?
<BlueMatt> seems like everything's moving
* ariard__ (~ariard@167.99.46.220) has joined
* ariard__ (~ariard@167.99.46.220) has left
<niftynei> probably best to use a deprecation plan?
* orccoin has quit (Remote host closed the connection)
<niftynei> e.g. email to list announcing move 
<BlueMatt> :shrug: thats what channel topics are for
<BlueMatt> also we could do the usual of just kicking everyone from here lol
<rusty> There are still many more people here than in the libera.chat channel though.  I had intended to at least warn ppl that if this goes away, we'll be there...
<BlueMatt> but i think we should at least get a logger set up on libera
* orccoin (~Rheanna@101.91.180.110) has joined
<rusty> BlueMatt: indeed.  Until then we remain here, where the meetbot is :)
<BlueMatt> heh, fair
<BlueMatt> who does the logging here anyway?
<ariard> let's do it here for this week and libera next time?
<BlueMatt> kanzure: or someone?
<BlueMatt> anyway, meeting time?
<rusty> BlueMatt: aj does the logging.
<BlueMatt> ah.....aj jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
<rusty> Who's chairing this week?
<roasbeef> is the libera thing even happening? never found a easily readable summary about what's even going on 
<roasbeef> everything is also still on freenode afaict 
<rusty> roasbeef: grab your nick now, in case Freenode stops working though :)
<roasbeef> do we have any reason to believe it
<roasbeef> 'll actually stop working? 
<BlueMatt> roasbeef: yea, I think its more of a "freenode has been sketchy af for years, and we should have moved then, but now is a good excuse" thing
* jkczyz waves
<BlueMatt> and, yes, freenode ownership is absolutely maximally sketch
<BlueMatt> but I assume it'll keep running
<roasbeef> BlueMatt: what's diff from now vs when the same thign happened like 2 yrs ago? idk I skimmed stuff and it just seems liek personal beef, why should we all be displaced because of that? 
<BlueMatt> (and highly filled with anti-bitcoiners)
<rusty> roasbeef: well, given all their tech staff seems to have left, I suspect they're one failure away from total fail.
<jkczyz> i've been getting a ton of spam on freenode since the switch
<BlueMatt> roasbeef: apparently something about them actually trying to impose control vs previously they'd just owned it, but personally I've wanted to move since the acquisition 2 years ago :p
<roasbeef> rusty: iirc some of them had like resignation notices "leak" but they weren't actually leaving? from my pov I still don't really understand the story 
<roasbeef>  ¯\_(ツ)_/¯
<BlueMatt> I mean either way, freenode is sketch af. the libera thing seems also kinda sketch, but I've never been against oftc either lol
* ariard_ has quit (Quit: Lost terminal)
<devrandom> IRC UX is a pain... how about matrix?
<rusty> OK, let's get this bikeshed started!
<rusty> #startmeeting 
<lndev-bot> rusty: Error: A meeting name is required, e.g., '#startmeeting Marketing Committee'
<rusty> #startmeeting https://github.com/lightningnetwork/lightning-rfc/issues/874
<lndev-bot> Meeting started Mon May 24 20:12:12 2021 UTC and is due to finish in 60 minutes.  The chair is rusty. Information about MeetBot at http://wiki.debian.org/MeetBot.
<lndev-bot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
<lndev-bot> The meeting name has been set to 'https___github_com_lightningnetwork_lightning_rfc_issues_874'
<rusty> #topic https://github.com/lightningnetwork/lightning-rfc/pull/872
<rusty> Ouch, roasbeef I hadn't seen this before.
<roasbeef> rusty: yeh something we ran into a while ago, but didn't know what was up, then the launch of our new node analysis thingy ended up triggering it w/ the ACINQ node, and we dug in exchanging error info till we found it 
* rusty wonders what c-lightning does, but fortunately with those test vectors I know I'm going to find out today.
<roasbeef> mpp kinda causes it to occur more often since the payhash is always re-used 
<roasbeef> and if you split in half, then the amts may be the same more often 
<roasbeef> rusty: I checked and iirc CL does msat 
<roasbeef> assuming this is where the sorting happens: https://github.com/ElementsProject/lightning/blob/ade10e7fc4dacbb9d635b05152c7dc38c0896ce7/common/permute_tx.c#L138 
<rusty> roasbeef: yeah, that would be the "obvious" approach. And wrong ofc.
<roasbeef> err does sat* 
<BlueMatt> I believe we do sat
<rusty> I mean.  Of course we do it correctly! Yeah!  Go us!  (Meh, only because we post-sort the tx after construction, so bullet dodged)
<roasbeef> yeh small fix which is nice, hard to know how many force closes over the past few years have happened because of this 
<roasbeef> acinq has a fix up on their end 
<rusty> BlueMatt: I thought there was a proposal to spell out and not just refer to BIP 69.  Maybe it got lost in the backlog?
<roasbeef> which has been merged 
<BlueMatt> rusty: I dunno, I've complained about that a few times, but I think I've always just been loud and annoying and never done the work to fix it
<midnight> fwiw, I have a reliable report of the new owner who has now direct control over the infrastructure, breaking into an email account and a slack server. Additionally, pointers in /topic which state the channel as moved to libera, Lee is directly using as justification for channel takeover. New staff who have demonstrated fairly extreme hostility (even for our sensibilities) are now invading/joining 
<midnight> channels, and Freenode infrastructure and services are being replaced. Why are they being replaced? Who knows.
<rusty> OK, I do not know how to ack #872 hard enough.  Would like to test the vectors though: assuming they pass, everyone happy with applying?
<BlueMatt> if we're fixing it for the 3rd time now I'd kinda prefer we finally actually drop the BIP 69 reference and be self-contained, but, yea, looks good.
<roasbeef> rusty: added some test vectors from pierre to include the msat rounding 
<midnight> Apologies, I didn't see the meeting start.
<rusty> roasbeef: yeah, I just wanna incorporate them in the c-lightning vector tests.  Should only take me 10 minutes.
<BlueMatt> thanks for context either way, midnight 
<midnight> \o
<rusty> #action BlueMatt to create PR to excise BIP-69 references in favor of explicit ordering requirements.
<rusty> OK, so seems like we have consensus?  Any objections to applying #872?
<kanzure> BlueMatt: i do logging
<kanzure> at least the one in the /topic
<BlueMatt> kanzure: plz2log libera
<kanzure> just let me know where to point it to
<BlueMatt> rusty: let me just suggest a different text for roasbeef on 872. we all know how long trivial prs lie around....
<kanzure> will the channel be moving entirely or should i be logging both places?
<kanzure> and if so, what is the new channel name?
<rusty> BlueMatt: not if we pre-approve the open-coding now?  I'd rather agree to merge 872 (pending final test vector check), then pre-approve an open-coded BIP-69 replace assuming two of us sign off.
<roasbeef> BlueMatt: which would be to essentially include the BIP 69 text? 
<BlueMatt> roasbeef: honestly, bip 69 is, itself, underspecified
<BlueMatt> rusty: eh, whatever.
<ariard> kanzure: i believe it's still lightning-dev
<roasbeef> in this context, only becauase we added this tie-breaker, where else is it underspecified? 
<rusty> roasbeef: I believe the shorter-wins tiebreak is also unspecified in BIP-69.
<roasbeef> gotcha, if so ofc it can be updated to make it more fully specified 
<rusty> i.e. "ab" < "abc".  Which is common convention, but still needs to be spelled out.
<rusty> It's pretty trivial tho, so I think it comes under "spelling rule", esp if we've agreed here to it?
<BlueMatt> anyway, either https://github.com/lightningnetwork/lightning-rfc/pull/872/files#r638250523 or I'll open a new pr with that text.
<BlueMatt> either way ack the total with that.
<rusty> BlueMatt: Ack that change.  We can clean it up into an ordered list as a spelling fix if we want.
<BlueMatt> right, or that.
<roasbeef> rusty: yeh I meant update BIP 69 itself for that length case
<rusty> roasbeef: yeah, but BIP-69 is DOA afaict?
<BlueMatt> roasbeef: bip 69 should basically be used absolutely never anyway. so why bother updating it? :)
<roasbeef> is it? 
<BlueMatt> yes, I think that's a pretty universally-held view that bip 69 is fundamentally a bad idea.
<roasbeef> BlueMatt: pretty useful in practice imo, just because you don't use it doesn't mean it isn't useful...
<BlueMatt> well, outside of lightning where you need a consistent sort order, of course
<BlueMatt> roasbeef: dunno, you're literally the first person I've spoken to who is sensible on these things who has thought it was a reasonable thing in almost any context outside of lightning.
<devrandom> the order could be pseudo-random instead?
<roasbeef> we use it in pool as an eaxmple, useful to have a cannonical ordering in any multi-party transaction context 
<BlueMatt> devrandom: yes, there is basically no reason to ever prefer sorted over random unless you're doing something like lightning
<BlueMatt> anyway, this is quite a tangent
<rusty> roasbeef: better is an order by shared secret-prefix SHA, though until taproot it's a bit pointless to try to hide IMHO?
<roasbeef> idk make your own sorting BIP then if you don't like it, BIPs just say how things can be done, if ppl find them useful (like in thsi context), they'll use it 
<BlueMatt> (and if we ever had commitment transactions that weren't obviously-lightning I'd strongly suggest we permute the order by a shared secret)
<roasbeef> seems this entire time, no one has cared enough to make another one 
<BlueMatt> roasbeef: why write a bip for the thing that all wallets that are halfway decent already do :)
<roasbeef> BlueMatt: something something standards 
<rusty> BlueMatt: snap!  In fact, dual funding gives you this control over ordering.
<BlueMatt> rusty: huh! good opportunity to do sorting with a shared-secret permutation?
<BlueMatt> rusty: I'd strongly suggest we do that to avoid standing out (though i assume it doesnt matter until taproot?)
<rusty> BlueMatt: on co-construction, you explicitly give an ordering.  So you can do whatever you want in this case.
<niftynei> iiuc the DLC project uses the same protocol for ordering as the dual-funding proposal
<BlueMatt> ah! hmmm, i guess sensible implementations will randomize. guess it'd be cool to just force it to be random based on some shared random values (like pubkeys that dont ever hit the chain)
* ryanthegentry has quit (Quit: Connection closed)
<BlueMatt> avoids one more parameter on the wire, which is also always good
* ryanthegentry (883195f5@136.49.149.245) has joined
<ariard> yes we have custom inputs'serial_id in dlcs
<rusty> BlueMatt: Naah, you may want explicit ordering for weird SIGHASH_SINGLE tricks etc.
<BlueMatt> ohhh, hum. yuck but ok
<rusty> Anyway, meanwhile I'm happy with Matt's text.  Simply spelling out the requirement is nice.
* BlueMatt has always found it confusing to have to click through to bip 69, and was confused the first time he read it and had to look up implementation details for what to do if lengths differ
* orccoin has quit (Remote host closed the connection)
<rusty> OK, so I think we're in bikeshed territory?  Let's hash it out on issue, and apply it with or without Matt's reword.  I'd rather not block it on tangential rewording.
<roasbeef> it's mergeable as is imo 
<rusty> #action rusty Merge #872 (after validating test vectors).
* orccoin (~Rheanna@101.91.180.110) has joined
<rusty> I will split Matt's change into a separate PR myself, how's that?  Then we can argue there :)
<rusty> #topic https://github.com/lightningnetwork/lightning-rfc/pull/672
<BlueMatt> this seems to have enough acks already?
<BlueMatt> more than enough
<rusty> BlueMatt: Indeed.  And we have Eclair and c-lightning tested interop on this (it's pretty trivial).  Note that c-lightning has not implemented bech32m yet, so there's no normal way for users to supply an actual v1 address.
<rusty> Any objections to applying this?
<BlueMatt> do it
<roasbeef> heh yeh thought this was merged a while ago 
<roasbeef> do eet
<rusty> #action rusty to merge #672
<rusty> #topic https://github.com/lightningnetwork/lightning-rfc/issues/873
<BlueMatt> yes, would be nice, someone's gotta do the work :/
<rusty> Someone remind me of the purpose of this?  Is it more a "why not, it's free?"
<BlueMatt> yea, basically, though marginally increases the cost of dos'ing a channel so you have to actually put up non-dust values
<roasbeef> basically that someone can add dust and make your channel less useful 
* lnd-bot (~lnd-bot@165.227.7.29) has joined
<lnd-bot> [lightning-rfc] TheBlueMatt opened pull request #876: Concretize Output Ordering to be less confusing (master...2021-05-no-bip69) https://github.com/lightningnetwork/lightning-rfc/pull/876
* lnd-bot (~lnd-bot@165.227.7.29) has left
<rusty> Well, since dust levels don't have to be the same on both sides, this proposal is insufficient?
<BlueMatt> tbh i didnt read the proposed text lol
<rusty> I think it has to be "if either side considers it non-dust, we count it"?
<BlueMatt> yea, probably what you said rusty
<roasbeef> rusty: well the dust is already asymmetric, so I imagine this would be as well?
<roasbeef> at a high level is proposes not counting dust htlcs to the max htlc limit 
<roasbeef> which would then be clamped only by the max dust settings 
<roasbeef> tho maybe just two limits a better? 
<BlueMatt> but if you have one side that adds a billion dust outputs but they aren't dust for the other side, then the other side cant broadcast a transaction cause it has a billion outputs?
<rusty> Well, I guess if you propose 1M dust htlcs which are not dust for yourself, that's on you?
<BlueMatt> cause the dust limit of the broadcaster applies to all the htlcs not just the broadcaster's htlcs
<BlueMatt> wouldnt you be able to prevent someone from broadcasting their commitment tx?
<rusty> BlueMatt: the proposal says you can't add an htlc if it would go over the max for the *other* side.
<roasbeef> BlueMatt: each side says what their dust limit is tho 
<joost_> hi all. won't this open up a new vector where an attacker can send indeed a billion htlcs along the same (26 hop) route into the network? it won't weigh down on the commit txes, but it does on processing
<rusty> BlueMatt: so I think you can only shoot yourself.
<joost_> (1 msat htlcs)
<roasbeef> joost_: yeh so there needs to be another count clamp on dust htlcs themselves 
<BlueMatt> joost_: I dont see why that needs a separate limit? you already have fees and its no different than any other htlc
<roasbeef> I think the ML post has more context here also 
<rusty> BlueMatt: if you set fees to zero you get this problem though, I guess.
<roasbeef> I g2g in 10 
<BlueMatt> rusty: I mean you get that problem if you send a bajillion 1sat htlcs too....
<BlueMatt> rusty: ok, I think I agree that as worded you can only shoot yourself. I'd propose mentioning that you can shoot yourself in the spec.
<roasbeef> BlueMatt: another limit since those would all be dust, you need to clamp dust and non dust in total count of each 
* orccoin has quit (Remote host closed the connection)
<BlueMatt> roasbeef: why
<BlueMatt> roasbeef: what is wrong with having a ton of dust outputs?
<roasbeef> that PR says don't count dust towards the max htlc number anymore
<rusty> Thinking more, I'm not sure what the actual CPU limitation BigNum dust HTLCs would be.  I suspect we may have some O(N^2) somewhere in our tx construction code maybe?
<roasbeef> what stops a commitment from being fully dusted if that isn't there BlueMatt ?
<BlueMatt> rusty: it doesnt feed into tx construction, though?
<BlueMatt> rusty: you dont have to send signatures for a dust htlc, too!
<rusty> BlueMatt: we discard them though, and turn them into fees.
<BlueMatt> roasbeef: you already have a limit on the total in-flight?
<rusty> BlueMatt: I think that's O(N).
<roasbeef> BlueMatt: that PR says you don't count dust towards total in flight anymore 
<BlueMatt> roasbeef: htlc count, I assumed we'd still count them towards value-in-flight
<rusty> BlueMatt: yeah, but a 1 BTC channel can easily have > 10M dust htlcs.
<BlueMatt> rusty: right, would need to read code carefully, but i was assuming signature creation/checking would basically dwarf you in any case
<BlueMatt> ie even one signature could dwarf 1M simple ifs in a for loop....depending on how the code is
* orccoin (~Rheanna@101.89.207.55) has joined
<rusty> BlueMatt: I'd like to benchmark, just to be sure.  We never bothered optimizations for < 1000 htlcs as we have now.
<BlueMatt> yes, agreed
<rusty> OK, so next step is to turn it into actual patch with caveats mentioned here?
<BlueMatt> +1
<roasbeef> i'll ping eugene about that
<roasbeef> (crypt-iq)
<rusty> #action rusty to assign feature bit for #873
<niftynei> seems like setting a "max fees from dust" limit might be a better way to approach this than a cap on the num of 'dust htlcs'
<BlueMatt> let me paste this on the issue
<BlueMatt> niftynei: sgtm
<rusty> OK, I say we end the meeting early, rather than trying to squeeze something in 4 minutes?
<BlueMatt> niftynei: its already hard to pick appropriate dust limits trying to munge a "max fees from dust" limit out of it. having an explicit "max fees from dust" seems strictly better than today in any case.
<BlueMatt> rusty: ack
<rusty> #endmeting
<rusty> #endmeeting
<lndev-bot> Meeting ended Mon May 24 20:58:06 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
<lndev-bot> Minutes:        https://lightningd.github.io/meetings/https___github_com_lightningnetwork_lightning_rfc_issues_874/2021/https___github_com_lightningnetwork_lightning_rfc_issues_874.2021-05-24-20.12.html
<lndev-bot> Minutes (text): https://lightningd.github.io/meetings/https___github_com_lightningnetwork_lightning_rfc_issues_874/2021/https___github_com_lightningnetwork_lightning_rfc_issues_874.2021-05-24-20.12.txt
<lndev-bot> Log:            https://lightningd.github.io/meetings/https___github_com_lightningnetwork_lightning_rfc_issues_874/2021/https___github_com_lightningnetwork_lightning_rfc_issues_874.2021-05-24-20.12.log.html
* rusty needs more coffee.