Silent Payments – Receive private payments from anyone on a single static address without requiring any interaction or extra on-chain overhead

Silent_Payments.md

Silent Payments

Receive private payments from anyone on a single static address without requiring any interaction or extra on-chain overhead.

Update: This now has a BIP and WIP implementation

Overview

The recipient generates a so-called silent payment address and makes it publicly known. The sender then takes a public key from one of their chosen inputs for the payment, and uses it to derive a shared secret that is then used to tweak the silent payment address. The recipient detects the payment by scanning every transaction in the blockchain.

Compared to previous schemes¹, this scheme avoids using the Bitcoin blockchain as a messaging layer² and requires no interaction between sender and recipient³ (other than needing to know the silent payment address). The main downsides are the scanning requirement, the lack of light client support, and the requirement to control your own input(s). An example use case would be private one-time donations.

While most of the individual parts of this idea aren't novel, the resulting protocol has never been seriously considered and may be reasonably viable, particularly if we limit ourselves to detecting only unspent payments by scanning the UTXO set. We'll start by describing a basic scheme, and then introduce a few improvements.

Basic scheme

The recipient publishes their silent payment address, a single 32 byte public key: X = x*G

The sender chooses an input containing a public key: I = i*G

The sender tweaks the silent payment address with the private key that corresponds to their chosen input: X' = hash(i*X)*G + X

Sincei*X == x*I (Diffie-Hellman Key Exchange), the recipient can detect the payment by calculating hash(x*I)*G + X for each input key I in the blockchain and seeing if it matches an output in the corresponding transaction.

Improvements

UTXO set scanning

If we forgo detection of historic transactions and only focus on the current balance, we can limit the protocol to only scanning the transactions that are part of the UTXO set when restoring from backup, which may be faster.

Jonas Nick was kind enough to go through the numbers and run a benchmark of hash(x*I)*G + X on his 3.9GHz Intel(R) Core(TM) i7-7820HQ CPU, which took roughly 72 microseconds per calculation on a single core. The UTXO set currently has 80 million entries, the average transaction has 2.3 inputs, which puts us at 2.3*80000000*72/1000/1000/60 = 221 minutes for a single core (under 2 hours for two cores).

What these numbers do not take into account is database lookups. We need to fetch the transaction of every UTXO, as well as every transaction for every subsequent input in order to extract the relevant public key, resulting in (1+2.3)*80000000 = 264 million lookups. How slow this is and what can be done to improve it is an open question.

Once we're at the tip, every new unspent output will have to be scanned. It's theoretically possible to scan e.g. once a day and skip transactions with fully spent outputs, but that would probably not be worth the added complexity. If we only scan transactions with taproot outputs, we can further limit our efforts, but this advantage is expected to dissipate once taproot use becomes more common.

Variant using all inputs

Instead of tweaking the silent payment address with one input, we could instead tweak it with the combination of all input keys of a transaction. The benefit is that this further lowers the scanning cost, since now we only need to calculate one tweak per transaction, instead of one tweak per input, which is roughly half the work, though database lookups remain unaffected.

The downside is that if you want to combine your inputs with those of others (i.e. coinjoin), every participant has to be willing to assist you in following the Silent Payment protocol in order to let you make your payment. There are also privacy considerations which are discussed in the "Preventing input linkage" section.

Concretely, if there are three inputs (I1, I2, I3), the scheme becomes: hash(i1*X + i2*X + i3*X)*G + X == hash(x*(I1+I2+I3))*G + X.

Scanning key

We can extend the silent payment address with a scanning key, which allows for separation of detecting and spending payments. We redefine the silent payment address as the concatenation of X_scan, X_spend, and derivation becomes X' = hash(i*X_scan)*G + X_spend. This allows your internet-connected node to hold the private key of X_scan to detect incoming payments, while your hardware wallet controls X_spend to make payments. If X_scan is compromised, privacy is lost, but your funds are not.

Address reuse prevention

If the sender sends more than one payment, and the chosen input has the same key due to address reuse, then the recipient address will also be the same. To prevent this, we can hash the txid and index of the input, to ensure each address is unique, resulting in X' = hash(i*X,txid,index)*G + X. Note this would make light client support harder (edit: not necessarily, see here).

Noteworthy details

Light clients

Light clients cannot easily be supported due to the need for scanning. The best we could do is give up on address reuse prevention (so we don't require the txid and index), only consider unspent taproot outputs, and download a standardized list of relevant input keys for each block over wifi each night when charging. These input keys can then be tweaked, and the results can be matched against client-side block filters. Possible, but not simple. (edit: some more ideas how to do light client support here)

Effect on BIP32 HD keys

One side-benefit of silent payments is that BIP32 HD keys⁴ won't be needed for address generation, since every address will automatically be unique. This also means we won't have to deal with a gap limit.

Different inputs

While the simplest thing would be to only support one input type (e.g. taproot key spend), this would also mean only a subset of users can make payments to silent addresses, so this seems undesirable. The protocol should ideally support any input containing at least one public key, and simply pick the first key if more than one is present.

Pay-to-(witness-)public-key-hash inputs actually end up being easiest to scan, since the public key is present in the input script, instead of the output script of the previous transaction (which requires one extra transaction lookup).

Signature nonce instead of input key

Another consideration was to tweak the silent payment address with the signature nonce⁵, but unfortunately this breaks compatibility with MuSig2 and MuSig-DN, since in those schemes the signature nonce changes depending on the transaction hash. If we let the output address depend on the nonce, then the transaction hash will change, causing a circular reference.

Sending wallet compatibility

Any wallet that wants to support making silent payments needs to support a new address format, pick inputs for the payment, tweak the silent payment address using the private key of one of the chosen inputs, and then proceed to sign the transaction. The scanning requirement is not relevant to the sender, only the recipient.

Preventing input linkage

A potential weakness of Silent Payments is that the input is linked to the output. A coinjoin transaction with multiple inputs from other users can normally obfuscate the sender input from the recipient, but Silent Payments reveal that link. This weakness can be mitigated with the "variant using all inputs", but this variant introduces a different weakness – you now require all other coinjoin users to tweak the silent payment address, which means you're revealing the intended recipient to them.

Luckily, a blinding scheme⁶ exists that allows us to hide the silent payment address from the other participants. Concretely, let's say there are two inputs, I1 and I2, and the latter one is ours. We add a secret blinding factor to the silent payment address, X + blinding_factor*G = X', then we receive X1' = i1*X' (together with a DLEQ to prove correctness, see full write-up⁶) from the owner of the first input and remove the blinding factor with X1' - blinding_factor*I1 = X1 (which is equal to i1*X). Finally, we calculate the tweaked address with hash(X1 + i2*X)*G + X. The recipient can simply recognize the payment with hash(x*(I1+I2))*G + X. Note that the owner of the first input cannot reconstruct the resulting address because they don't know i2*X.

The blinding protocol above solves our coinjoin privacy concerns (at the expense of more interaction complexity), but we're left with one more issue – what if you want to make a silent payment, but you control none of the inputs (e.g. sending from an exchange)? In this scenario we can still utilize the blinding protocol, but now the third party sender can try to uncover the intended recipient by brute forcing their inputs on all known silent payment addresses (i.e. calculate hash(i*X)*G + X for every publicly known X). While this is computationally expensive, it's by no means impossible. No solution is known at this time, so as it stands this is a limitation of the protocol – the sender must control one of the inputs in order to be fully private.

Comparison

These are the most important protocols that provide similar functionality with slightly different tradeoffs. All of them provide fresh address generation and are compatible with one-time seed backups. The main benefits of the protocols listed below are that there is no scanning requirement, better light client support, and they don't require control over the inputs of the transaction.

Payment code sharing

This is BIP47². An OP_RETURN message is sent on-chain to the recipient to establish a shared secret prior to making payments. Using the blockchain as a messaging layer like this is generally considered an inefficient use of on-chain resources. This concern can theoretically be alleviated by using other means of communicating, but data availability needs to be guaranteed to ensure the recipient doesn't lose access to the funds. Another concern is that the input(s) used to establish the shared secret may leak privacy if not kept separate.

Xpub sharing

Upon first payment, hand out a fresh xpub instead of an address in order to enable repeat payments. I believe Kixunil's recently published scheme³ is equivalent to this and could be implemented with relative ease. It's unclear how practical this protocol is, as it assumes sender and recipient are able to interact once, yet subsequent interaction is impossible.

Regular address sharing

This is how Bitcoin is commonly used today and may therefore be obvious, but it does satisfy similar privacy requirements. The sender interacts with the recipient each time they want to make a payment, and requests a new address. The main downside is that it requires interaction for every single payment.

Open questions

• Exactly how slow are the required database lookups? Is there a better approach?
• Is there any way to make light client support more viable?
• What is preferred – single input tweaking (revealing an input to the recipient) or using all inputs (increased coinjoin complexity)?
• Are there any security issues with the proposed cryptography?
• In general, compared to alternatives, is this scheme worth the added complexity?

---

Thanks to Kixunil, Calvin Kim, and Jonas Nick, holihawt and Lloyd Fournier for their help/comments, as well as all the authors of previous schemes. Any mistakes are my own.

There is also a discussion of this scheme on the bitcoin-dev mailing list.

Footnotes

1. Stealth Payments, Peter Todd: https://github.com/genjix/bips/blob/master/bip-stealth.mediawiki ↩

2. BIP47 payment codes, Justus Ranvier: https://github.com/bitcoin/bips/blob/master/bip-0047.mediawiki ↩ ↩²

3. Reusable taproot addresses, Kixunil: https://gist.github.com/Kixunil/0ddb3a9cdec33342b97431e438252c0a ↩ ↩²

4. BIP32 HD keys, Pieter Wuille: https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki ↩

5. 2020-01-23 ##taproot-bip-review, starting at 18:25: https://gnusha.org/taproot-bip-review/2020-01-23.log ↩

6. Blind Diffie-Hellman Key Exchange, David Wagner: https://gist.github.com/RubenSomsen/be7a4760dd4596d06963d67baf140406 ↩ ↩²

I implemented the "Basic scheme" section to test the concept.
There is a description of Usage Example.

https://github.com/w0xlt/silent-payment-lib

What would be the best way to implement UTXO set scanning? Via ZMQ notification or rewriting some logic in BDK/rust-bitcoin?

@w0xlt nice work. I'd imagine modifying rust-bitcoin would be a way forward, so you'd have full access to the UTXO set. If you happen to get around to it, benchmarks would be welcome, as we currently only have naive numbers that don't take into account lookups.

@RubenSomsen Very nice that you consider working on this. I think there are indeed a few examples of interesting use cases. Some comments:

• Is there a reason why you don't call it stealth addresses? It's very close similar to the original stealth address proprosal and people know it by that name, so it may be good to call it like that.
• I think ECDH performance can still be improved in libsecp256k1, so maybe these numbers will get better even. But on the other hand, two hours sounds very nice already and I guess the DB lookups are more problematic anyway,
• I suggest reaching out to other projects that have implemented stealth addresses, most notably Monero. I think they did a lot of the engineering work / design already and there's no need to reinvent all these ideas (e.g., watch-only key) in our community.

@real-or-random thanks for taking the time to look at the proposal.

Is there a reason why you don't call it stealth addresses?

I don't have a strong opinion on the name, but the implemented stealth address proposal (as well as how it's implemented in Monero, I believe) ended up adding an op_return for ECDH to every transaction, which seemed like an important enough distinction to give it a different name. In an earlier draft I had it named "Simplified Stealth Payments".

I think ECDH performance can still be improved in libsecp256k1, so maybe these numbers will get better even

Nice, that's certainly good news.

I guess the DB lookups are more problematic anyway

I briefly discussed this with @Kixunil who had some ideas about it, but as of yet I still don't have a good intuition for how bad it'll be.

I suggest reaching out to other projects that have implemented stealth addresses, most notably Monero

Thanks for the suggestion, they must have dealt with some of the problems. I did talk to Grin devs briefly, since they do recovery from the UTXO set, but it seemed too different to be relevant.

And to clarify, the watch-only key idea is also described in the stealth payments draft BIP, I just listed it again to give a complete overview.

Strong agreement with naming differently to avoid confusion. Regarding speed, I think we will have to write a benchmark to know exactly. Shouldn't be rocket science anyway. :)

As for the benchmark, I implemented the basic scheme in c (not sure if it's completely correct), on top of PR #994 from https://github.com/bitcoin-core/secp256k1/.

The file is:

https://github.com/w0xlt/secp256k1/blob/a9677ad9f064efd6c1f91afb9fa2f5d2ab43cd03/examples/spbs.c

I think it would be better to integrate that PR with Bitcoin Core and make the change in the wallet to get the benchmark than to re-implement the scan logic in rust-bitcoin, wouldn't it?

@w0xlt I'd like to see both benchmarks. Many wallets are built on top of Core as an external program which I believe is good architecture so would be interesting to know if the overhead is prohibitive.

@w0xlt Nice work implementing it into secp256k1. I agree with @Kixunil that both benchmarks would be interesting and that it would be nice if the implementation could function as software that operates external from Core.

I think there are some ways to improve the performance using the lower level internal APIs of the library. Some ideas:

1. I guess ECDH is doing constant time multiplication (which it should) but remember the "scanning key" doesn't necessarily hold funds so you should actually make the "scanning key" a mandatory part of the spec (i.e. 64 byte silent addresses) and always do the scanning using variable time multiplication. I wonder if Jonas was doing this with his benchmarks.
2. After multiplying the shared secret by G and adding it back don't convert back to affine coordinates -- do the comparison by using a fast Jacobian == Affine function (not sure if this exists in the library). This saves you doing a modular inversion. Whether this is faster may depend on whether you have to done comparison per tx or many (i.e. is the output always in the same place).

@LLFourn nice ideas however lack of constant time scanning may leak privacy. I don't expect the improvement to be significant, is it?

@Kixunil leak privacy in what adversarial model? The improvement would be significant ~25%.

@LLFourn hmm, now that I think about it more, I guess it's not that easy to get the timings from e.g. a sandboxed application. 25% sounds good!

I made a basic implementation: bitcoin/bitcoin#24897.

I believe it can be used for bench-marking. There is a functional test in the implementation (test/functional/wallet_silentpayment.py). I've also been running some silent payments on signet and apparently this implementation is working as expected.

To send a silent payment, a new silent_payment: true option has been added to send RPC.
Example: /src/bitcoin-cli -regtest -named send outputs="[{\"bcrt1pwlh5xuyrpgfunwyww8cfu78yfs2yqyevl7yturavahh5kgxwdd2q5hzgfu\": 1.1}]" fee_rate=1 options="{ \"silent_payment\": true}"

This option instructs the command to verify that all outputs are Taproot addresses and tweaks all of them using the private key of the first input.
The resulting transaction will have different outputs than those originally entered by the user.

This can be verified in DescriptorScriptPubKeyMan::CreateSilentPaymentAddress and in spend.cpp::CreateSilentTransaction.

The recipient's wallet needs a new flag called SILENT_PAYMENT. This flag allows an additional scan that verifies that the wallet keys match the silent payment scheme. When it detects a silent payment that belongs to the wallet, it is stored in a rawtr() descriptor.

./src/bitcoin-cli -regtest -named createwallet wallet_name="recipient" silent_payment=true

Scanning logic can be verified in DescriptorScriptPubKeyMan::VerifySilentPaymentAddress.

The file that implements the Basic Scheme section of this article is src/wallet/silentpayment.cpp.

There are more details in the PR description. Let me know if more information is needed.

@RubenSomsen Just to be clear (maybe this should be explicit in the BIP) the private key the silent payment recipient needs to derive to spend form the silent output is: x' = hash(x*I) + x (mod p) ?

Just because ECDH and a shared secret are involved, I wanted to make sure the sender does not also have the private key to spend the silent output.

@w0xlt well done, this looks promising. Could you elaborate a little bit about the scanning process? I take it you're scanning every block and not just the UTXO set? Note the variant using all inputs should actually be faster in terms of scanning, so it may be preferable.

@pinheadmz yes that is correct, by adding the recipient key to the shared secret, we ensure only the recipient can spend it.

@RubenSomsen yes that is true. The silent payment scanning methods (CWallet::VerifySilentPayment and DescriptorScriptPubKeyMan::VerifySilentPaymentAddress) are called in CWallet::AddToWalletIfInvolvingMe.

If I understand correctly, this method is a central point to check transactions coming from mempool and new blocks. As this implementation adds the transaction to the wallet in a rawtr description, I first considered this method.

Would an RPC like scantxoutset() be better for benchmarking?

I use the private key of the first input to tweak all public keys of the outputs (except change). Therefore, it is one tweak per output. When it comes to iterations, I think it's the same as if a hash of all inputs were used. In the text you mention "one tweak per transaction". Wouldn't that be one tweak per output ?

@w0xlt

Would an RPC like scantxoutset() be better for benchmarking?

I think both could be interesting. The advantage of scanning during block validation is that you don't have to do any additional database lookups, and you'll also find spent outputs. The downside is that you won't take advantage of the fact that you can skip spent outputs.

In terms of benchmarks for your current implementation, the main question I'd have is how much slower block validation becomes compared to when you don't scan for silent payments.

I use the private key of the first input to tweak all public keys of the outputs (except change). Therefore, it is one tweak per output. When it comes to iterations, I think it's the same as if a hash of all inputs were used.

Yes, that seems like the same amount of effort. Just note that such a design would be limiting, as it'd be incompatible with receiving from coinjoins.

In the text you mention "one tweak per transaction". Wouldn't that be one tweak per output ?

I can see how that can be confusing. If you want to see if you received a payment, for every transaction you do the following: you take the combined key of all inputs, use it to calculate your silent payment address, and then you compare it with all the outputs to see if any of them match. So while you're comparing all the outputs, you're only tweaking once.

I changed the scantxoutset RPC to have a silent_payment parameter. But scanning the UTXO Set for silent payments takes much longer than regular transactions, possibly prohibitively.

The main reason seems unrelated to the silent payment scheme, but that the Bitcoin Core UTXO Set only stores the unspent transaction outputs, not the inputs.
Therefore, for each coin in the UTXO Set, it is necessary to retrieve the transaction to know the inputs.
And after that, it is necessary to retrieve the previous transaction to know the scriptPubKey being spent by the first input, so we will know how to extract the pubkey from the scriptSig.

Even the GetTransaction() method being used alone, without any silent payment scheme, it is possible to notice a long time to finish the scan.

I added a cache to store the EDCH results for each input. Probably this implementation can be quite optimized, but I think the basic structure is this.

However, Bitcoin Core does not support UTXO Set scanning as a wallet operation. This RPC is a tool for scanning descriptors. Maybe assumeUTXO will change that.

When receiving a valid new transaction from the mempool, the transaction being spent is already available in the UTXO Set, so this performance issue does not occur. The cost is only related to the EDCH for each wallet key.

For this reason, I initially prioritized how to handle silent transactions coming from mempool and this operation seems quite feasible. But it's good to have both metrics.

To test (example):
$ bitcoin-cli -regtest scantxoutset start "[{\"desc\": \"tr(tprv8ZgxMBicQKsPe4mDP1295ti2BqcgFzPWkKnvsGyKVerGqf9tDif6yR4yLcK6Pf49tQ1HRQK2vjXrAVqxVUJCtXWn3AAiacXnXhUf6nxBJAp/86'/1'/0'/0/*)#pskyw6xd\", \"range\": [0,5]}]" true

@w0xlt

for each coin in the UTXO Set, it is necessary to retrieve the transaction to know the inputs. And after that, it is necessary to retrieve the previous transaction

Yeah, this ends up being roughly 264 million lookups for the entire UTXO set...

I added a cache to store the EDCH results for each input. Probably this implementation can be quite optimized

Perhaps during IBD you could consider storing the relevant input key with each Taproot UTXO and then use this additional information to scan the UTXO set after IBD completes. Note that the simplest implementation of this might have duplicate entries, as there is only one relevant input per transaction but there can be multiple UTXOs. Incidentally, the resulting list of input keys is the same information that light clients would need to perform their own scanning.

I initially prioritized how to handle silent transactions coming from mempool and this operation seems quite feasible. But it's good to have both metrics.

Makes sense to me

I added a functional test for the scantxoutset that uses the silent_payment parameter, so it will be easier to make changes and confirm that the test works. The current logic passes the test because there are no performance issues in regtest.

Perhaps during IBD you could consider storing the relevant input key with each Taproot UTXO and then use this additional information to scan the UTXO set after IBD completes.

It can work. I will try to build an index for this.

Note I gave an online presentation on Silent Payments. You can check it out here.

I will try to build an index for this

@w0xlt 👍

@RubenSomsen thanks for the url. I will watch the presentation.

I implemented the index in the latest commit. The node can be started with ./src/bitcoind -silentpaymentindex.
With this index, the UTXO Set scan for one key is relatively fast on signet, at least.
This index is also used when the wallet is loading and scanning blocks.

I also added a restriction to not allow the creation or loading of wallets with silent_payment flag without the index not being active or synchronized. This changes the interface a bit, but I think it's the best way to prevent the wallet from being in an inconsistent state.

scantxoutset RPC can be run without risks on mainnet, as it does not require wallet or coins.

I think that the current implementation can be used for benchmarking.

I implemented the index in the latest commit

@w0xlt Very well done, happy to see you've taken it this far already. I'd love to play around with it, but I don't have a lot of time on my hands right now. Any chance you'd be willing to provide the numbers? What I'd be curious about is a.) how many UTXOs there are on signet and b.) how long it took you to scan them c.) on what kind of hardware. This could then be extrapolated to the Bitcoin full UTXO set.

And a question: are you currently keeping an index for taproot UTXOs only, or for all UTXOs? While technically only taproot UTXOs are required, a benchmark on the full set seems good for now.

Also, would you be open to a more direct line of communication via IRC or Telegram? I'm RubenSomsen on both.

Edit: another thing I'd be curious about: how much longer does it take to do IBD with and without this additional index.

@RubenSomsen Here are some numbers. I haven't benchmarked it before because I use a virtual machine for development, so it's not isolated enough for a reliable stat, but these numbers can be used as a rough estimate. The configuration I used is a VMWare guest machine with 8 cores of virtual processors and 20 MB of RAM. I don't think this reflects performance if the same configuration was used on the host machine.

The stat below shows the number of UTXO (on the signet and mainnet) and how long it takes. I used std::chrono::steady_clock for more accurate results. This also includes the silent payment index created from genesis block to the tip.

(on signet) silentpaymentindex is enabled at height 90627 in 5m
(on mainnet) silentpaymentindex is enabled at height 736986 in 330m (5h30min)

scantxoutset signet:
  "txouts": 1053822,
  "height": 90805,
  "silent_payment": true,
  "duration_seconds": 82,
  "keys": 1

scantxoutset mainnet:
  "txouts": 81893320,
  "height": 737045,
  "silent_payment": true,
  "duration_minutes": 18,
  "keys": 1

Wallet Rescan (Signet):
Rescan completed from block 90149 to 91029 (880 blocks): 20 minutes

Wallet Rescan (Mainnet):
Rescan completed from block 737050 to 737097 (47 blocks): 43 seconds

And a question: are you currently keeping an index for taproot UTXOs only, or for all UTXOs?

Only transactions where all outputs are Taproot.

    for (auto& vout : tx->vout) {
        std::vector<std::vector<unsigned char>> solutions;
        TxoutType whichType = Solver(vout.scriptPubKey, solutions);

        if (whichType != TxoutType::WITNESS_V1_TAPROOT) {
            return false;
        }
    }

In previous versions, there was no index but now all the above methods (except mempool transactions) use the index. If the above numbers are reasonable, a non-indexed version can be considered.

Thanks for making the contact available. I'll be in touch.

Thank you @RubenSomsen for the proposal and kudos @w0xlt for the implementation.

We were discussing this proposal with folks at Trezor and here are our takes:

• for a hardware wallet we need to use the "scanning key option" for obvious reasons
• address reuse prevention via txid:vout of input is great idea
• we should come up with an address format for silent payments so people won't accidentally send coins to void
• address format could encode different options, such as the scanning key use, address reuse prevention, etc. see the proposal below

Address format rough proposal:

• bech32m address, hrp = sp1
• encoded data
  • 00 || X_spend (don't use scanning key, don't use address reuse prevention)
  • 01 || X_spend || X_scan (use scanning key, don't use address reuse prevention)
  • 02 || X_spend (don't use scanning key, use address reuse prevention)
  • 03 || X_spend || X_scan (use scanning key, use address reuse prevention)
  • bit 0 of the first byte says whether to use the scanning key, bit 1 of the first byte says whether to use address reuse prevention

For efficient implementation of scanning it would be great if we were able to come up with a scheme where only one input pubkey is required. (In this case we could just enrich each entry of the UTXO set with one particular pubkey). It could trivially be always the first input of the TX, but this unfortunately breaks the BIP69 lexicographical ordering of inputs. Need more thoughts on this.

Nice proposal, I wonder if it's beneficial to have it configurable that much. Seems to increase complexity and going with mandatory scanning key and mandatory address reuse prevention looks fine to me. That being said, some form of feature flags/versioning would be good to have.

Nice proposal, I wonder if it's beneficial to have it configurable that much.

Yeah. For Trezor we would use both the scanning key and the address reuse prevention. If others are fine with not having these two flags configurable, so am I and we could just use 00 || X_spend || X_scan and mandate address reuse prevention.

If we forgo detection of historic transactions and only focus on the current balance, we can limit the protocol to only scanning the transactions that are part of the UTXO set when restoring from backup, which may be faster.

From the above benchmarks (and implementation requirements) it seems that detection of historical transactions (spent TXOs) is not particularly practical. The silentpaymentindex would need to support the whole blockchain and not just the UTXO set. I think this is worth noting when comparing to other protocols, for example BIP47. On the other hand, enabling hardware wallet participation using scanning keys is a nice advantage.

Unlike BIP47 however, I don't believe this scheme will be compatible with popular light client protocols such as the Electrum protocol, even if modified to support it, as the scanning burden on the server would be too great for that model of operation. This will make widespread wallet support difficult to achieve.

The silentpaymentindex would need to support the whole blockchain and not just the UTXO set.

I am not sure I understand this comment. The only thing you need is the enriched UTXO set where each UTXO also has associated public key(s) used for ECDH. If we come up with a method where it's always deterministic which public key has been used, this index will be only (32 * utxo_count) bytes big.

The only thing you need is the enriched UTXO set

I was referring to retrieving the whole transaction history (including spent TXOs), which would mean scanning beyond the UTXO set.

Thanks for the numbers @w0xlt.

Thanks for making the contact available. I'll be in touch.

Note I have seen any messages yet, in case something went wrong.

@prusnak thanks for the feedback.

it would be great if we were able to come up with a scheme where only one input pubkey is required

Other than BIP69, another issue with picking the first input is that only one coinjoin participant can make a silent payment. To summarize our options:

Using the 1st input:

• Breaks BIP69, thus leaking information
• Only one silent payment per coinjoin
• Coinjoin recipient knows which input is yours

Using a random input:

• Increases scanning cost and input database size (at least 2x)
• Coinjoin recipient knows which input is yours

Using all inputs:

• Increases coinjoin complexity to collaboratively generate the address

If others are fine with not having these two flags configurable, so am I

Yeah, minimizing the flags where possible, as @Kixunil said, seems preferable. I think reuse prevention and the scanning key can just be the default, as the downsides seem fairly minimal.

detection of historical transactions (spent TXOs) is not particularly practical. The silentpaymentindex would need to support the whole blockchain

@craigraw the index is not required if you want to find historic transactions. In that case you'd just actively scan the entire history during IBD.

And in general, we'd need to compare scanning every tx during IBD versus creating the index during IBD and then scanning the UTXO set. If the latter is not significantly faster, it should perhaps be left out of v1 to keep things simple.

Note that BIP69 is not widely implemented and I believe most developers are in favor of randomization. The main argument is that some protocols can't order the inputs/outputs so random ordering helps hide them.

In that case you'd just actively scan the entire history during IBD.

Generating a silent address before IBD, and redoing IBD for every new silent address does not seem particularly practical to me. It also seems to rule out light client support for historic transactions.

Note that BIP69 is not widely implemented

That could be said for coinjoin as well. But it seems that over 60% of transactions actually are BIP69 compliant. Source

I believe most developers are in favor of randomization

That's just an assumption.

The main argument is that some protocols can't order the inputs/outputs so random ordering helps hide them.

Could you give an example?

🚀

@pajasevi you should take into account that some transactions are compliant by accident - randomly ordering inputs/outputs that way (for 1 input, 2-output txes there's 50% chance it's compliant, for 1 input, 1 output chance is 100%...), so actual support is lower, not sure how large.

That's just an assumption.

That's my understanding of the discussion that was in bitcoin mailinglist.

Could you give an example?

I think OTS is one. Anyway I suggest you search that discussion in the ML archive and see for yourself.

I dig (haha) @1440000bytes's suggestion to use this in combination with DNS* records / .well-known URLs.

This paves the way for send-to-email setups. User enters an email address and the super smart wallet checks for bolt12, lnurl, silent payments and whatever else, picks the most suitable option and sends off the coins.

But this brings me to the Hotel California concern: since it's quite expensive to scan the chain, a user may eventually want to stop doing this. But nothing prevents people from sending coins to the stealth address. We should probably add an expiration block height to the address format (cc @prusnak). Receiver wallets should add a safety margin and e.g. scan 10K blocks more. And maybe wallets should have a feature to manually scan a specific transaction, "hey did you get my coins in TX_ID?".

DNS records and .well-known entries can relatively easily be updated, allowing for short expiration times.

• = not sure how secure DNS can be made, for .well-known you can use https

@Sjors cool ideas! Agree with everything you said. AFAIK DNSSec should be fine but I'm not deeply knowledgeable about it.

Very neat and promising concept! 🎉

The UTXO set currently has 80 million entries, the average transaction has 2.3 inputs, which puts us at 2.38000000072/1000/1000/60 = 221 minutes for a single core (under 2 hours for two cores).

Why would we need to scan the whole UTXO set for backup restoration? If the recipient creates/publishes a silent payment address at block time N, we can safely ignore all UTXO entries that were created at block height < N (or better < (N - SAFETY_MARGIN)) to take possible reorgs into account). Though we still have to iterate over all UTXOs, the number of total database lookups and calculations could be drastically reduced with that filtering. (A block height index would be helpful to skip iterating old UTXOs). For example, about ~78% of all UTXOs were created pre-Taproot currently:

$ sqlite3 ~/.bitcoin/utxos.sqlite
SQLite version 3.38.5 2022-05-06 15:25:27
Enter ".help" for usage hints.
sqlite> select count(*) from utxos where height < 709632;
64661607
sqlite> select 64661607.0/max(rowid)*100 from utxos;
78.4715574372404

(If anyone wants to try this out, the UTXO set in sqlite-Format can be created either directly with PR bitcoin/bitcoin#24952, or converted from legacy to sqlite format with my conversion tool: https://github.com/theStack/utxo_dump_tools)

Thinking even further, maybe it makes sense to save the creation/publishing block time as part of the address format, so the user doesn't have to store that extra data for faster restoring from a backup? Together with @Sjors' idea of adding an expiration block, this would lead to a block range being included in the address format. Recipients that neither want to disclose their address creation (block) time nor want to have their address expired can always simply set this interval to [0, MAX]*. In practice it would make sense though to never scan for pre-taproot-activation UTXOs, i.e. before block 709632.

*) 3 bytes for a block height should be more than enough (like done for the short_channel_id in lightning: https://github.com/lightning/bolts/blob/bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730/07-routing-gossip.md#definition-of-short_channel_id), i.e. in this case MAX would be 2^24-1

I don't think we should publish information that's not necessary for the sender. Tracking the birth date of an address can be handled by the user wallet. It's not the end of the world if they lose that part of the backup.

I don't think we should publish information that's not necessary for the sender. Tracking the birth date of an address can be handled by the user wallet. It's not the end of the world if they lose that part of the backup.

Right, on a second thought I agree that's not a good idea. If at all, the birth date would better fit into a format for backing up the silent payment private key (i.e. only used by the recipient), similar to e.g. WIF, but I guess this is absolutely not high priority at this point.

What about using a taproot change output for the sender's pubkey rather than an input? It would mean that spent TXO lookups aren't required and in many cases, I think it would require less searching than required for using an input's pubkey. I think it would also work better for coinjoins as it does not reveal which input was the senders, and change is typically considered dirty anyways so less harm there?

A few downsides is that it would make changeless coin selection impossible as a change output would always be required. This has a chain space impact, but also change outputs are often created so I'm not sure how bad that would actually be. Additionally, it would mean that only Taproot outputs could be used for the sender's pubkey, although it seems like that is already expected?

@achow101

spent TXO lookups aren't required

The change could get spent which still may require a lookup but obviously just one, not two.

it does not reveal which input was the senders

It does if one can analyze the amounts, which is likely. E.g. if CJ equal outputs are 0.1 and there's one input with 0.12345 and an output 0.02345 then those are linked unless there are other people with same inputs&changes.

However, if you CJ more sats, so that you have more equal-amount outputs you could use one of those. It'd still link them but at least not to the sender.

That being said, making lookup less costly looks very interesting! Also I think your mentioned downsides are minimal.

@achow thanks for the thoughtful suggestion.

It's something I have considered and ultimately think is a bit worse, but as you note it has upsides too so I should clarify my reasoning so it can be properly evaluated by yourself and others.

Upsides:

• Simplifies lookups (scope is limited to one tx)
• Coinjoin no longer leaks input (also not an issue with blind ECDH, but questionable how practical this is)

Downsides:

• Coinjoin leaks change output to recipient*
• Indirectly may make it easier to identify the input (thanks @Kixunil, had not previously considered this)
• Doesn't work if there are no change outputs
• Increases number of required ECDH calculations (by ~2x) compared to when you add all input keys together (1 per output vs. 1 per tx)
• Sender restricted to using taproot for change output (for coinjoins this would already be the case, otherwise the change output stands out)

So overall I think you move the coinjoin info leakage problem from the input side to the change output side, you lose the ability speed up ECDH by adding all relevant keys from the tx together prior to calculating the shared secret, and you lose the potential to at least deal with the leakage problem in theory via blind ECDH. I currently feel this does not weigh against the one upside, which is a reduction in lookups, particularly since there may be reasonable ways to deal with it, such as trading off disk IO for disk space with an additional input database.

*Note that this is a problem since the notion of dirty change outputs does not apply to all variants of coinjoin (e.g. Wasabi 2.0).

Shouldn't blind ECDH be still possible with outputs? At least if the participants are not sending to someone else (cold storage). So still worse I guess but not that much.

@Kixunil if all payments go to the participants in the coinjoin, then there would be no reason to use silent payments as you're already interacting. But there is a more fundamental problem: you'd need to add all outputs together and use that to generate the tweaked silent payment output, but that would mean you're adding the resulting output to itself – a circular reference.

Ah, yes, good point!

@prusnak wrote:

Yeah. For Trezor we would use both the scanning key and the address reuse prevention. If others are fine with not having these two flags configurable, so am I and we could just use 00 || X_spend || X_scan and mandate address reuse prevention.

I agree with this. BIP47 tried to leave too many things configurable and it created implementation complexity and fragmentation (Bitmessage notification etc.)

Regarding the payment code format, the sp1 prefix looks fine.

One issue is that neither BIP47 nor SP say anything about recipient address types. We should define a bitflag array denoting script types that the recipient is scanning for. As the number of standard script types increases with time, it'll become increasingly cumbersome to scan for all of them. If we go with two bytes for this purpose, we could have the following bitflags:

1	p2pkh
2	p2sh-p2wpkh
4	p2wpkh
8	p2tr

So in terms of byte structure the payment code format could look something like:

[0] - spend/scan flags
[1:3] - address type flag array (big endian)
[3:36] - compressed pubkey

Another issue is how a SP key is derived. I believe this shouldn't be left to individual implementations since there will be no standard way to reconstruct a SP private key on a different device. We probably need a HD path, for instance m / [BIP_NUMBER]' / 0' / 0'.

Performance considerations

While there is no way to remove the need for checking every transaction for a payment, there seems to be a way to greatly speed it up.

A SP transaction could set the nLockTime field to a low fixed constant (or some modulus), rendering it meaningless in terms of actual lock times, thereby repurposing it for scanning purposes. Since this field is at a fixed offset in a transaction, indexing into it is fast. This would limit the need to perform hash(i*X)*G + X to a small subset of transactions in a block.

The downside is that this would potentially signal that a payment is potentially a silent payment, but there would be no way to know for sure.

@alfred-hodler SP simply mandates p2tr, not need to specify. But you're onto something. We should have feature bits to enable future scripts or extensions. Version means incompatible upgrade. Perhaps some key-value extension would be useful as well.

Thanks for the clarification. I think mandating p2tr makes little sense as it's not a replacement for p2(w)pkh. Adding address type bitflags is cheap and it'll prevent the BIP from falling out of favor if p2tr ever becomes a "legacy" script.

Not having addressed this issue, BIP47 forces users to watch all address types, which is getting increasingly expensive.

@alfred-hodler I misremembered that only P2TR can technically work but others can, so yeah, it'd be nice but maybe skip some sad types and support only (native?) witness. I agree bit flags should be added anyway even if currently they only set one type.

I think mandating p2tr makes little sense as it's not a replacement for p2(w)pkh.

Isn't it?

Isn't it?

Taproot is just a new standard script type. There's nothing in the update that deprecates other standard scripts. Besides, if Segwit is anything to go by, many wallets won't have taproot support for a while. We don't want to lock them out of using SP.

At any rate, we should avoid getting bogged down in these kinds of discussions. It's all about giving choice and freedom to the end user.

@alfred-hodler too much freedom when implementing a protocol can lead to people deciding not to implement a complicated protocol. I also doubt that wallets will rather spend time on implementing somewhat complicated SP than adding support for already-deployed soft fork which is seems simpler (definitely simpler on sender side). Incentivizing people to switch to P2TR is also interesting because of the privacy properties. That being said I wouldn't mind supporting native SegWit v0.

I agree with all that in principle, I pointed out the same thing re BIP47 and its various splintered notification types.

However, supporting existing and widely used script types doesn't create extra implementation complexity. I think we agree that Segwit v0 should be supported, along with probably legacy p2pkh.

As mentioned earlier, not having script type bitflags recreates the biggest issue that BIP47 suffers from -- being unable to use new script types in the future without having to scan for all of them. If you look at the way Paynym directory entries look, they communicate out-of-band whether certain payment codes can process Segwit or not. We can inexpensively solve that issue.

I'm also curious to hear thoughts on this earlier proposal regarding scanning performance.

I just want to correct this statement:

they communicate out-of-band whether certain payment codes can process Segwit or not.

It is not out-of-band. Samourai Wallet uses non-standard (not defined in BIP) extension of BIP47 where segwit-compatibility is signalled in the payment code, in byte 79 to be precise. See code for reference

There is also an existing PR from straylight to extend existing BIP with this functionality.

@alfred-hodler I definitely agree with bitflags and I propose to also make them usable for other future features.

@alfred-hodler I also agree with bitflags for future extensibility, but see little reason to support anything other than taproot outputs for the reasons that @Kixunil mentioned.

As for reducing scanning costs, various people have made similar suggestions, but it doesn't seem wise to me. Any such reduction in scanning also reduces the anonymity set and this seems to be quite damaging for privacy. Imagine you're spending multiple received silent payments in the same transaction. The nlocktime of the inputs (e.g. all the same modulo) will now trivially give away that a.) you're likely using SP, and b.) all inputs likely belong to the same person.

@achow101 I replied to you earlier but see now that I tagged the wrong person, my bad.

@RubenSomsen I agree with you regarding reducing scanning costs. It occurred to me that it would end up damaging privacy but I wanted a second opinion. We can go with the original proposal of scanning the entire transaction set.

Regarding taproot scripts only, I disagree. Silent Payments is a chance to do BIP47 correctly and fix a lot of its mistakes. The principal goal here should be to create something that helps increase the amount of privacy in the world, and in order to do so the standard has to be reasonably inclusive and allow older/lagging wallets to use it. The goal of "incentivizing" taproot use is nice but it should be secondary to the stated primary goal. In other words, I don't think this BIP should concern itself with evangelizing taproot at the expense of improving privacy.

Let's have a look at output types over the past 90 days:

Pubkey Hash 47.8%
Script Hash 24.8%
SegWit v0 Pubkey Hash 26.1%
SegWit v0 Script Hash 1.4%

Even 5 years after its activation, Segwit usage is at ~ one quarter. Wallets take time to update, companies need time to build first consensus and then infrastructure and so forth. Whether we like it or not, that's simply the reality we live in.

So my question is really: what is the benefit in not allowing p2(w)pkh addresses?

@alfred-hodler

what is the benefit in not allowing p2(w)pkh addresses?

• Keeping the implementation simple
• Incentivizing taproot use which ultimately improves Bitcoin's privacy
• Bootstrapping with a low initial scanning requirement since any tx without a taproot output can be skipped

In my opinion there also isn't that much value in staying compatible with old wallets when being a SP recipient already requires a pretty advanced wallet that can handle scanning. And since people will have to create a new wallet with a new address type anyway, this seems like the perfect moment for people to switch to taproot.

I do acknowledge that low usage of taproot means the anonymity set is initially low. This is probably the strongest argument against it. It's sort of a chicken and egg problem. It'd be much better if everyone was on taproot, but as long as people are not, nobody wants to be first.

As long as we have the bitflags we can always define a flag at a later stage to add support if it's in high demand.

@RubenSomsen

Keeping the implementation simple

Most Bitcoin libraries have something like Address::p2wpkh(pubkey) or Address::p2tr(pubkey). Address creation from pubkey is trivial these days and adds no special complexity. Besides, by allowing only Taproot initially with possible later expansion, you're getting the same supposed complexity but later on.

Incentivizing taproot use which ultimately improves Bitcoin's privacy

As I said, I don't think this BIP should concern itself with evangelizing Taproot. Taproot has its own set of incentives and using borderline coercion ("Use this script type we like or no added privacy for you") isn't the correct approach.

Bootstrapping with a low initial scanning requirement since any tx without a taproot output can be skipped

As you probably noticed yourself, this sort of recreates the same problem as my earlier proposal of using nLockTime to help with scanning. Not quite on the same scale (especially as Taproot usage increases), but it's not a valid reason to force the use of Taproot.

I believe supporting p2pkh and p2wpkh comes at no special cost and if the BIP doesn't do that for political or other non-technical reasons, that'll probably be a NACK from many people in the community, and we're likely to end up with more Stealth Address / BIP47/ SP derivatives down the road simply because we were unable to compromise a little and make these standards more inclusive from the get go.

As long as we have the bitflags we can always define a flag at a later stage to add support if it's in high demand.

And how are you going to measure this demand? Do we expect people to see the standard and then go to the mailing list or to open a PR in the bips repo? More likely than not, most people will just ignore the standard if it doesn't serve their needs.

@alfred-hodler As I already mentioned, thus far the small taproot anonymity set seems to be the one strong argument in favor of supporting older formats. If you're hoping to convince me specifically, I suggest this is what you focus on.

this sort of recreates the same problem as my earlier proposal of using nLockTime to help with scanning

It's not quite the same. Taproot use is expected to increase over time and SP can add to that growth, whereas marking SP transactions (with nlocktime or otherwise) creates completely new anonymity sets that don't add to anything.

If we only support taproot, at least we ensure that SP users will be in the same anonymity set with each other. And in general taproot has a lot of potential to unify anonymity sets, as most use cases and complex scripts can now be satisfied with an indistinguishable key path spend.

probably be a NACK from many people in the community

Then I hope to hear from these people, so we can assess their arguments.

if the BIP doesn't do that for political or other non-technical reasons

If you want this conversation to be constructive, I suggest you stick to making arguments as opposed to dismissing what is said as "political or non-technical".

Do we expect people to see the standard and then go to the mailing list or to open a PR in the bips repo?

The standard is not even defined yet. Again, not very constructive.

Note I'm not interested in having an adversarial debate. In my opinion the goal should be to come to a mutual understanding of what tradeoffs seem best. I hope we can do this in good faith, otherwise I don't see how this conversation will end up with a productive outcome.

@RubenSomsen I agree that we should keep the conversation constructive.

It is my belief that allowing two bits to signal that pubkeyhash addresses are accepted isn't that big a deal. My concern is that if the standard is too restrictive, it'll suffer from the same fate as BIP47 -- new standards will attempt to supplant it because it doesn't serve the needs of enough entities and then we just get more fragmentation. I work for a fairly large business and I haven't seen any roadmaps to enable taproot at any level (as of yet). But integrating SP using pubkeyhash addresses probably wouldn't be too hard a sell internally. Another argument is that Bitcoin is all about freedom, and mandating certain requirements if there isn't a strict technical limitation doesn't strike me as being in the spirit of Bitcoin.

Hope that makes sense. If not, I'm out of further arguments so I'll leave it at that.

@alfred-hodler

I agree that we should keep the conversation constructive

Glad to hear it, then I hope you'll continue to provide input.

new standards will attempt to supplant it because it doesn't serve the needs of enough entities

That seems like a potentially valid concern to me. I think the path to avoiding this (for as far as we can) probably starts with understanding those needs.

I haven't seen any roadmaps to enable taproot at any level (as of yet). But integrating SP using pubkeyhash addresses probably wouldn't be too hard a sell internally

I think it could be valuable if you could elaborate on this, as this is surprising to me. The scanning requirement of SP seems like much more of an implementation barrier than adding support for taproot.

Another argument is that Bitcoin is all about freedom

I understand and agree with the spirit of the statement, but in this context that seems like a circular argument. For instance, I assume you'd agree with me the specifications shouldn't add support for near-deprecated formats like p2pk even though it'd provide more "freedom". So really the argument is not about freedom, but about supporting what is useful. But whether it is useful to support things other than taproot is exactly what our discussion is about. This argument presumes that conclusion, so it cannot count in favor of it.

@RubenSomsen

I think it could be valuable if you could elaborate on this, as this is surprising to me. The scanning requirement of SP seems like much more of an implementation barrier than adding support for taproot.

I actually don't think scanning is that big a hurdle if we are talking about large businesses. These systems are fairly large and there would likely be a secondary process/system that receives entire blocks and tries to detect SP payments. Scanning is only an issue for small underpowered computers like phones and older Raspberry Pis. It's spending that's harder as the company now has to be able to spend a new script type that it hasn't integrated into its treasury infrastructure yet. This might sound inane, but it's just the reality of how these things move. There was a big exchange, I think Binance, that took several years to be able to interact with Segwit addresses.

On the withdrawal side I can imagine a situation where a user has a wallet that supports SP but cannot use that due to his counterparty not having taproot support yet, thus being unable to calculate SP addresses and therefore integrate the standard in the first place.

I assume you'd agree with me the specifications shouldn't add support for near-deprecated formats like p2pk even though it'd provide more "freedom".

I disagree. While I don't think anyone should have a valid reason for using p2pk addresses at this point in time, if they want to (for any reason), they should be able to. As long as a transaction is included in a block, it's valid from the consensus perspective and I prefer not to make judgements as to whether people should be allowed to use it. That being said, excluding p2pk, wrapped Segwit and other "legacy" script types is a compromise I can live with. But from the standpoint of usability, excluding p2(w)pkh will stunt the standard too much.

The fact that we're 5 years in and almost 3/4 of all transcations are still non-Segwit (and Segwit has some serious incentives), I wouldn't be too optimistic about the prospect of tying the success of a privacy standard to the adoption of Taproot.

@alfred-hodler

I actually don't think scanning is that big a hurdle if we are talking about large businesses

I don't fully know what model you have in mind, but keep in mind that the scanning cost increases linearly with the number of static SP addresses you generate, so if you want to make a separate key for lots of users, this would be a significant amount of effort. I'm also wondering if anything would be gained from this, as it's easy to interact with a custodian while SP is mainly useful for non-interactive scenarios.

Maybe you could clarify what exactly the model of your company is, otherwise I can't evaluate if any of this makes sense.

On the withdrawal side I can imagine a situation where a user has a wallet that supports SP but cannot use that due to his counterparty not having taproot support yet

This confuses me as well, could you clarify? It sounds like you're saying the sender supports SP but the recipient does not. The recipient chooses how they receive the money, so this wouldn't be an issue. It would just be a regular payment.

from the standpoint of usability, excluding p2(w)pkh will stunt the standard too much.

Yes, the above is your argument, and it could be valid if p2(w)pkh + SP is indeed in demand (which is unclear to me at the moment). My point was that this argument is in fact not primarily about "freedom".

But I think right now the most important thing is that you say you have a business use case that could benefit from having SP + p2(w)pkh as opposed to SP + taproot, so I'd like to understand this. If the use case makes sense, it could be a strong argument in favor of your point, and it seems to be what motivates you.

PR #24897 has been updated with a new silent payment version, which eliminates some manual steps from the previous version (such as the need to set the keypool to avoid costly multi-key scan).

This is achieved by using a new descriptor type (sp()) that has no range and contains exactly one key.

Example: sp(cQq73sG9....JD51uaRD)#9llg6xjm

This descriptor introduces a new type of output: silent-payment. This output type returns a standard Taproot script (Segwit V1), but with HRP changed from bc to sp on the mainnet (or tsp on testnet and signet).

This output type will always generate the same address (unless another sp descriptor is enabled on the same wallet).

$ ./src/bitcoin-cli -signet getnewaddress '' 'silent-payment'
tsp1pfmjyl7ecpmx8yf8cu6g3ez36jy7s9mzuh5pdnal3k0n588uzgmfs4s4fws

To create a silent transaction, simply use the silent payment address as one of the outputs.
The send RPC will automatically identify and tweak it.

The transaction can contain multiple outputs, combining silent and standard addresses.

I have written a step by step signet tutorial so reviewers can test this new version easily.
https://gist.github.com/w0xlt/a7b498ac1ff14b8c292a22be789bd93f

I tested the Silent Payment PR and it works great !

One feature of Silent Payment is that the recipient cannot identify who send him bitcoins just with the Silent Payment transaction. This is great for donors privacy but it makes it hard to use by exchanges or services to receive deposits.

I think it would be nice for the sender to be able to inform the recipient in the transaction who paid him (pseudonymous). I found two ways of doing it inside the silent payment transaction, each solution has its trade-off:

• The sender can add an OP_RETURN output which contains sender payment code blinded with a value derived from the shared secret x*hash(txid,index)*I
• The sender uses a Taproot change address with publickey P_change = HMAC(x*hash(txid,index)*I, CHANGE_CONSTANT)*G + Y where Y would be the public key of the sender payment code, the recipient can identify the sender by computing the difference
  P_change - HMAC(x*hash(txid,index)*I, CHANGE_CONSTANT)*G
CHANGE_CONSTANT is just a protocol key for HMAC to derive a blinding key tweak from shared secret so that no relation can be deduced from X', P_change and Y without knowing x or inputs private keys.

This second solution has the advantage to still look like a standard transaction while providing sender identification. However it makes backup harder to handle for the sender with just the words seed: if he didn't move the funds in the change output, he has to remember that he made a payment to a payment code with public key X to be able to claim funds back. Also, I think that using a scanning key is valuable but the change output can only be used to reveal one public key unless we add another output to the second public key (blinded by shared secret).

The OP_RETURN data don't have to be a spendable public key so we are free to include full sender's payment code blinded with shared secret (scanning key or not) and sender can still claim all his funds with the seed backup. TheOP_RETURN can also include a notification code part to improve scanning. Obviously, the drawback is that it is an OP_RETURN so we may detect it is a silent payment looking at the chain, but it doesn't reveal more than that.

We could imagine that the recipient service indicates in his payment code that he expects his users to identify themself with such protocol (with OP_RETURN or change) so that sender's wallet do it only if required by recipient to make the UX simpler (and it should be possible for both to derive addresses from both payment codes for future payment without any need of identification)

Silent Payment is a great solution to the notification problem, it would be nice to solve the sender's identification too, maybe someone has a better idea ? :)

Personnaly, I find adding an OP_RETURN in the silent payment to be the best tradeoff.

@Pantamis, thanks for examining the implementation and sharing your thoughts.

great for donors privacy but it makes it hard to use by exchanges or services to receive deposits

In my view, any service that you can interact with shouldn't be using a non-interactive protocol. The right model for such use cases is for the service to simply hand you an xpub.

Also note that @w0xlt just added support for an identifier which allows you to do distinguish the payment purpose (but it still won't let you identify who paid you). See this comment and the discussion here.

I find adding an OP_RETURN in the silent payment to be the best tradeoff

Once we start adding OP_RETURN data, it seems to me like we'd end up losing some of the defining features of the protocol (no extra overhead, indistinguishable from regular payments). If we did make that tradeoff, a safe one-time method of communicating an xpub as opposed to adding an OP_RETURN to each individual payment seems preferable, such as this variant of BIP47.

Something that does seem reasonable is for the sender to optionally identify themselves out of band to the recipient after the payment was made, though keep in mind the potential failure case where a payment is made but you're unable to contact the recipient.

I spent some time capturing the proposed protocol as jupyter notebook using this Bitcoin DSL I have been working on. It helped me concretely grok the proposal and hope it will help others too. Dropping the link here in case it is useful: https://opdup.com/bitcoin-dsl/examples/silent_payments.html

Re scans. I wonder if we should think of a solution outside the bitcoin node? There are some merits in running a process with its own separate database of input pubkeys and their spending taproot internal keys. With such a setup, we can optimise the db scans and don't need to complicate the bitcoin node source code. Given we'll be running sequential scans here most of the time, we can tune the db setup for such a work load. We can even evaluate writing stored procedures to run inside the database - further optimising on data copies etc. Another advantage is we can keep evolving this separate indexing process as we need to, without requiring patches to bitcoin source.

The above separate process and database setup is akin to the electrum server setup.

@pool2win nice work. I'm sure it'll help people. Be sure to also check out the BIP, it has a lot of new additions.

While I agree that conceptually it'd be nice to have the scanning take place outside of Bitcoin Core, in practice it has been difficult to do this efficiently. The reason for this is that scanning a block for payments requires knowledge of the scripts that relate to the all the inputs. To my knowledge the only performant options are to fetch these scripts from the UTXO set or the rev*.dat files (this contains spent UTXOs on a per block basis, allowing us to roll back the chain). In theory an application can be built that reads the latter data from disk, but that's a significant undertaking.