Skip to content

Instantly share code, notes, and snippets.

@achow101
Last active October 29, 2020 01:55
Show Gist options
  • Star 6 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save achow101/94d889715afd49181f8efdca1f9faa25 to your computer and use it in GitHub Desktop.
Save achow101/94d889715afd49181f8efdca1f9faa25 to your computer and use it in GitHub Desktop.

A Document About Native Descriptor Wallets

We'll discuss issues with legacy wallets and how descriptor wallets specifically solves them as well as their limitations.

Legacy wallet problems

Key based

In a legacy wallet, everything is based around the keys. Keys are generated, and from those keys, we produce scripts. So we can really only natively support and generate single key things. But we also only have a single keychain that is used for every address type. So you could have the first key be bech32, but the next key in the chain is p2sh-segwit. Except this distinction isn't even in the wallet itself. It's only a user facing thing. What actually happens is that the wallet will be watching all address types for a given key. So even if you requested a bech32 address, the corresponding p2sh-segwit and legacy addresses for the same key will still be watched and coins can be sent to them.

Watchonly

In addition to our single key stuff, we also have added this watchonly thing into legacy wallets which allow for other scripts to be watched. This watchonly thing makes things confusing for us though. The wallet ends up having two balances, one watchonly and one not watchonly. This is extra annoying because we have to put include_watchonly arguments everywhere. And forgetting to do that can cause headaches. But there's also no way to exclude non-watchonly things. So if you have watchonly stuff, and you set include_watchonly because you want just watchonly things, if your non-watchonly things have been used, you can end up having transactions and returned information include both watchonly and non-watchonly without necessarily knowing that. This is particularly bad when dealing with multisigs.

Multisig

A standard multisig workflow is to use addmultisigaddress to add a multisig into your wallet that involves at least one of your keys. So you have a wallet with a bunch of private keys, and a multisig. But don't forget that your private keys are still tied to single key addresses. And if you receive any transactions to those single key addresses, even if you never explicitly requested those addresses, the wallet will mark them as used and add those transactions to your wallet. The balance will change, and you might accidentally think that those coins sent to the single keys are part of the multisig because it's not always clear they aren't. So that's a problem.

Split key setup (including hardware wallets)

Then exporting your wallet to be used with other software, or even just neuter it to be purely watchonly. Even though the legacy wallet is BIP 32 wallet, we don't have any xpubs to export to another wallet because we use exclusively hardened derivation. So the only way to import watchonly into another wallet is to get every single address in your address and import them all into a wallet. But that's hard to do because there's no way to really export all the addresses. And when you run out of addresses, you gotta go back to core and get a bunch more. But then you also can't use a legacy wallet as the watching wallet because we can't do xpub derivation in watchonly. All we can do is import a bunch of addresses. This doesn't work for hardware wallets or watching other BIP 32 wallets.

Privkey export/restore

Lastly there's restoring your wallet and your wallet's private keys. Other wallets use mnemonics of some sort, and while they aren't perfect, at least they do allow for easily exportable and importable private keys. Well that just doesn't work with legacy wallets.

Descriptor Wallet Fixes

Key based

What actually matters in Bitcoin are the scriptPubKeys. So instead of being key based, we should be script based. Output script descriptors are the easiest way for us to do script based. A descriptor maps to exactly one type of script and provides all of the information necessary to solve for those scripts. So it's all script based, there's no guessing about what scripts to make from keys. This makes determining what is ours easier too. We just say any script that we have is ours and we consider any outputs with those scripts as scriptPubKey become part of our balance. This also means that keys in multisigs don't become ours and what is ours is completely unambiguous. This change also means that keys are tied to specific descriptors and are not being shared between all of the descriptors.

Watchonly

Descriptor wallet by itself doesn't specifically solve watchonly, but it's a good time to move us to a new wathconly model. Instead of having a wallet that has both private keys and scripts that its watching, we instead require wallets to either have private keys for its scripts, or the wallet has no private keys at all and only scripts. This means that we don't need to have these include_watching things and there's no confusion the mixed watched and non-watched things. We make use of the disable_private_keys feature that was added. Any wallet that has private keys disabled is considered a watchonly wallet. Even so, we don't use the old watchonly enums within that wallet.

Multisig

Because we moved to a script based world, multisigs are simply a descriptor which can be imported to the wallet. So the wallet generates a multisig script but the keys in the script don't matter and don't becomme ours. But there are a few limitations to this. Because of the above watchonly and script changes, you can't just import a normal public only multisig descriptor into a normal descriptor wallet. Either that multisig needs to have a private key (e.g. for your key) or you must import it into a watchonly wallet. But constructing this descriptor is not necessarily a good user experience. In general though, this will require exporting private keys from a descriptor, in which case, we need to be using hardened derivation.

Split key setup

Split key setups are easier to do with descriptors becase we could just export watch only descriptors. These can be imported into another descriptor wallet. However such descriptors require unhardened dverivation. For a descriptor wallet to watch, we just need a descriptor and can watch the scripts produced by that descriptor.

Restore

Backup and restore can easily be done in the same way as split key setups, except with a descriptor containing private keys.

Current Implementation Limitations

The current implementation of descriptor wallets doesn't quite do all things that were just mentioned. It has a few limitations, listed here:

  • We currently cannot export descriptors of any form. So currently making split key setups and backup/restore won't work.
  • Related to exports not being implmented, there's no way to create a multisig descriptor that has a private key. This means that traditional multisig workflows using a single wallet and signrawtransactionwithwallet won't work. Two wallets are needed and PSBT must be used for now.
  • By default we only make descriptors that use unhardened derivation. So implementing private key export also has a couple UX issues.
@ryanofsky
Copy link

Descriptor wallet by itself doesn't specifically solve watchonly, but it's a good time to move us to a new wathconly model.

IIRC, the "watchonly wallet" model described here is different than the model sipa used to talk about. Obviously, the pre-descriptor model where balances depended on the wallet knowing keys was a mess. But instead of having "watchonly wallets" and "non-watchonly wallets" I believe the original descriptor plan gave each descriptor in a wallet a watchonly bool field, so coins matching watchonly=false descriptors would straightforwardly count to your own balance, and coins matching watchonly=true descriptors would count to your watchonly balance. This would all be independent of having private keys, avoiding the confusion in the legacy wallet.

The current "watchonly wallet" model described here where no descriptors in the wallet can be watchonly or every descriptor must be is simpler, because watchonly RPC options can be dropped and the wallet can display one balance instead of two. But are there downsides to this model?

In IRC sipa was saying "we just need a good way to import a multisig descriptor + individual key into a descriptor wallet". I wonder if in this kind of wallet, ability to mark individual descriptors watchonly or not, ability to display two balances, and ability to have RPCs that know which descriptors are intended for signing regardless of whether private keys are present might help with UX, and maybe let someone get away with just having have one bitcoin wallet instead of two and having to exporting/import between them.

Probably the answer is no, having watchonly and non-watchonly descriptors in the same wallet is useless for everything except displaying two balances. Just curious for some confirmation or intuition here.

@instagibbs
Copy link

I wonder if in this kind of wallet, ability to mark individual descriptors watchonly or not, ability to display two balances, and ability to have RPCs that know which descriptors are intended for signing regardless of whether private keys are present might help with UX, and maybe let someone get away with just having have one bitcoin wallet instead of two and having to exporting/import between them.

I agree, for the reasons laid out here bitcoin/bitcoin#16528 (comment) . If we want a single-wallet solution out the gate we should make them act sane.

@achow101
Copy link
Author

IIRC, the "watchonly wallet" model described here is different than the model sipa used to talk about. Obviously, the pre-descriptor model where balances depended on the wallet knowing keys was a mess. But instead of having "watchonly wallets" and "non-watchonly wallets" I believe the original descriptor plan gave each descriptor in a wallet a watchonly bool field, so coins matching watchonly=false descriptors would straightforwardly count to your own balance, and coins matching watchonly=true descriptors would count to your watchonly balance.

I never interpreted the watchonly discussions in that way. To me, it was always having separate wallets.

I also don't really like doing this because it still ends up with a mixed wallet, just less so than legacy wallets. So you can still make mistakes by forgetting the include_watchonly parameter (which I guess would just be a "watchonly only" and not just include).

@jonatack
Copy link

Adding this relevant wallet meeting discussion from April 10, 2020
#topic watchonly and descriptor wallets
http://www.erisian.com.au/bitcoin-core-dev/log-2020-04-10.html#l-425

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment