This document outlines some of the upcoming Bitcoin Core 0.21 release changes and provides steps on how to test these changes.
The release candidate for version 0.21 was just tagged and is ready for testing. And, oh boy, is 0.21 the right time for you to get involved as a tester. It’s jam-packed with changes that need to be run on your operating system with your hardware. Database changes? Welcome SQLite. A new network to test on? Hello, Signet. Have you heard that Tor v2 is being deprecated? The upgrade to Tor v3 is in 0.21. What about the wallet? Total re-write.
You can get involved by running through this guide and checking that everything works as it should on your machine. Please report back your findings here. If everything went smoothly, let us know. If everything broke, definitely let us know!
Current Release Candidate: Bitcoin Core 0.21rc5 (changelog)
There are two ways to grab the latest release candidate: pre-compiled binary or source code. The source code for the latest release can be grabbed from here: latest release source code.
If you want to use a binary, make sure to grab the correct one for your system. There are individual binaries for macOS, Linux, Arm (64 bit), Arm (32 bit), and RISC-V.
If you grabbed a binary, skip this step.
Before compiling, make sure that your system has all the right dependencies installed. As this guide utilizes the Bitcoin Core GUI, you must compile support for the GUI and have the qt5 dependency already installed. To test the new wallet changes, make sure that you installed the sqlite3 dependency. Here are some guides to compile Bitcoin Core from source for UNIX/Linux, macOS, Windows, FreeBSD, NetBSD, and OpenBSD.
We will be creating and supplying a new data directory for our node to run from. Starting from the root of your Bitcoin release candidate directory, make yourself the new testing data directory with:
mkdir /tmp/21-rc-testThis will ensure you aren't using any old data and that you can easily access it. We'll nuke it when this is all over.
The Bitcoin Core 0.21 release introduces sweeping changes to the wallet to move towards a well-designed wallet, capable of full-utilization of Bitcoin. This release introduces descriptor wallets, a new type of wallet that generates addresses from descriptors instead of private keys. Tied together with this new wallet type is a new SQLite database that aims to replace the aging BerkeleyDB 4.8 database currently used.
What's wrong with the current (legacy) wallets?
The current wallet was designed when what Bitcoin could be used for was not yet fully understood. This led to a wallet design language that focused on maintaining a collection of private keys. As Bitcoin has progressed, this design language has held back the wallet from fully utilizing the expressiveness of Bitcoin Script. New features have had to be hacked on to the wallet.
Why the Switch to SQLite?
BerkeleyDB 4.8 is 10 years old. This database is not actively maintained, not meant to be used as an application database, and is susceptible to file corruptions. Since the move to descriptor wallets introducing breaking compatibility changes, SQLite was chosen as a new database because it provides certain guarantees necessary for ensuring that the wallet remains backwards compatible moving forward. Furthermore, unlike BerkeleyDB 4.8, SQLite allows us to have one file wallet instead of a wallet directory. This helps with wallet portability.
AChow101 (the guy that wrote most of the code you are about to test) wrote up some details about descriptor wallets if you want to learn more.
If you grabbed the binary for this release candidate, you're good to go. If you went down the source route, it is required that you installed the sqlite3 dependency and compiled the source code with wallet functionality. Skip this section if you intentionally do not want wallet functionality or don't want to test it.
Skip to section below if you'd prefer to use the command line.
We will now run bitcoin-qt and provide a data directory:
./src/qt/bitcoin-qt -datadir=/tmp/21-rc-test./bin/bitcoin-qt -datadir=/tmp/21-rc-testUpon start, a node no longer creates a wallet by default. We will need to create a new wallet. If you are starting up a node for the first time or using a fresh data directory (as we are), you will be met with the following screen:
Clicking on "Create a new Wallet" will bring you to the following screen. Give your wallet a name and make sure to have Descriptor Wallet enabled under Advanced Options. Hit Create. Congratulations, you've created your first descriptor wallet!
First, shut down your node using File->Quit. Then, navigate to your wallet's data directory and ensure that a wallets.dat file has been created under a directory with the value you supplied as Wallet Name. In the case of this example, it is my-descriptor-wallet. You should see something like this:
Reopen bitcoin-qt, then navigate to Window->Console. A console will pop up and in the text box enter getwalletinfo.
You should see something like:
{
"walletname": "my-descriptor-wallet",
"walletversion": 169900,
"format": "sqlite",
"balance": 0.00000000,
"unconfirmed_balance": 0.00000000,
"immature_balance": 0.00000000,
"txcount": 0,
"keypoolsize": 3000,
"keypoolsize_hd_internal": 3000,
"paytxfee": 0.00000000,
"private_keys_enabled": true,
"avoid_reuse": false,
"scanning": false,
"descriptors": true
}
See that last line?!? Descriptors true! 💃
You'll need two terminal windows. (Make sure you don't have the GUI running as you can't use it as the same time as bitcoind.)
In the first window, you'll need to start your node:
./src/bitcoind -datadir=/tmp/21-rc-test./bin/bitcoind -datadir=/tmp/21-rc-testAnd in the other, you can create your wallet:
./src/bitcoin-cli -datadir=/tmp/21-rc-test -named createwallet wallet_name="my-descriptor-wallet" descriptors=true./bin/bitcoin-cli -datadir=/tmp/21-rc-test -named createwallet wallet_name="my-descriptor-wallet" descriptors=trueWhile the logs fly by in that bitcoind window, once you've created a wallet you should see something like:
{
"name": "my-descriptor-wallet",
"warning": "Wallet is an experimental descriptor wallet"
}Now let's check it's what we want. In that wallet window add:
./src/bitcoin-cli -datadir=/tmp/21-rc-test -rpcwallet=my-descriptor-wallet getwalletinfo./bin/bitcoin-cli -datadir=/tmp/21-rc-test -rpcwallet=my-descriptor-wallet getwalletinfoWhich should result in something like:
{
"walletname": "my-descriptor-wallet",
"walletversion": 169900,
"format": "sqlite",
"balance": 0.00000000,
"unconfirmed_balance": 0.00000000,
"immature_balance": 0.00000000,
"txcount": 0,
"keypoolsize": 3000,
"keypoolsize_hd_internal": 3000,
"paytxfee": 0.00000000,
"private_keys_enabled": true,
"avoid_reuse": false,
"scanning": false,
"descriptors": true
}
If you see "descriptors": true in that last line, you are 💸. Welcome to the future!
Current nodes are limited to relaying addresses that fit into 128 bits. This limitation hinders Bitcoin nodes to a small set of network types. The upcoming Bitcoin Core release incorporates an implementation of BIP 155, which introduces a new P2P message that allows network nodes to gossip addresses that are longer than 128 bits. This opens up the possibility of running nodes on new network types such as I2P and Tor V3.
Why do we want to add compatibility for Tor v3 addresses?
Tor v2 addresses contain various vulnerabilities that expose a node to a variety of attacks. V2 addresses are also over a decade old, and they are scheduled to be retired by October 15, 2021. Tor Onion v3 addresses use a stronger encryption format that fixes some of v2's weaknesses.
What else do I need to know about this change?
Accommodating for the new address sizes makes the peers.dat backwards incompatible. A peers.dat file created with a 0.21 node will not be backwards compatible with a node <0.21.
Tor must be installed on your system for the Tor related tests to function properly. The script currently assumes that Tor is running on the default UNIX port of 9050. Below are some guides to setting up the tor package on your system.
The easiest way to install tor for MacOSX is to use homebrew. This package manager allows you to easily install packages right from the command line easily.
To install homebrew (if not already installed):
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Assuming that goes well, run:
brew install torAnd then, to get Tor running:
torFor those wanting to dig deeper, Bitcoin Core provides documentation on how to test running a node on Tor. There is little manual config to be done. In fact, on some Linux distros, if there is a Tor daemon running on the machine, bitcoind will pick it up and authenticate with a cookie file.
We are going to setup our node to listen on Tor automatically. This means that the node is going to look for other peers on the Tor network.
The bitcoin.conf file is used to configure how your node will run. This file is not automatically created and must be created manually. This file will be created in the data directory that we previously created while testing the wallet. From your data directory, run:
touch /tmp/21-rc-test/bitcoin.confWe will be adding settings to the bitcoin.conf that will allow us to connect and find peers through the Tor network. Since the Bitcoin Core 0.21 release has not actually been released yet, Torv3 nodes are rare to come across. Because of this, we are going to manually add a Torv3 node that has been tracked down. Using your favorite text editor, add the following to the newly created bitcoin.conf file:
proxy=127.0.0.1:9050 #If you use Windows, this could possibly be 127.0.0.1:9150 in some cases.
listen=1
bind=127.0.0.1
onlynet=onion
addnode=sxjbhmhob2xasx3vdsy5ke5j5jwecmh3ca4wbs7wf6sg4g2lm3mbszqd.onion:8333
addnode=rp7k2go3s5lyj3fnj6zn62ktarlrsft2ohlsxkyd7v3e3idqyptvread.onion:8333
addnode=d6jwdcoo2l3gbjps6asgg4nhp2gn5oao3wj333o43ssqnjaliehytfad.onion:8333(Cutting and pasting the text above should do the trick, but you can see the example bitcoin.conf file for an overview of the available options.
Ready to test? There are two ways to try this.
./src/bitcoind -datadir=/tmp/21-rc-test./bin/bitcoind -datadir=/tmp/21-rc-testYou will see a flurry of messages as the logs pass by. Open a new terminal window and let's query our running node to see who we've connected to:
./src/bitcoin-cli -datadir=/tmp/21-rc-test -netinfo 4./bin/bitcoin-cli -datadir=/tmp/21-rc-test -netinfo 4This should show you a list of peers. This is what your netinfo dashboard might look like:
Peer connections sorted by direction and min ping
<-> relay net mping ping send recv txn blk age id address version
out full onion 452 452 0 0 0 0 2 rp7k2go3s5lyj3fnj6zn62ktarlrsft2ohlsxkyd7v3e3idqyptvread.onion:8333 70016/Satoshi:21.99.0(Medea)/
out full onion 792 792 0 0 0 1 0 sxjbhmhob2xasx3vdsy5ke5j5jwecmh3ca4wbs7wf6sg4g2lm3mbszqd.onion:8333 70016/Satoshi:0.21.0/
out full onion 795 795 0 0 0 0 9 knbmm3arciehs6d7.onion:8333 70015/Satoshi:0.20.0/
out full onion 987 987 0 0 0 0 4 paot7ercftbiyb5o.onion:8333 70015/Satoshi:0.18.1/
out full onion 1090 1090 0 0 0 1 1 js5qbirosytw42jg.onion:8333 70015/Satoshi:0.20.1/
out full onion 1361 1361 20 17 0 6 4le6zxaxgstmk2w7.onion:8333 70015/Satoshi:0.20.1/
out full onion 1834 1834 1 1 0 0 7 thcyj2dhddbe45z6.onion:8333 70015/Satoshi:0.20.0/
out full onion 5230 5230 4 0 0 8 bvkbr4qudict6pv7.onion:8333 70015/Satoshi:0.20.0/
ms ms sec sec min min min
ipv4 ipv6 onion total block-relay
in 0 0 0 0 0
out 0 0 8 8 0
total 0 0 8 8 0
Local addresses: n/a
If you have rp7k2go3s5lyj3fnj6zn62ktarlrsft2ohlsxkyd7v3e3idqyptvread, sxjbhmhob2xasx3vdsy5ke5j5jwecmh3ca4wbs7wf6sg4g2lm3mbszqd, or d6jwdcoo2l3gbjps6asgg4nhp2gn5oao3wj333o43ssqnjaliehytfad in the response,
you've successfully connected to a Tor v3 node! 🎉
Launch bitcoin-qt and provide the data directory we have been using:
./src/qt/bitcoin-qt -datadir=/tmp/21-rc-test./bin/bitcoin-qt -datadir=/tmp/21-rc-testNavigate to and click on Window->Peers to bring up information on the connected Peers.
You can differentiate a Torv3 node from a Torv2 node by looking at how long it is. A Torv3 node is 56 characters long and always ends in a d such as: sxjbhmhob2xasx3vdsy5ke5j5jwecmh3ca4wbs7wf6sg4g2lm3mbszqd.onion. A Torv2 node is 16 characters long, such as: uovsp2yltnaojq6l.onion:8333. Your Peer window should look something like this:
We want to delete the bitcoin.conf in our data directory as we no longer need these configurations.
rm /tmp/21-rc-test/bitcoin.confThe Bitcoin testnet is a proof-of-work based testing framework where volunteers are relied on to mine blocks with real CPU power and in turn receive worthless testnet coins. Since the economics of the mainnet are not at play here, we get a network that is unpredictable and, frustratingly, unreliable.
This release introduces Signet, a new testing network. Signet does away with decentralized proof-of-work in favor of a centralized consensus mechanism where a group with authority is in charge of creating new blocks based on valid signatures. The aim is to create a testing network that is predictable and reliable.
The Bitcoin Wiki contains excellent documentation on connecting to and testing Signet. Building a custom signet is pretty involved but jumping on the default global signet should be just as fun. Give it a whirl!
An eclipse attack is an attack on bitcoin's p2p network. In order for the attack to be effective, the attacker aims to restart your node and then supply your node with IP addresses controlled by the attacker. Eclipse attacks reduce the soundness of second layer solutions such as the lightning network.
When you're node connects to the Bitcoin network, it makes two outbound block-relay-only connections. This release introduces Anchor Connections. Anchors are the two outbound block-relay connections your node is connected to; logged to an anchors.dat file so that they can be used upon a node restart. Under the assumption that you were connected to honest nodes before the attack, this aims to reduce the chances of an eclipse attack from being successful.
When a node shuts down cleanly, then an anchors.dat file should appear in the node's data directory. We want to check that this file is created upon node shut-down, and deleted on node start-up.
Start your node however you do so. If the release candidate is integrated into your desktop environment or is packaged into a .dmg in the case of macOS, launch bitcoin-qt from your application launcher. Otherwise, starting from the root directory of your binary or source download, run:
./src/qt/bitcoin-qt./bin/bitcoin-qtNavigate to and click on Window->Peers to bring up information on the connected Peers.
At the peer information page, visually check that you are connected to peers.
Shut down your node by navigating and clicking on File->Exit.
Navigate to the data directory for your node.
Restart your node, then navigate to your data directory. The image below is the data directory for a Bitcoin node while it is running. Notice that the anchors.dat file is missing. This is the expected behavior.
You've reached the end of the regularly scheduled programming. Take a little time to explore some of your recent use-case and/or revisit your favorite feature of QT or bitcoind (if any). Just poke around and make sure everything works as expected.
Use this command to remove all the data you played around with:
rm -rf /tmp/21-rc-test/Thank you for your help in making Bitcoin as robust as it can be. Please remember to add a comment on v0.21.0 testing issue detailing:
- Your hardware and operating system
- Which release candidate you tested and whether you compiled from source or used a binary (e.g. 0.21rc2 binary or 0.21rc3 compiled from source)
- What you tested (e.g., created a descriptor wallet, connected over Torv3, ran through the anchors test, your favorite features)
- Any other relevant findings
Don't be shy about leaving a comment even if everything worked as expected! We want to hear from you and so it doesn't count unless you leave some feedback.
Thanks for your contribution and for taking the time to make Bitcoin awesome.
In the tor v3 section:
testers may want to launch bitcoind with the
-debug=torconfiguration option to see useful info on the onion configuration./src/bitcoin-cli -datadir=/tmp/21-rc-test getpeerinfocould be replaced or complemented by(prefix that command with
watchif you are on linux), which is RPC getpeerinfo + RPC getnetworkinfo for humans, as well as being a new 0.21 feature worth knowing and testing