What follows is an architecture that satisfies the Project Requirements outlined in the Bitcoin Full Node Provider and Wallet bounty
The recommended way to interface with Bitcoin is through a full node, so this architecture involves the implementation of an interface between an Urbit ship and a full node. Any ship that connects to a full node in this way can provide this service to other ships that do not run their own full node, making the providing ship a full node provider (referred henceforth as just provider). Access via providers to the bitcoin network requires trusting the provider with only derived public addresses, meaning that little identifying information ever needs to be shared with the provider.
This approach has the advantage of catering to Bitcoin maximalists that want to run their own full nodes, but also to the security and privacy-conscious Bitcoin user that does not have the resources to administer their own full node. The amount of trust given to a provider is small, and the high trust within the Urbit network makes this even more palatable.
The proposed architecture is broken into two high-level components:
- Bitcoin Full Node Provider: Provides full node access to its host ship, and possibly to other ships with a scheme determined by the host.
- Bitcoin Wallet: Initiates payment requests and manages known addresses (ours and others').
This proposal is probably incomplete. The worker should consider this a good starting point for beginning the implementation, but know that modifications can and should be made as the project develops.
The full node provider allows Urbit ships to communicate with a Bitcoin full node. Any ship operator that runs a full node and connects their ship to it will be referred to as a host. A host can become a provider by providing access to the full node via gall agent(s) to other ships. At minimum, the host provides access to itself (a host is a trivial provider), but should be able to provide access to other ships as well.
The full node provider should satisfy the following user stories:
- As a ship, I can connect to a full node that I control, becoming a host.
- As a ship, I can connect to a full node via another ship, becoming a client.
- As a host, I can control which other ships can access my full node.
- As a host or client, I can check the balance of my Bitcoin addresses.
- As a host or client, I can broadcast a signed transaction to the Bitcoin network.
- As a host or client, I can obtain a list of transactions for a specified Bitcoin address.
The bitcoin provider connects to the full node and uses its JSON RPC API to communicate. Its role is to talk to the bitcoin network directly, therefore all of its communication should be restricted to the host ship.
connect: connect to a full node at a specific IP & port along with required authentication or other connection settings.status: retrieve the connection status of the bitcoin node.create-wallet: create a watch-only wallet for the provided address.balance: look up the balance for the provided address.transactions: produce all transactions for the provided address.broadcast-transaction: broadcast a signed transaction to the bitcoin network.
The bitcoin provider hook handles the connection to the full node provider in one of two capacities:
%local: the full node runs on our ship, or%remote: the full node runs on another ship.
Its role is to manage the connection between ship(s) and full node. It will define and enforce access controls, perform initial and ongoing setup when full node connection statuses change, and map ships to bitcoin addresses.
If a ship has a %local connection to a full node, the hook will communicate
directly with the bitcoin-provider on that ship. If the connection is
%remote on the other hand, communication will instead be proxied to the remote
bitcoin-provider-hook, whereby they will be resolved on the local
bitcoin-provider and then relayed back to the point of origination.
connect: given a ship to connect to, attempt to establish a connection to its provider store.set-access: determines which ships can interact with the hook. Can only be set to a value other than %host if the connection type is %local.%host(default): only the host%kids: only kids of this ship (e.g. moons, ships sponsored)%ships: only specified ships%all:any ship
access-level: current level of access- responds with null if the requesting ship is not within current level of access or is not running a node.
add-addresses: create watch-only wallets for the source ship and provided bitcoin addresses.balance: look up the balance for the source ship or specific address.%ship: produce the balance of each address known for source ship. Known addresses are those that watch-only wallets have been created for.%address: given an address, produce the balance.
transactions: produce all transactions for the provided address.broadcast-transaction: broadcast a signed transaction to the bitcoin network.
The bitcoin-provider is strictly an interface to a locally-connected full
node. You can think of it as a wrapper around the RPC API that provides a
domain-specific API to the Bitcoin network.
The bitcoin-provider-hook is how ships communicate with with a
bitcoin-provider, whether locally or remotely. The hook serves primarily to
manage access to a bitcoin node and field requests from foreign ships to
communicate with that node.
That these two concerns be separated into different agents is less important than having the semantics around local/remote connections and permissions preserved.
The wallet's responsibilities include facilitating payment between two parties and managing the addresses required to do so.
When receiving Bitcoin from another ship, a new address should be generated for just that transaction. This prevents anyone other than the owner from knowing the receiver's total balance, and also provides some beneficial accounting properties as well. See the BIP32 proposal for more detail on the motivation behind this mechanism. All that's needed to derive a new address is an extended public key (or xpub for short). An extended public key can come from any pre-existing Bitcoin wallet, e.g. a Hardware wallet.
The xpub key corresponds to a single Bitcoin “account” within the wallet, and should only be stored on its owner’s ship. The owning ship will use its xpub to derive subsequent addresses for transactions, revealing only derived addresses to the outside world. Given the set of derived addresses we can determine the overall account balance and transaction history:
- The balance of an account is the sum of balances across all addresses derived from that account.
- The transactions of an account are the concatenated list of transactions on each address derived from that account, sorted by timestamp.
When sending Bitcoin to another ship, the wallet should be able to use the address-level balances as inputs to a single transaction. For example, if account A has the following addresses and balances:
- Ax: 0.45BTC
- Ay: 0.2BTC
- Az: 0.01BTC
Then an attempt to send 0.6 BTC to account B should use the balance of Ax and Ay as inputs to the transaction.
To initiate payment (sending or receiving), a transaction is constructed that
includes the addresses of the sender and recipient, amount to send in satoshis,
and transaction fee parameters--this transaction is then signed using the
corresponding private key and broadcast to the Bitcoin network via the
bitcoin-provider-hook.
NOTE: The Bitcoin Core implementation provides facilities for constructing transactions and performing coin selection. Ideally Bitcoin Core should be leveraged wherever possible, but the architecture of this system may not allow us to leverage Bitcoin Core for all of these functions.
The design of this wallet system closely resembles the signing-only wallet described in the Bitcoin docs. The signing-only wallet corresponds to a hardware wallet or master ticket, while the networked wallet is what is being outlined here.
The wallet should satisfy the following user stories:
- As a ship, I can see my overall balance for an account.
- As a ship, I can see all transactions for a specific account.
- As a ship, I can send bitcoin to other ships, becoming a payer.
- As a payer, I can tell when my payment has succeeded.
- As a ship, I can receive bitcoin from other ships, becoming a payee.
- As a payee, I can provide unique address for each transaction.
- As a payee, I can tell when I have received payment.
The wallet store is responsible for holding extended public key(s) and keeping track of addresses that have been derived from xpubs, both ours and others'. A mapping of ship to address will be required to match transaction histories against Urbit ships, of which only the parties involved in a transaction need to be aware of.
Another possible use-case of the wallet store is as a cache of balance and transaction data. The bitcoin network should be queried for updated information regularly (a way for the bitcoin provider to publish this information to the appropriate ship(s) would be ideal), but the responses can be saved locally for quick presentation while awaiting updated information from the Bitcoin network.
create-account: register a new extended public key from which to derive addresses from. Other metadata can be provided, like aname, that can be used to identify this account in the future.- The wallet can be implemented such that only one xpub can be saved at a time, or so that more can be stored. Storing multiple xpubs has implications for the API that will need to be thought through.
derive-address: given an account, derive a new address for their transaction.addresses: produce all addresses derived from an account.accounts/account: produce all known accounts.construct-transaction: given an account, destination address and amount (and transaction fee inputs), determine the appropriate address(es) to use to send payment to the destination address.
The wallet hook combines the bitcoin-provider-hook and bitcoin-wallet-store.
When a foreign ship wishes to pay our ship, it will poke the
bitcoin-wallet-hook with the request, at which point a new address is derived
using the bitcoin-wallet-store and saved in application state along with
relevant metadata. Then, the hook will create a watch-only wallet using the
bitcoin-provider-hook to enable efficient fetching of transaction history and
balance information.
request-address: request a new address to be derived (remote)- in the case of multiple known accounts, one will have to be chosen as a "default"
derive-address: derive a new address given a specific account (local)
I love this phrasing, even if it sounds sort of job-like, because it makes the UF sound like an agency or an institute.
I don't know the details of Bitcoin implementations but I think the spec here is pretty good? — If only because you have user stories, and I have yet to see those in a bounty.