jcnelson/appchains.md

## appchains.md

      
    Raw
  

              appchains.md
            
          
    Revisions


Oct 19 2021: deployed MVP v2 of the appchain mining contract and updated the links and examples in this document to use it instead.

Introduction

Blockchains don't scale.  The fact that all nodes process all transactions means that the blockchain only goes as fast as the slowest node allowed on the network.  If that's something like a Raspberry Pi, then that's as fast as the blockchain goes.
And that's okay!  The upside is that the more people can run nodes, the more resilient the blockchain will be.  It's much harder to break a 10,000-node blockchain where most nodes run on home computers all across the world than a 10,000-node blockchain where most nodes run in a few datacenters.
But the demand for cheap transactions isn't going anywhere.  What's a distributed systems hacker to do?
A Composite Blockchain

A single blockchain can't handle unbound transaction volumes, but many blockchains can.  Just like how the world's most expensive high-powered mainframe cannot match the capacity of a cloud made out of cheap servers, a single high-powered blockchain cannot match the combined capacity of a linked network of many small blockchains.  This is especially true if adding new blockchains to the network is a permissionless act, since this would let the network grow with user demand.
Enter appchains.  The PoX consensus algorithm in Stacks lets it use the hashpower of Bitcoin to securely host its chain structure.  With a couple modifications, it also allows on instance of the Stacks blockchain to use another instance of the Stacks blockchain to host its chain structure as well!  PoX generalizes: many, many L1 blockchains may run at varying degress of separation between themselves and Bitcoin, but Bitcoin will commit to the chain structures of all of them.
So, now the world looks like this at each Bitcoin block:
                                                                              Degree-2 Appchain blocks
                                               Degree-1 Appchain blocks      /.---------------.
                                               /.----------------.          / |               |
Bitcoin blocks           Stacks blocks        / |====== tx ===== | <-------{  |               |
.--------------.      /.--------------.      /  |                |          \ |               |
|              |     / |              |     /   |                |           \`---------------`
|==== tx ======| <--{  |===== tx =====| <---\---`----------------`                    ^
|              |     \ |===== tx =====| <-----------/---.------------------.          |
`--------------`      \`--------------`             \   |                  |          |
       ^                      ^                      \  |                  |          |
       |                      |                       \ |                  |          |
       |                      |                        \`------------------`          |
       |                      |                                  ^                    |
.--------------.       .--------------.                          |                    |
|              |       |              |                          |                    |
|              |       |              |                          |                    |
     . . .                   . . .                             . . .                . . .

Each time a Bitcoin block is mined, it causes a Stacks block to be mined.  Now, each time a Stacks block is mined, it causes one or more appchain blocks to get mined, which in turn can trigger their own subsequent appchain blocks to be mined.
The appchain approach makes Stacks a composite blockchain.  You can add more capacity to the Stacks blockchain by adding more appchains.  Instantiating an appchain is just a matter of deploying a specially-crafted smart contract that can store the relevant mining state.  Just as with Stacks and Bitcoin, each appchain's blocks are hashed to a single transaction in the host chain, so to host chain nodes, mining an appchain just looks like doing a particular contract-call? transaction.
App chains are classified by degress of separation from Bitcoin.  This is the number of host chains between the chain and Bitcoin.  For Stacks, this is 1 -- Bitcoin is its host chain.  For appchains mining on Stacks, this is 2 -- appchain state is separated from Bitcoin by two blockchains.  For appchains running on those appchains, this is 3, and so on and so forth.  However, because each host chain contains the hashes of its client chains' blocks, it still ends up being the case that Bitcoin commits to the state of all appchains, everywhere.
The reason this scales is because the host chain doesn't need to know or care about the state of its client chains (just like how Bitcoin doesn't know or care about Stacks).  Only the people who want to use your smart contracts need to run nodes for your appchain; everyone else can ignore it entirely.
When to use Appchains

Appchains are one of several possible scalability solutions for Stacks being developed, but they are some trade-offs.  If you're a developer, you'd use an appchain if all of the following are true:

Your app has its own token that is distinct from STX.
The smart contracts for your app don't really interact much with smart contracts for other apps.
Users need lots of block capacity to use your smart contracts, and/or need to send lots of transactions.

If these are all true, then it might make sense for you to start your own appchain.  Your appchain will be instantiated with your smart contracts built-in, and it will have its own distinct token separate from STX.  Your appchain will have its own miners, who will send transactions on the host chain to mine blocks in your appchain and in doing so, earn a block reward in your appchain's token.
Because appchains are simply instances of Stacks, they come with all the features Stacks currently has.  In particular:

Your appchain comes with its own PoX, its own cost-voting, and its own BNS pre-installed.
Your users can Stack your appchain's token and earn a yield in STX (or whatever token powers the host chain).
You can choose your own PoX parameters and block capacity limits.

In addition, appchains have the following new features enabled by mining on Stacks as a host chain:

You're free to make any changes to the appchain consensus rules without affecting Stacks (or whatever host chain).
You can implement your own rules on who gets to mine and under what circumstances via the mining contract on the host chain.  If you can write them in Clarity, you can make them apply to your appchain.

Limitations

Because appchains are independently mined from their host chains, the security budget for resisting a reorg is dependent on how much miners of that appchain are willing to spend to keep the chain safe.  This budget is going to be less than the budget for Stacks, which in turn is less than the budget for Bitcoin.  However, the set of blocks that exist in your appchain across all its forks (even non-canonical ones) is secured by the host chain's security budget.  So even if your appchain gets reorged, it would be temporary, since the old canonical fork is still visible in the host chain.
App chain P2P messages and transactions are made non-replayable on other networks by a 4-byte chain ID field.  However, there is currently no convention or means of forcing all appchains to have globally-unique chain IDs.  Someone could clone your appchain and use its chain ID, thereby making your chain's transactions replayable on their network.  This will need to be solved later by a dedicated appchain registry smart contract, which will first need to be spec'ed out and ratified via the SIP process.  We're still in the very early days!
I don't know off the top of my head if the distinct chain ID makes it impossible to use a hardware wallet at this time.  I'd imagine that adding support would be trivial, though.
Setting Up an Appchain

The code for this is currently in a feature branch on the Stacks blockchain.  You can build a copy yourself as follows:
$ git clone https://github.com/blockstack/stacks-blockchain appchains
$ cd appchains
$ git checkout feat/app-chain-mvp
$ cd testnet/stacks-node
$ cargo build --release

First, you'll need to deploy a Stacks mainnet or testnet node from this feature branch (depending on where you want to run your appchain).  I highly recommend using only testnet for now.  This step is required because the new Stacks node software includes RPC endpoints that appchains need, such as querying Stacks headers and data var state.
After you have that running, you will need to create a mining contract for your appchain.  The mining contract has a few required define-data-var and define-data-map definitions that will be needed for the appchain to query it and boot up, in addition to having a few required define-public functions.  I've reproduced the one I'm testing with here, with source comments.  Feel free to adapt to your needs.  Note in particular that the mining contract has a way to list the bootstrap nodes for your appchain.
Once the contract is deployed, you'll need to spin up a boot node for your appchain at the IP and port listed in the mining contract.  To do so, you'll first need to figure out the contract's address, as well as the appchain's genesis hash.
Let's say the contract address is ST17ABWV76GQCGWKFQR4G5N23HDXGZ3D3869A8Z5N.appchains-mvp-v2.  You will want to add that to your config file as follows:
[burnchain]
chain = "stacks"
mining_contract = "ST17ABWV76GQCGWKFQR4G5N23HDXGZ3D3869A8Z5N.appchains-mvp-v2"
# since the address is a testnet address, you'll also want this:
mode = "testnet"
# Fill this in with the corechain Stacks node you deployed earlier
peer_host = "44.199.104.134"
rpc_port = 30443
peer_port = 30444

To get the genesis hash, you run the following:
$ stacks-node appchain-genesis --config /path/to/your/config.toml
Appchain genesis state instantiated in /tmp/appchain-genesis-1632538390
Appchain genesis hash: a32a3ff4b42a3d87d771396ca5ffab850afbc1376927ad2789726a43c881fe72

You'll put that hash into your config as follows:
[burnchain]
genesis_hash = "a32a3ff4b42a3d87d771396ca5ffab850afbc1376927ad2789726a43c881fe72"

So, your [burnchain] section should now have four fields:

chain = "stacks"
mining_contract = "ST17ABWV76GQCGWKFQR4G5N23HDXGZ3D3869A8Z5N.appchains-mvp-v2"
mode = "testnet"
genesis_hash = "a32a3ff4b42a3d87d771396ca5ffab850afbc1376927ad2789726a43c881fe72"

Now you're ready to go.  Simply start up your appchain boot node as follows:
$ stacks-node appchain --config /path/to/your/config.toml

As long as you tell your other users and miners the mining_contract and genesis_hash values, they can go and do likewise and join your appchain network.
Adding Bootcode

One thing you can do with appchains that you can't do with Stacks is add additional boot code.  This is code that will live under the boot address ST000000000000000000002AMW42H on testnet, along with .pox, .bns, and .costs.  If you want to make sure your custom smart contracts are available the minute your chain boots up, you can ship them as boot code.
Shipping your appchain's desired smart contracts as boot code is advantageous for a few reasons:

Shipping code as boot code ensures that it's always available for the duration of the chain's existence.  No one can reorg it away.
There's no block limit on the boot code, so you can run any expensive pre-calculations you want in a boot code smart contract.  This includes things like importing data from another chain, or pre-populating a set of accounts.

To deploy your own boot code, you'll need to do the following three things:


Add the contract names to the list of boot code contracts in the mining contract, within the appchain-config data var's boot-code list.  In the linked example above, this includes "hello-world".


Generate the genesis hash with the boot code.  You can do this by passing in the --boot-code ClI argument.  You can pass it multiple times for multiple contracts.  They must have the same filenames as the contract names


$ stacks-node appchain-genesis --config /path/to/your/config.toml --boot-code ./hello-world.clar


Launch the boot node with the boot code, so other appchain nodes can go and download it from you.  You do this by also passing the source files in with --boot-code:

$ stacks-node appchain --config /path/to/your/config.toml --boot-code ./hello-world.clar

Other users do not need to pass --boot-code; only the appchain creator and people who cold-boot their nodes will need to do this (which will basically be just you, the developer).  However, other users will of course need both the contract address and the genesis hash, which you can provide them out-of-band.
One day, there will be a registry for appchains so developers can both reserve chain IDs and publish their mining contract addresses and genesis hashes under a human-readable identifier.  Maybe there could be a BNS namespace for it.  I haven't thought that far ahead.
Running a Node

I have deployed a  sample appchain: on the Stacks testnet that you can play around with.  Here is a sample config you can use to connect to it:
[node]
rpc_bind = "0.0.0.0:15301"
p2p_bind = "0.0.0.0:15300"
seed = "<FIXME: fill in with your private key>"
miner = false
# you may want to change this
working_dir = "/var/stacks/appchain-follower/chainstate"
mine_microblocks = true
microblock_frequency = 1000
wait_time_for_microblocks = 5000
deny_nodes = ""

[burnchain]
mining_contract = "ST17ABWV76GQCGWKFQR4G5N23HDXGZ3D3869A8Z5N.appchains-mvp-v2"
genesis_hash = "a32a3ff4b42a3d87d771396ca5ffab850afbc1376927ad2789726a43c881fe72"
chain = "stacks"
mode = "xenon"
peer_host = "44.199.104.134"
rpc_port = 30443
peer_port = 30444
poll_time_secs = 10

[connection_options]
connect_timeout = 5
handshake_timeout = 5

# if you want to mine...
[miner]
min_tx_fee = 600
first_attempt_time_ms = 1000
subsequent_attempt_time_ms = 1000

To run the node, simply execute:
$ stacks-node appchain --config /path/to/the/above/config.toml

Your node should boot up and remain in sync with the appchain described in this contract.  Do note that you must use a stacks-node built from the feat/app-chain-mvp feature branch.
Sending Appchain Transactions

If you have some appchain tokens, then sending transactions is basically the same as sending them to the Stacks network.  The only difference is that you must supply the appchain's chain ID to the transaction.  In this example, this is 0x80000002, or 2147483650.  You can get the chain ID from the mining contract, in the appchain-config data var.
To interat with the example appchain above, you would pass --chain_id 2147483650 to blockstack-cli to generate transactions.  For example:
$ blockstack-cli --testnet --chain_id 2147483650 publish b8d99fd45da58038d630d9855d3ca2466e8e0f89d3894c4724f0efc9ff4b51f001 515 0 kv-store ./kv-store.clar
Note that in addition to supplying a chain ID, you must also supply --testnet.  This is because the appchain itself is the testnet variant of chain ID 2147483650 (there could also be a mainnet chain 2147483650).
Future Work

I consider appchains to be in an alpha state.  They work -- you could even ship products with them! -- but there are known limitations in the ecosystem that I'd like to get done before we can declare them a fully-supported feature.


There needs to be a SIP that standardizes the mining contract's structure.  This includes providing a trait definition for mining, as well as standardizing the data types for the appchain-version, appchain-config, and appchain state.


There needs to be an authoritative registry contract for appchains on the Stacks corechains, whereby developers can claim their appchains' chain IDs and specify their mining contract addresses and genesis hashes.  There also needs to be a way to expire and re-use chain IDs for defunct appchains.  This could be BNS, but I'm not married to the idea if that proves difficult for whatever reason.  This registry should be standardized via the SIP process.


The appchain codebase needs to use the appchain registry contract to resolve appchain names to their mining contracts and genesis hashes, so the act of spinning up an appchain only requires the user to know the human-readable identifier of the appchain, instead of its more inscrutible (and spoofable) metadata.


Appchains will likely want different token emission schedules, and even different rules for when to consider transactions and blocks valid.  I think this could be addressed by a special "consensus" smart contract in the boot code that hooks into the block and transaction validation code paths.  Basically, it would adhere to a trait that contained public functions for the following:

At a given block height, how many tokens need to be emitted?
Given a transaction and read-only access to the chain state, is the transaction valid?
Given a block and read-only access to the chain state on top of which it is built, is the block valid?

The reason appchain developers would want the ability to decide these things is to constrain what they want the appchain to be used for.  Appchains don't need to have the exact same consensus rules as Stacks -- they can be more restrictive.  For example, an appchain that is only meant to process transactions for a particular smart contract could declare all SmartContract transactions invalid, so no one else could create a smart contract on that appchain.  As another example, an appchain might require miners to hold host chain tokens as collateral in the mining contract, and execute a "peg-out" to transfer wrapped host chain tokens from the appchain back to the host chain each time they mine a block.  This consensus smart contract could verify that this happens as part of deciding whether or not a block is valid.


Appchain tokens are not STX.  The Clarity VM should allow STX-specific Clarity functions to be given different names by the appchain to reflect the true name of the appchain token.


The stacks.js library needs to permit the application to specify the chain ID.


There needs to be wallet support for appchains, including Ledger at some point.  The web wallet needs to learn the chain ID(s) of the per-application appchain(s) in use.  I have not looked into how difficult this will be.


There needs to be a fully fleshed-out light client for Stacks, which would permit the user to boot up an appchain without having to boot up instances of all of its ancestor chains.  This is blocked on feat/1805, which will ship in Stacks 2.1.