Skip to content

Instantly share code, notes, and snippets.

@ltfschoen

ltfschoen/w3f-polkadot-wiki-notes.md

Last active July 11, 2021 14:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save ltfschoen/f0e3e9e78ce3912dc3d9c54a50d45c95 to your computer and use it in GitHub Desktop.
Save ltfschoen/f0e3e9e78ce3912dc3d9c54a50d45c95 to your computer and use it in GitHub Desktop.
Download ZIP
W3F Polkadot Wiki Notes
Raw
w3f-polkadot-wiki-notes.md

Accounts

Different accounts are used to manage funds. The underlying cryptography of their account keys is the same:

Stash

  • Primary account
  • Holds funds. Usually a cold wallet (on piece of paper in a safe, or protected by layers of hardware security)
  • Bonds (locks) a portion of its funds with a Controller for participation (since the stake key is kept offline)
  • Unbonding requires waiting ~600 blocks before unlocked funds accessible again
  • Rarely exposed to the internet or used to submit extrinsics, since it may store a large amount of funds (metaphor: like a savings account at bank, or buried treasure hidden on a random island and only known by the priate that hid it)

Controller

  • Directly controlled by the user.
  • Semi-online account key
  • Control switching between Validator or Nominator, or Idle roles
  • Starts/Stops Staking process (Validating or Nominating) using funds kept in the Stash account's key
  • Unable to move or claim funds that are in the Stash account, so if the Controller is ever compromised the Stash account remains secure.
  • Minimum DOT funds required to pay fees when sending transactions
  • Exposed to the internet frequently so should not be used to store large funds, and should be replaced occasionally.

Session

  • Hot keys that must be kept online.

  • "session key" is a Session Certificate since it is actually 3x keys used by Validators for network operations, and is used for keypairs and signing, and :

    • BLS (Boneh-Lynn-Shacham) key. It is used for consensus signatures when voting since it is desirable to be used with GRANDPA finality gadget. BLS is preferred for voting in consensus algorithms and threshold signatures. It allows more efficient signature aggregation than when using Schnorr signatures. It will be used by Validators in Polkadot since they are more efficient than Schnorr signatures in some ways
    • Schnorrkel/Ristretto x25519 ("sr25519") key used for producing blocks with BABE (using Schnorr signatures) was created specifially to handle Polkadot use cases.
      • "sr25519" is based on the same underlying Curve25519 as EdDSA counterpart, Ed25519, but it uses Schnorr signatures instead of the EdDSA scheme, since they are more efficient, retain the same feature set and security assumptions, and allows for native multisignature through signature aggregation. It is superior to ed25519 for implementing complex protocols
    • Ed25519 key (using Schnorr signatures) used for identifying itself to other nodes over libp2p. It is used for simpler implementations like account keys that only need to provide signature capabilities. It has a broader support ecosystem (i.e. HSMs are already available for it)

  • Dedicated account (best practice) to prevent theft of funds incase Validator node compromised due to leaking of the --key

  • Seed of the account used for the Session Key --key option of a node

  • No recommended to store funds since does not perform transactions, and since they may be cycled through (possibly automatically)

  • Multiple Validators may use the same Session Key, but must ensure that only one is set to be validating at a given time (using a sophisticated script)

Additional Notes from Gav and Stephie

  • Why do we return the controller keys as session.validators and not the actual stash keys? We store validator keys in session because session expects a transaction dined by one of these keys to update the session key. We don't want to force the user to use their stash key for this since session keys change regularly and the stash key is meant to be in hibernation. Since controller keys can change during the lifetime of a staker, then we typically key everything by stash. This is an exception. It's fine in this case since the validator keys in session get updated every era from the staking module anyway.

Funds

DOTs

Request Testnet DOTs

Usage of DOTs

  • Governance (of the network)
    • DOT holders entitled to participate in control the Polkadot platform governance include determining network fees, add/remove parachains, and exceptional events such as platform upgrades/fixes.
  • Staking (for operation of the network)
    • DOT holders stake to facilitate Polkadot's consensus mechanism so the platform functions, and to allow valid transactions to be carried out across parachains. Staking of DOTs acts as a disincentive for malicious behaviour that is punished by their DOTs being slashed. DOTs required to participate in the network will vary depending on the activity, the staking duration, and total DOTs staked.
  • Bonded (to connect a chain to Polkadot as a parachain).
    • DOTs may be bonded/locked for a period when adding a new parachain to the network's relay chain and then released back when parachain removed. It ensures only valid and running chains are attached to the relay chain.

Roles

Validator

Background

  • Technical knowledge required to setup

  • Services

    • "full-node" 24/7 with critical network operations
    • Produce new block candidates in BABE
    • Vote and reach consensus using GRANDPA
    • Validate the STF of parachains
    • Data availability (i.e. blocks)

  • Staking "Reserving" DOT tokens to be rewarded with more, by putting them up as "collateral" for a chance to produce a valid block (and thus obtain a block reward). Validators and nominators (who back validators through NPoS) together stake their DOTs in order to add blocks to the relay chain.

  • Validator Slots are limited. Mainnet will have 50-100 validators initially, then 1000.

  • Reward Distribution

    • Relay chain distributes the block reward to Validators (and associated Nominators)
    • Recorded per Session (i.e. every ~5 minutes)
    • Payment Distribution Period is each Era (i.e. every ~1 hour).
      • Reward Distribution Time = PER_ERA * BLOCK_TIME = 600 * 6 = 1 hour

  • Reward Modification

    • Validator Allocations
      • Validator's Own Cut - Validators declare their own allocation (i.e. validator_payment = 50 DOTS of their payout from the staking mechanism before declaring the share that's dividied amongst their Nominators.
      • Voter's of the Validator Remaining Cut - After Validators deduct their cut of the reward, the remaining portion is split between Validators and Nominators that voted for them depending on their portion of stake.
    • Propose a Referendum to change the block reward parameters

  • Reward Re-investment Strategies for Validators or Nominators

    • Payment destination as Controller account - just accumulate rewards
    • Payment destination as Stash Account - either increasing the staked value with each reward received, or just keep using the original staked value)

  • Inflation

    • 10% in the first year
    • Validators will get 1000-2000 DOTs per month to share with their Nominators

  • Election of Validators is performed using the Sequential phragmen method that is used in the NPoS algorithm, and is based on weight (amount staked to them by nominators)

  • Slashing of a Validator occurs when they are reported offline a number of times more than Offline Slash Grace + Unstake Threshold == Max Offline Reports, and a balance is deducted from both the Validator's balance and all Nominators that voted for them:

    • Offline Slash Grace = 4 (network defined, testnet set to 4)
    • Unstake Threshold = 5 (Validator defined, max. value 10)

https://wiki.polkadot.network/en/latest/polkadot/node/

Create Stash, Controller, and Session Accounts

Accounts keys may be either ed25519 or the sr25519 variant.

Development

  • Connect to latest PoC-4 "Alexander" network or Development mode
  • Go to Accounts page in the Polkadot UI (https://polkadot.js.org/apps/#/accounts/create) under the "Create account" tab
  • Create Stash, Controller, and a Session account (i.e. MyName-Stash, MyName-Controller, and MyName-Session).
    • Tip: In Development mode, checkout this branch of the Polkadot-JS Apps UI, select "Development" for the "mnemonic seed", and then enter "//Bob//stash" for the "secret derivation path" to generate an account with a large test balance
  • Choose "Raw Seed" and save it since the Session account's seed is used as the value of --key when running the node
  • Enter password to encrypt the seed for the account being created.
  • Choose "Edwards (ed25519)" for "key pair crypto type" in "Advanced creation options"
  • Download backup of each account JSON file to a secure location
  • Protect Account Private Keys using a HSM (Hardware Security Module). TODO
  • Request Testnet DOTs for each account for staking (at least 150 million DOTs to cover existential deposit and tx fees)

Setup Node and Network Security

TODO

Download Polkadot

  • Setup Rust, dependencies for Polkadot
curl https://sh.rustup.rs -sSf | sh
sudo apt install make clang pkg-config libssl-dev
rustup update
  • Build Polkadot from source
git clone https://github.com/paritytech/polkadot.git
cd polkadot
cargo clean
git checkout v0.4
git pull origin v0.4
./scripts/init.sh
./scripts/build.sh
cargo install --path ./ --force
cargo build --release

My Substrate Setup

git clone https://github.com/paritytech/substrate.git
cd substrate
git checkout master
git pull --rebase origin master
./scripts/init.sh
./scripts/build.sh
./target/debug/substrate purge-chain --dev;
./target/debug/substrate --dev --alice --telemetry-url ws://telemetry.polkadot.io:1024

https://github.com/paritytech/polkadot

Run Validator Node.

  • Note: Alexander (PoC-3, PoC-4) Testnet

  • Run node using the latest version that tries to validate when it sees itself as a listed authority on the network. Secret key to make it a listed authority on the network and sign statements to authorise creation of new blocks.

Development (with in-built dev accounts)

target/release/polkadot \
--dev \
--alice \
--validator \
--key <SEED>

Production (PoC-4)

Sync to current highest block on Telemetry and PolkadotJS Block Explorer

target/release/polkadot \
--chain alex \
--validator \
--key <SEED>

Note:

Check Balances using Polkadot-JS

// See https://github.com/paritytech/polkadot/pull/242/files
const keyring = new Keyring({ type: "sr25519" });

/**
 * Since Bob or Eve are not actually nominating/validating on `--dev`
 * the stash it not locked and is available
 */
const user = keyring.addFromUri('//Bob//stash');
const previous = await api.query.balances.freeBalance(user.publicKey());

Bond Validator DOTs

Bond DOTs of the Stash account to be put at stake for the security of the network and can be slashed.

  • Risks:

    • Validators that misbehave (i.e. not online, wrong software version) will be slashed. Slashing starts with 0.001 DOTs penalty and doubles. If Validator node passes a session then the penalty is reduced by 50% with a minimum of 0.001 DOTs

  • Mitigation Measures:

    • Set Unstake Threshold shown below (max slashes your Validator node will tolerator before its automatically unstaked)

  • Go to Staking app, "Account Actions" http://localhost:3000/#/staking/actions

  • Select the Stash account, and click "Bond Funds" to open popup

  • In the Bonding Preferences popup choose the Stash account for "stash account", choose the Controller account for "controller account", enter a value to bond (i.e. not whole balance as need to pay for transaction fees), and choose the "payment destination" (for Rewards Distribution), click "Bond" and sign the transaction with the Stash account's password

  • Check that the Controller account card listed at http://localhost:3000/#/staking/actions now shows its associated Stash account, and now shows the "Set Session Key" and "Nominate" options.

  • Click "Set Session Key" to open the "Session Key" popup

  • In the Session Key popup, choose the Controller account for "controller account" and the Session account for "session account" and click "Set Session Key"

  • Check that the Controller account card listed at http://localhost:3000/#/staking/actions now shows its associated Session account, and now shows the "Validate" and "Nominate" options.

  • Click "Validate" to open the "Validate" popup

  • In the Validate popup, enter "Unstake Threshold" (number of times reported offline and slashed before being removed from validator set), and "Payment Preferences" (rewards you will keep, with rest shared amongst you and your nominators), and click "Stake"

  • Wait until your node is fully synchronised

  • Re-run node with the seed associated with the Session account

    • Development
target/release/polkadot \
--dev \
--key <SEED>
--name VALIDATOR_NAME \
--telemetry \
--telemetry-url ws://telemetry.polkadot.io:1024
  • Production
target/release/polkadot \
--chain alex \
--validator \
--key <SEED>
--name VALIDATOR_NAME \
--telemetry \
--telemetry-url ws://telemetry.polkadot.io:1024

  • Go to Telemetry and check if your Validator is listed https://telemetry.polkadot.io/#/Alexander

  • Go to Staking app, "Staking Overview" http://localhost:3000/#/staking

  • Check that the list of active Validators under "validators", and Validator Intentions under "next up", and at the top it shows the proportion of online Validators, and the amount of "waiting" Validator Intentions, which should be the same as the number listed below.

  • Find your Validator Intention in the "next up"

  • Wait until the next era (up to ~1 hour) and if there is a Validator Slot available it'll become an active Validator and listed under "validators"

Run Validator Node as systemd Process

http://wiki.polkadot.network/en/latest/polkadot/node/guides/how-to-systemd/

Nominator

Background

  • Secure relay chain by selecting good validators and staking DOTs.
  • Stake/lock the necessary amount of DOTs to nominate, and carefully nominating a set of validators to earn more DOTs, since if they misbehave you get slashed and lose DOTs. Staked/locked DOTs prevent further staking as a validator or sending DOTs to others until you un-nominated (effective at next era, ~1 hr currently), however receiving DOTs is permitted.

Setup

Create Stash, Controller, and Session Accounts

Same as for Validator setup guide.

Nominate a Validator

  • Settup Stash and Controller accounts (as explained in guide to setup a Validator)
  • Go to Staking page, choose Account Actions tab
  • Click "Set Session Key" for ...

INCOMPLETE - see remainder in https://wiki.polkadot.network/en/latest/polkadot/node/guides/how-to-nominate/

Collator

  • Maintain parachains by aggregating recent parachain transactions from users and executing them to create an unsealed parachain blocks, then providing it and a zero-knowledge state transition proofs to validators who are responsible for validating each state transition and proposing parachain blocks.
  • Monitor network and prove bad behavior to validators
  • Run a "full-node" parachain (retain all info necessary to be able to author new blocks and execute transactions, similar to miners in PoW blockchains).

Apps

Explorer app

View validated blocks as they are sealed

Blocks listed include:

  • Block height/number
  • Hash
  • Parent/previous block hash
  • Extrinsics root (merkle root of all extrinsics in the block).
  • State root (merkle root of all the storage in the block).

Transfer app

Send Transaction with DOTS

  • Extrinsics tab and select sender account with DOTs, recipient, an amount, click sign and send transaction, enter your password to unlock your account (unless in-built default account like ALICE, BOB, etc), wait a few blocks until account balances change.

Accounts app

  • View, create, edit, backup (private key), or restore Polkadot accounts

Vanity Address app

Vanity address generated by entering a string and start hashing. Longer strings take longer to compute. Some characters are not allowed due to the hashing algorithm used in Polkadot.

Staking app

Validator Info

  • Current operating validators
  • Empty validator slots available
  • All validator addresses and their latest block number

Staking System / Account Actions Info

  • Start staking with an accounts as a Validator
  • Start staking with an account as a Nominator
  • Stop staking Validator account (if account set as a Controller account)
  • Set a session key and bond DOTs (if account set as a Stash account)

Validator Staking of Testnet DOTs

Prerequisites of registration to be staked:

  • More DOTs than the lowest staked validator, OR
  • Open validator slot exists, you can register to be staked.

Validator Selection

  • New validators are chosen every 24 hours on mainnet.

Democracy app

  • Shows current proposals
  • Shows current referenda

Chain State app

The Chain State tab allows you to query the Polkadot PoC-5 blockchain for statistics and information.

  • consensus.authorityCount(): number of current validators
  • session.validators(): current validators by address, to verify a validator's status
  • staking.intentions(): accounts with the desire to stake and become validators
  • consensus.code(): Wasm code of the current runtime, to monitoring changes in the STF
  • system.accountIndexOf(who): number of extrinsics a given account has submitted
  • staking.freeBalanceOfWho(): balance of a given account

Extrinsics app

Extrinscis are the bits of information added to the blockchain, i.e. the things 'extrinsic' from the system. These include transactions, staking, proposing, voting, and many other consensus-level actions.

Start by selecting the account you wish to act with, then choose one of the following extrinsics from the extrinsics section.

Extrinsic format: https://wiki.polkadot.network/en/latest/polkadot/build/extrinsic-format/

Sudo app

Background

  • Polkadot Runtime (i.e. Relay Chain) and Parachains use SRMLs (i.e. Sudo SRML) - higher layer with code implementing chain logic (decoupled from the PRE so runtimes easily upgradable without having to upgrade the PRE), compiled to Wasm blobs (stored on-chain). Mainnet Relay Chain block time ~2-12 secs decided by governance.
  • Polkadot Runtime Environment (PRE) - lower-layers of tech stack (networking libp2p, consensus (GRANPDA finality gadget), Wasm VM). Support for swappable "plug-in" Polkadot Runtime through governance mechanism. Substrate is an implementation of the PRE, which allows developers to generate parachains compatible with the Polkadot relay chain. External chains that do not support the PRE must use a bridge.

Parachain Economics

  • Parachains may implement custom monetary system or incentive structure for collators, since they need collators to progress.
  • Parachain and Polkadot network security is completely separate from parachain economics.

Parachain Builders

Community Links

Rust Style Guide for Polkadot/Substrate

Polkadot Implementations

References (Read & Summarised):

TODO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment