Archive of Substrate Wiki Pages

Substrate-wiki-archive-SEE-COMMENTS-IN-GIST.md

https://github.com/paritytech/substrate/wiki/External-Address-Format-(SS58)

SS58 is a simple address format designed for Substrate based chains. There's no problem with using other address formats for a chain, but this serves as a robust default. It is heavily based on Bitcoin's Base-58-check format with a few alterations.

The basic idea is a base-58 encoded value which can identify a specific account on the Substrate chain. Different chains have different means of identifying accounts. SS58 is designed to be extensible for this reason.

Basic Format

The basic format conforms to:

base58encode ( concat ( <address-type>, <address>, <checksum> ) )

That is, the concatenated byte series of address type, address and checksum then passed into a base-58 encoder. The base58encode function is exactly as defined in Bitcoin and IPFS, using the same alphabet as both.

Address Type

The <address-type> is one or more bytes that describe the precise format of the following bytes.

Currently, there exist several valid values:

• 00000000b..=00111111b (0..=63 inclusive): Simple account/address/network identifier. The byte can be interpreted directly as such an identifier.
• 01000000b..=01111111b (64..=127 inclusive) Full address/address/network identifier. The low 6 bits of this byte should be treated as the upper 6 bits of a 14 bit identifier value, with the lower 8 bits defined by the following byte. This works for all identifiers up to 2**14 (16,383).
• 10000000b..=11111111b (128..=255 inclusive) Reserved for future address format extensions.

The latter (42) is a "wildcard" address that is meant to be equally valid on all Substrate networks that support fixed-length addresses. For production networks, however, a network-specific version may be desirable to help avoid the key-reuse between networks and some of the problems that it can cause. Substrate Node will default to printing keys in address type 42, though alternative Substrate-based node implementations (e.g. Polkadot) may elect to default to some other type.

Address Formats for Substrate

There are 16 different address formats, identified by the length (in bytes) of the total payload (i.e. including the checksum).

• 3 bytes: 1 byte account index, 1 byte checksum
• 4 bytes: 2 byte account index, 1 byte checksum
• 5 bytes: 2 byte account index, 2 byte checksum
• 6 bytes: 4 byte account index, 1 byte checksum
• 7 bytes: 4 byte account index, 2 byte checksum
• 8 bytes: 4 byte account index, 3 byte checksum
• 9 bytes: 4 byte account index, 4 byte checksum
• 10 bytes: 8 byte account index, 1 byte checksum
• 11 bytes: 8 byte account index, 2 byte checksum
• 12 bytes: 8 byte account index, 3 byte checksum
• 13 bytes: 8 byte account index, 4 byte checksum
• 14 bytes: 8 byte account index, 5 byte checksum
• 15 bytes: 8 byte account index, 6 byte checksum
• 16 bytes: 8 byte account index, 7 byte checksum
• 17 bytes: 8 byte account index, 8 byte checksum
• 34 bytes: 32 byte account id, 2 byte checksum

Checksum types

Several potential checksum strategies exist within Substrate, giving different length and longevity guarantees. There are two types of checksum preimage (known as SS58 and AccountID) and many different checksum lengths (1 to 8 bytes).

In all cases for Substrate, the Blake2-256 hash function is used. The variants simply select the preimage used as the input to the hash function and the number of bytes taken from its output.

The bytes used are always the left most bytes. The input to be used is the non-checksum portion of the SS58 byte series used as input to the base-58 function, i.e. concat( <address-type>, <address> ). A context prefix of 0x53533538505245, (the string SS58PRE) is prepended to the input to give the final hashing preimage.

The advantage of using more checksum bytes is simply that more bytes provide a greater degree of protection against input errors and index alteration at the cost of widening the textual address by an extra few characters. For the account ID form, this is insignificant and therefore no 1-byte alternative is provided. For the shorter account-index formats, the extra byte represents a far greater portion of the final address and so it is left for further up the stack (though not necessarily the user themself) to determine the best tradeoff for their purposes.

Simple/full address types and account/address/network identifiers

The table above and, more canonically, the codebase as well as the registry express the status of the account/address/network identifiers (identifiers).

Identifiers up to value 64 may be expressed in a simple format address, in which the LSB byte of the identifier value is expressed as the first byte of the encoded address.

For identifiers of between 64 and 16,383, the full format address must be used.

The encoding of this is slightly fiddly since we encode as LE, yet the first two bits (which should encode 64s and 128s) are already used up with the necessary 01 prefix. We treat the first two bytes as a 16 bit sequence, and we disregard the first two bits of that (since they're already fixed to be 01. With the remaining 14 bits, we encode our identifier value as little endian, with the assumption that the two missing higher order bits are zero. This effectively spreads the low-order byte across the boundary between the two bytes.

Thus the 14-bit identifier 0b00HHHHHH_MMLLLLLL is expressed in the two bytes as:

• 0b01LLLLLL
• 0bHHHHHHMM

Identifiers of 16384 and beyond are not currently supported.

https://github.com/paritytech/substrate/wiki/FAQ/_edit

How do I store a values using a struct?

Ensure the struct implements everything needed for Parameter. This basically means putting #[derive(Encode, Decode, Clone, Eq, PartialEq)] #[cfg_attr(feature = "std", derive(Debug)] before it. If you also intend for your struct to be used in a field that can be configured in a JSON chain spec file, then the second part should be need #[cfg_attr(feature = "std", derive(Debug, Serialize, Deserialize))] (and you'll need to use serde_derive;).

See staking/lib.rs and specifically ValidatorPrefs for an example of it working.

How do I work with addresses in a struct (or an appropriate reference that deals with the public address) other than origin (which already has T::Origin)

Ensure they are stored as T::AccountId. In a struct, you'll want to keep it generic and parameterise the type with T::AccountId (where T is your module's configuration trait (which implies system::Trait).

See staking/lib.rs and specifically ValidatorPrefs for an example of it working in a similar way with with T::Balance instead of T::AccountId.

Anything special I need to know about dealing with a hashed value in a struct?

Not really; just ensure it's a generic argument in your struct which is specialised from your module
as T::Hash.

How do I store put strings in storage/accept them as parameters to functions?

In general you should not be using, modifying, working with human-readable strings in the runtime, since human-readability and strict consensus rarely go together. Working with hashes of those strings should generally be fine. However, if you're really insistent on placing human-readable data into the (very expensive Merklised) runtime state, then the way to do it is for convert to some particular encoding (e.g. UTF-8) and provide that byte array to the runtime (as a Vec<u8>) to store.

https://github.com/paritytech/substrate/wiki/Feature-freeze-towards-launching-Polkadot

Towards launching Polkadot / releasing Substrate 2.0

Owners: Ben (@gnunicorn), Martin (@s3krit)

Alpha: Accepted blockers past freeze in #4961
During stabilization: only release-critical PRs may change the public API or add new features (check with your lead-tech whether your PR falls under that)

Conditions apply

While this previously was focusing on creating an API-stability release, the goal is now to stabilize towards a launch of Polkadot.

2.0 will not be

having any stability guarantees beyond our commitment to adhere to semver: an update from any minor within 2.x will not break your build. HOWEVER, it still might change internals and thus storage data without automatic migrations yet. A 2.x minor upgrade might still mean you have incompatible chains or networks between releases. Release notes will inform you whether that would be the case.

Goals

• Get everything ready for a 2.0 release. In particular:
  • crates.io release procedure
  • automatic tooling for releases
  • getting documentation, API docs and DevHub up to date
• make an effort to stabilize on the way towards the launch
• Communicate externally, as well as internally, that we are on the way to stabilize for the launch

Path to the launch

Alpha -> Stabilization -> Release.

Alpha

We feature-freeze substrate master at noon Feb 21st 2020 (Berlin time). After that time only bug- and documentation-fixes, CI/Automatizing-Tooling-PRs and the already known blockers will be merged. Only PRs that fix security issues and PRs of extreme high severity may change the externally exposed API or rename crates from this point forward.

There might be multiple alpha releases on a rolling bases until all known blockers are merged and the automatic tooling is in place and expected to work. Alpha releases have the proposed version number for each create and the suffix -alpha. with a steadily increasing number, starting with alpha.1. All crates are released with the same suffix, even if there were no changes between releases for it.

Stabilization

As of Friday, April 24th 2020 (23:59 Berlin Time), we will enter the stabilization-phase. After this point, only bug-, automation-, documentation-fixing- and launch-critical PRs are to be merged. The goal of this phase is to stabilize the system, run the network, report and fix last minute bugs.

• All PRs should therefore limit changes to what is absolutely necessary to achieve the goal,
• generic refactors and non-release-essential enhancements will be rejected (and should be put into draft mode).
• Only launch-critical PRs are allowed to change public APIs from this point.
• PRs need to have two positive reviews, pass the CI (incl. the companion) and have a positive review on the polkadot companion (if any), before they can be merged.

Further alpha releases are expected to take place, once we are sure to have reached API stability, we might switch over to releasing them as -beta. and the release count, starting at 1. All crates are released with the same prefix, even if there are no changes in between. There might be multiple beta releases on a rolling bases.

Launch & Release

Once we have all release critical features in, have that running smoothly and are confident it is stable enough, we might launch it as polkadot. We might also release the creates as 2.0 (or their respective release number, sometimes 0.8.0) to crates.io, however we might delay this further to include fixes that we recognize post-launch. The feature freeze ends.

Process to get PRs merged during the freeze

If you have a polkadot-release-critical change (ask your team -lead if you are unsure yours is), cleanups (before stabilization, this includes not-too-involved refactors and API changes ), bug-, documentation-, CI- or hotfix, just raise the PR as usual and get it reviewed by at least two core devs–don't forget to put the appropriate B- andF- labels on it!

Once it is ready to merge please tag, has been reviewed by at at least two core developers and has a positive review on its polkadot companion, tag it with A8-mergeoncegreen (and remove the A0-review) and assign your/the responsible team lead. We might also change the tags accordingly, just switch it back to A8-mergeoncegreen whenever you fixed it and we'll get on it.

https://github.com/paritytech/substrate/wiki/Public-RPC

Substrate node contains a set of CLI flags that allow you to expose the RPC interfaces publicly, namely:

--ws-external
--rpc-external

And their --unsafe-* versions. By default the node will refuse to start if you try to expose RPC and run a validator node at the same time. --unsafe-* flags allow to suppress this security measure.

The fact that the options are provided does not mean we endorse doing so, in fact, quite opposite, exposing your RPC interfaces may open up a huge surface of attacks and has to be carefully reviewed. There are quite a few RPC methods that can be used to control the node's behaviour and should never (or rarely) be exposed:

• author_submitExtrinsic - allows submitting transactions to local pool.
• author_insertKey - allows inserting private keys to local keystore.
• author_rotateKeys - session keys rotation.
• author_removeExtrinsic - remove and ban extrinsic from the pool.
• system_addReservedPeer - add reserved node.
• system_removeReservedPeer - removed reserved node.

Other RPCs are not so much dangerous but can take a long time to execute, potentially blocking the client from syncing. These include:

• state_getKeys - get all the keys in the state with a particular prefix.
• state_getPairs - get all the keys in the state with a particular prefix together with their values.

It's critical to filter out these kind of calls if the requests are coming from untrusted users. The way to do it is through a JSON-RPC proxy that is able to inspect calls and only pass through whitelisted set of APIs.

Here is an example configuration file for jsonrpc-proxy:
https://github.com/tomusdrw/jsonrpc-proxy/blob/master/examples/permissions.json#L5

Note that there might be other solutions in the wild that could be used for this purpose, also note that the project listed above should be considered experimental.

https://github.com/paritytech/substrate/wiki/Reclaiming-an-index

To reclaim an index I, you must be sure that I is currently mapped to a dead account (i.e. one with zero total balance). You need to craft a transaction that creates a new account by placing some balance B in it. B must satisfy these conditions:

B % 256 == 105

B / 256 % E == I

Where E is the quantized account index count:

E := (next_set_index * enum_set_size / 256 + 1) * 256

Given an amount to transfer A, you can calculate the lowest amount that is at least A, A' which will attempt to reclaim I as:

(Q(N / 256 + (E - 1 - I), E) + I) * 256 + N % 256

Where

N := Q(A + (256 - 1 - 105), 256) + 105

See for an exemplar spreadsheet https://docs.google.com/spreadsheets/d/1C9kQSlnTygRFDP-B8DDRKWwDcr8EOhVzJM6Ju1S6VwQ/edit?usp=sharing

https://github.com/paritytech/substrate/wiki/Secret-URI-Test-Vectors

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk///password is account:

• Public key (hex): 0xb69355deefa7a8f33e9297f5af22e680f03597a99d4f4b1c44be47e7a2275802
• Address (SS58): 5GC6LfpV352HtJPySfAecb5JdePtf4R9Vq49NUU8RhzgBq1z

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk is account:

• Public key (hex): 0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a
• Address (SS58): 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqAS7

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk/foo is account:

• Public key (hex): 0x40b9675df90efa6069ff623b0fdfcf706cd47ca7452a5056c7ad58194d23440a
• Address (SS58): 5DXZzrDxHbkQov4QBAY4TjpwnHCMrKXkomTnKSw8UArBESDT

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk//foo is account:

• Public key (hex): 0x547d4a55642ec7ebadc0bd29b6e570b8c926059b3c0655d4948075e9a7e6f31e
• Address (SS58): 5DyV6fZuvPemWrUqBgWwTSgoV86w6xms3KhkFU6cQcWxTwuv

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk//foo/bar is account:

• Public key (hex): 0x3841947ffcde6f5fef26fb68b59bb8665637e30e32ec2051f99cf6b9c674fe09
• Address (SS58): 5DLU27is5iViNopQb2KxsTyPx6j4vCu8X3sk3j3NNLkPCsLj

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk/foo//bar is account:

• Public key (hex): 0xdc142f7476a7b0aa262aeccf207f1d18daa90762db393006741e8a31f39dbc53
• Address (SS58): 5H3GPTqDSpjkfDwbHy12PD6BWm8jvGSX4xYC8UMprHpTPhQg

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk//foo/bar//42/69 is account:

• Public key (hex): 0xa2e56b06407a6d1e819d2fc33fa0ec604b29c2e868b70b3696bb049b8725934b
• Address (SS58): 5FkHmNgbg64MwStgCyDi2Uw3ufFu11mqQgmWT9uwK4Lgi3SE

SURI bottom drive obey lake curtain smoke basket hold race lonely fit walk//foo/bar//42/69///password is account:

• Public key (hex): 0x0e0d24e3e1ff2c07f269c99e2e0df8681fda1851ac42fc846ca2daaa90cd8f14
• Address (SS58): 5CP8S23JBNXYNpJsL7ESPJBNnUZE6itcfM4EnDxEhaVEUAWL

https://github.com/paritytech/substrate/wiki/Suggestions-for-Key-Derivation-Paths

SS58 supports a rich key derivation path system, allowing hard and soft derivation from both indices and arbitrary length text at each junction.

Here's an example of how we manage their keys in practice that might help inform your choices for path organisation.

Root

We begin with a root phrase P, generated in the usual means, perhaps through subkey or Polkadot JS Apps (though there are many wallets and key management system that support SS58 now). P is not used directly, we assume it is a paper wallet and that the secret for it is kept on a cold wallet device like Parity Signer.

Network/Chain

The first junction point is for the chain or network. We typically keep a single path for a "network" even though there may be several candidate chains (as was the case in Kusama, for example). This is always a hard derivation.

We call this the Network Root, which should not be confused with the Root, at the level above.

Public

Following this, we immediately branch into keys that we want to be public using a hard derived path pub. All keys beyond this path will use soft derivation. This address may be published to allow others to determine and verify typical public keys.

Typical sub-paths off of //pub include:

• /id: The main public identity of the individual/organisation.
• /technical: The Technical Committee member identity/-ies of the individual/organisation, if different to /id.
• /council: The Council member identity/-ies of the individual/organisation, if different to /id.
• /1, /2, ...: The general public accounts of the individual/organisation for asset management and other activities that use multiple fungible accounts.

Hot

It may be useful to have a separate set of hot wallets since the private key may be derived and placed on a hot device such that the secrets can be derived from P in case of loss, and yet the cold root phrase is never compromised by being present on the hot device.

These wallets are derived from a hard path //hot. Right now the only way of moving this from a cold device is to use subkey on a cold device:

> subkey inspect "P//polkadot//hot"

And then manually copy the secret seed into the hot device. Support for importing derived secret keys from Parity Signer into Polkadot JS Apps is planned.

Hot Public Paths

Public keys (including, e.g., identity) may alternatively be derived from the //hot//pub path instead of the cold //pub path. Which is chosen depends on your risk appetite. For some users on some chains, it may be desired to have the main identity key be on a hot device. For others, it may be unacceptable. We have found both variants useful under different circumstances.

General/throwaway Accounts

General accounts used for asset management and other throwaway uses will generally be derived from the Network Root as //1, //2.

Examples

• P//polkadot//hot//pub/id: Hot public identity for Polkadot mainnet.
• P//kusama//pub/technical/2: Cold second tech committee identity for Kusama.
• P//polkadot//1: First cold throwaway account for Polkadot.
• P//kusama//hot//mobile//10: Tenth throwaway/general purpose account on a (hot) mobile device for the Kusama network.

https://github.com/paritytech/substrate/wiki/Upgrading-2.0-to-3.0

(never completed, see https://github.com/paritytech/substrate/blob/master/docs/Upgrading-2.0-to-3.0.md)

https://github.com/paritytech/substrate.wiki.git

wiki repo for reference and history.