Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

  BIP: ??? (tbr after sending to mailing list)
  Layer: Applications
  Title: Bech32X encoded keys
  Author: Jonas Schnelli <dev@jonasschnelli.ch>
  Comments-Summary: No comments yet.
  Comments-URI: 
  Status: Draft
  Type: Standards Track
  Created: 2018-06-03
  License: BSD-2-Clause

Table of Contents

Introduction

Abstract

This document proposes a new serialization and encoding format for key material up to 520bits with minimal amount of metadata.

Copyright

This BIP is licensed under the 2-clause BSD license.

Motivation

Missing key metadata leads to inefficient blockchain rescans and can potentially lead to lost funds.

This proposal introduces a serialization format with minimal metadata for various key types as well as **Bech32X**, a BCH code that can correct up to 7 error for a maximal length of 1023 characters (including the 27 characters checksum).

Specification

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119[1].

Serialization

Type Name Description
1bit version bit Version bit for further extensions
15bit
(0-32767)
key-birthday A 15 bit timestamp expressed in days since genesis (valid up to ~2098).
9bit
(optional)
gap-limit-multiplier Transaction detection child-key-derivation gap limit multiplier
8bit
(0-255)
script_type Script type restriction

256bit key material results in 280 bits to encode and thus resulting in a Bech32X string (with human-readable-part) of 86 characters (vs WIF 51 characters)

512bit key material results in 545 bits to encode and thus resulting in a Bech32X string (with human-readable-part) of 139 characters (vs xpriv 111 characters)

520bit key material results in 553 bits to encode and thus resulting in a Bech32X string (with human-readable-part) of 141 characters (vs xpub 111 characters)

key-birthday

A 15 bit timestamp expressed in days since genesis (valid up to ~2098). The birthday MUST be set to the first possible derivation of the according extended key, if unknown, the used seed birthday MUST be used. If both unknown, 0 MUST be used.

gap-limit-multiplier

If the total decoded serialization length is 280 bits (decode), the gap limit MUST not be present.

The minimum gap limit is 100. The final gap limit is defined by (gap-limit-multiplier + 1) * 100 allowing a gap limit from 100 to 51'200 in steps of 100.

During scanning for related transactions or unspent outputs (scan through blocks, scan utxo-set, etc.) the application MUST ensure to always respect the gap limit. Applications processing a disaster recovery based on a serialized key after this proposal MUST scan with a childkey lookup-window of 0 up to the used-key-with-highest-child-index + gap-limit.

Script Types

Script Type ID Restriction
0x00 No restriction
0x01 P2PKH compressed
0x02 P2PKH | P2SH
0x03 P2WPKH P2WSH nested in P2SH
0x04 P2WPKH

If the script type restriction is set, the according key (or derived child keys) MUST only be used to derive addresses with the selected script type. This does not stand in contradiction to derivation path proposals[2].

Further script types numbers MUST be reserved in a new BIP document.

Bech32X Encoding

Serialized keys after this proposal MUST use Bech32X as encoding format. Bech32X is a BCH code (hamming distance 15, degree 27) that can correct up to 7 errors with a checksum of 27 characters and up to the maximal length of 1023 characters (including checksum).

Bech32X is based on Bech32[3]. The specification for the human-readable part, seperator and charset are identical to Bech32.

Checksum

The last 27 characters of the data part form a checksum and contain no information. Valid strings MUST pass the criteria for validity specified by the Python3 code snippet below. The function bech32x_verify_checksum must return true when its arguments are:

def bech32x_polymod(values):
    generator = [0x48ad7da5f5dffe2565cb2f7406b4a2bcbc, 0x55af243bb2f7943c3d4da6d046345795d, 0xa91cc8534da778785f9247a01ceda669f, 0x11a7b84839246b2e49b0c8d1039da6ddbb, 0x234e589060c8d655d131058707391d2f53]
    chk = 1
    for value in values:
        top = chk >> 130
        chk = (chk ^ top << 130) << 5 ^ value
        for i in range(5):
            chk ^= generator[i] if ((top >> i) & 1) else 0
    return chk

def bech32x_hrp_expand(s):
  return [ord(x) >> 5 for x in s] + [0] + [ord(x) & 31 for x in s]

def bech32x_verify_checksum(hrp, data):
  return bech32x_polymod(bech32_hrp_expand(hrp) + data) == 1

The human-readable part is processed by first feeding the higher bits of each character's US-ASCII value into the checksum calculation followed by a zero and then the lower bits of each

To construct a valid checksum given the human-readable part and (non-checksum) values of the data-part characters, the code below can be used:

def bech32x_create_checksum(hrp, data):
  values = bech32x_hrp_expand(hrp) + data
  polymod = bech32x_polymod(values + [0] * 27) ^ 1
  return [polymod >> (5 * (26 - i)) & 31 for i in range(27)]

Bech32X HRPs

Usage HRP Keymaterial size
Mainnet Private Extended xp 512 bits (32byte chaincode, 32byte private key)
Mainnet Public Extended xpu 520 bits (32byte chaincode, 33byte public key)
Testnet Private Extended tp 512 bits (32byte chaincode, 32byte private key)
Testnet Public Extended tpu 520 bits (32byte chaincode, 33byte public key)
Mainnet Key pk 256 bits (gap limit is omitted in serialization)

Further key types HRPs MUST be reserved in a new BIP document.

Testvectors (Bech32)

        • NEW TEST-VECTOR WILL FOLLOW *****
  • Extended private key
  • Key: 0x71 0x54 0x70 0x43 0x29 0xd1 0x17 0x25 0xd1 0xbf 0x5a 0x6d 0x44 0x9c 0x80 0xdf 0xb9 0x3f 0xf2 0x27 0xa0 0x7d 0xac 0x75 0xb9 0x78 0x88 0xe8 0x56 0x84 0x7e 0xb5
  • Chaincode: 0x50 0x1a 0x4f 0x15 0x2d 0x1d 0xb6 0xb8 0x48 0xdb 0x6e 0xe2 0x05 0xfa 0x18 0xee 0x5c 0x6d 0x25 0x02 0x3d 0x7c 0xec 0x47 0x59 0x87 0x8f 0x1a 0x46 0x57 0x82 0x8c
  • Birthday: 2011 (Monday, 7. July 2014)
  • Gap-limit-multiplier: 10
  • Resulting Gap-limit: 1100
  • script_type: 1
xp1kc8s5qhz4rsgv54z9a92yla4m2yrsqdlwdl7gn6qldvwkuh3zrg66z8ad2snf832tgaxcuv3kmwugzl5x8wtnkj2q3a03ky0kg8p7dvv4czpjqg9trlw9

  • Extended private key
  • Key: 0x71 0x54 0x70 0x43 0x29 0xd1 0x17 0x25 0xd1 0xbf 0x5a 0x6d 0x44 0x9c 0x80 0xdf 0xb9 0x3f 0xf2 0x27 0xa0 0x7d 0xac 0x75 0xb9 0x78 0x88 0xe8 0x56 0x84 0x7e 0xb5
  • Chaincode: 0x50 0x1a 0x4f 0x15 0x2d 0x1d 0xb6 0xb8 0x48 0xdb 0x6e 0xe2 0x05 0xfa 0x18 0xee 0x5c 0x6d 0x25 0x02 0x3d 0x7c 0xec 0x47 0x59 0x87 0x8f 0x1a 0x46 0x57 0x82 0x8c
  • Birthday: 0 (3th. January 2009)
  • Gap-limit-multiplier: 0
  • Resulting Gap-limit: 100
  • script_type: 0
xp1qqqqqq8z4rsgv54z9a92yla4m2yrsqdlwdl7gn6qldvwkuh3zrg66z8ad2snf832tgaxcuv3kmwugzl5x8wtnkj2q3a03ky0kg8p7dvv4czpjqgvv4zgn

  • Extended private key
  • Key: 0x71 0x54 0x70 0x43 0x29 0xd1 0x17 0x25 0xd1 0xbf 0x5a 0x6d 0x44 0x9c 0x80 0xdf 0xb9 0x3f 0xf2 0x27 0xa0 0x7d 0xac 0x75 0xb9 0x78 0x88 0xe8 0x56 0x84 0x7e 0xb5
  • Chaincode: 0x50 0x1a 0x4f 0x15 0x2d 0x1d 0xb6 0xb8 0x48 0xdb 0x6e 0xe2 0x05 0xfa 0x18 0xee 0x5c 0x6d 0x25 0x02 0x3d 0x7c 0xec 0x47 0x59 0x87 0x8f 0x1a 0x46 0x57 0x82 0x8c
  • Birthday: 32767 (Saturday, September 2098)
  • Gap-limit-multiplier: 511
  • Resulting Gap-limit: 51200
  • script_type: 255 (undefined)
xp1lmlllllr4rsgv54z9a92yla4m2yrsqdlwdl7gn6qldvwkuh3zrg66z8ad2snf832tgaxcuv3kmwugzl5x8wtnkj2q3a03ky0kg8p7dvv4czpjqgzu2je4

  • private key
  • Key: 0x71 0x54 0x70 0x43 0x29 0xd1 0x17 0x25 0xd1 0xbf 0x5a 0x6d 0x44 0x9c 0x80 0xdf 0xb9 0x3f 0xf2 0x27 0xa0 0x7d 0xac 0x75 0xb9 0x78 0x88 0xe8 0x56 0x84 0x7e 0xb5
  • Birthday: 32767 (Saturday, September 2098)
  • script_type: 255 (undefined)
pk1lmll7u25wppjn5ghyhgm7kndgjwgphae8lez0gra436mj7ygaptggl447a4xh7

Acknowledgements

  • Pieter Wuillie for generating the Bech32X parameters and for the sample code

Compatibility

Only new software will be able to use this serialization and encoding format.

Reference implementation

https://github.com/jonasschnelli/bech32_keys

References

  1. ^ RFC 2119
  2. ^ BIP 0049 Derivation scheme for P2WPKH-nested-in-P2SH based accounts
  3. ^ BIP 0173 / Bech32

@jmcorgan

This comment has been minimized.

Copy link

commented Jun 3, 2018

Is this orthogonal to derivation schemes? The gap limit specified makes sense in the case of a single "terminal" chain (fixed derivation path up to the final level), but how would one deal with a more general derivation scheme that has multiple chains separated by different values higher up the derivation path?

@jonasschnelli

This comment has been minimized.

Copy link
Owner Author

commented Jun 3, 2018

As written in the initial proposal draft sent to the mailing list, metadata in this proposal is very limited and does focus on disaster recovery. Complex setup metadata should be stored in an app specific format since it is probably not achievable that such complex metadata formats are inter-application compatible.

Also, this format could be included in another, more complex format where not all parts needs error correction.

@nopara73

This comment has been minimized.

Copy link

commented Sep 2, 2018

I think it would be smarter to to merge this proposal with a future SF that requires new address scheme. I think we should be careful not only with consensus changes, but also with UX breaking changes. I think an introduction of a new address scheme is due only when it is unavoidable.

My concern about this proposal is the following scenario I see playing out: bech32X gets moderately popular and cool devs who are always jumping on the newest things implement it. However devs at big companies, who are working 12 hours a day as firefighters, trying to hack around the bugs and keep their bosses happy will delay the implementation of bech32 and bech32X further, because the complexity of the job has doubled, or drop the work altogether, because they don't even think it's a good long term solution anymore, since bech32 was just obsolated after a year of its intruduction. And indeed, that would be my thinking, and even though Wasabi Wallet is currently bech32 only, if the above scenario plays out it'll hinder bech32 adoption and my rational decision would be to downgrade Wasabi to P2WPKH over P2SH so to ensure full compatibility instead of jumping on the bandwagon again.

@meeDamian

This comment has been minimized.

Copy link

commented May 4, 2019

Not sure what the status is on this right now, but perhaps making HRP for private keys a bit more human-obvious might be a better UX. xprivate1…, or xsecret1… might prevent some less experienced users from being bamboozled into sharing their private keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.