BIP: 324
Layer: Peer Services
Title: Version 2 Peer-to-Peer Message Transport Protocol
Author: Jonas Schnelli <dev@jonasschnelli.ch>
Dhruv Mehta <dhruvkaran@pm.me>
Status: Draft
Type: Standards Track
Created: 2019-03-08
License: PD
This BIP describes a new Bitcoin peer to peer transport protocol with opportunistic encryption.
Add end-to-end encryption: The current Bitcoin p2p protocol(referred to as v1 in this document) is in plaintext. With the current unencrypted message transport, BGP hijack, block delay attacks and message tampering are inexpensive and can be executed covertly (undetectable MITM)[1].[2]
Increase observability of attacks:: Adding opportunistic encryption as described in this document, introduces a high risk for attackers of being detected. Peer operators can compare encryption session IDs or use other form of authentication schemes [3] to identify an attack.
Add encryption without computation overhead: Each v1 p2p message uses a double-SHA256 checksum truncated to 4 bytes. Roughly the same amount of computation power would be required for encrypting and authenticating a v2 p2p message with ChaCha20 & Poly1305.
Add encryption without risk of network partition: Implementing this proposal maintains compatibility between v1 peers and v2 peers. There should be no risk of network patitions.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119[4].
A peer that supports the v2 p2p transport protocol as defined in this proposal MUST accept encryption requests from all peers.
The encryption handshake MUST happen before any other messages are exchanged between the peers. A shared secret is established at the end of the Elliptic Curve Diffie-Hellman(ECDH) handshake.
Both communication directions have different symmetric cipher keys deterministically derived from the same shared secret.
If the responding peer closes the connection after receiving the handshake request, the initiating peer MAY try to connect again with the v1 p2p transport protocol. Such reconnects allow an attacker to "downgrade" the encryption to plaintext communication and thus, accepting v1 connections MUST not be done when the Bitcoin peer-to-peer network has almost entirely embraced v2 communication.
Peers supporting the transport protocol proposed here MUST signal the NODE_P2P_V2 = (1 << 11) service flag. Such peers MUST accept encrypted communication specified in this proposal.
A peer usually learns an address along with the expected service flags which MAY be used to filter possible outbound peers.
Peers MAY make outbound connections exclusively to peers supporting NODE_P2P_V2.
---------------------------------------------------------------------------------------- | Initiator Responder | | | | x, X := SECP256k1_KEYGEN() | | CLIENT_HDATA := X | | | | --- CLIENT_HDATA ---> | | | | y, Y := SECP256k1_KEYGEN() | | ECDH_KEY := SECP256k1_ECDH(X,y) | | SERVER_HDATA := Y | | | | <-- SERVER_HDATA ---- | | | | ECDH_KEY := SECP256k1_ECDH(x,Y) | ----------------------------------------------------------------------------------------
To request encrypted communication (only possible if yet no other messages have been sent or received), the initiating peer generates an EC secp256k1 ephemeral key and sends the corresponding 32-byte public key to the responding peer and waits for the remote 32-byte public key from the counterparty.
ODD secp256k1 public keys MUST be used (public keys starting with 0x03). If the public key from the generated ephemeral key is an EVEN public key (starting with 0x02), its private key SHOULD be negated and then recalculated. Only using ODD public keys makes it more complex to identify the handshake based by analyzing the traffic and looking for 33 bytes that start with 0x02 or 0x03. This in turn increases the cost of censorship.
The handshake request and response message are raw 32 byte payloads containing no header, length or checksum and MUST be sent before anything else.
To aid identification of the v2 handshake from a v1 p2p message, public keys starting with the 4-byte network magic are forbidden and MUST lead to local regeneration of an ephemeral key.
Pseudocode for the ephemeral-key generation
ecdh_priv_key = MakeNewKey()
for {
ecdh_pub_key = ecdh_priv_key.GetPubKey()
if (ecdh_pub_key[0] == 2) {
// Public key is even, negate the private key and try again
ecdh_priv_key.Negate();
} else if (ecdh_pub_key[0..3] == NETWORK_MAGIC) {
// Public key cannot start with the network magic bytes
ecdh_priv_key = MakeNewKey();
} else {
break;
}
}
Once a peer has received the public key from its counterparty, the shared secret MUST be calculated by using secp256k1 ECDH.
Private keys will never be transmitted. The shared secret can only be calculated if an attacker knows at least one private key and the counterparty's public key.[5]
After a successful v2 handshake, both peers:
- MUST use the v2 message structure described in this document. Unencrypted v1 messages from either peer MUST lead to an immediate connection termination.
- MUST wipe the ephemeral session key from memory and persistent storage.
NETWORK_MAGIC || "version".
The authenticated encryption construction proposed here requires two ChaCha20Forward4064-Poly1305 cipher suite instances per communication direction. Each cipher suite instance requires a 256-bit key. We derive the four required keys from the established ECDH secret (ECDH_KEY = SECP256k1_ECDH(X,y) = SECP256k1_ECDH(x,Y)) known to both peers after the handshake. The keys MUST be derived with HKDF [6] as shown below. Peer A is the handshake initiator, peer B is the responder.
Both peers MUST also calculate the 256-bit session id, SID.
// Convert ECDH_KEY input key material into a pseudo-random key using HKDF extract. PRK = HKDF_EXTRACT(hash=SHA256, salt="BitcoinSharedSecret||INITIATOR_32BYTES_PUBKEY||RESPONDER_32BYTES_PUBKEY||NETWORK_MAGIC", ikm=ECDH_KEY) // Derive 32 byte keys used to encipher data A-> B K1A = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_1_A", L=32) K2A = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_2_A", L=32) // Derive 32 byte keys used to encipher data B -> A K1B = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_1_B", L=32) K2B = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_2_B", L=32) // Derive session id SID = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinSessionID", L=32)
The v2 protocol is based on ECDH key exchange and cannot prevent MITM attacks. However, comparing the session id SID derived by both peers on a secure channel can help authenticate the session and make any MITM attacks observable.
v2 peers supporting this BIP, MUST present the session id to the user on request.
BIP324 leverages two cryptographic primitives designed by Daniel Bernstein:
The ChaCha20 PRF (pseudo-random function)[7] takes as input: (1) 128 fixed bits (2) 256 bits of key material (3) a 64 bit IV/nonce and (4) a 64 bit counter and outputs: 512 pseudo-random bits. We will represent it as ChaCha20PRF(key, iv, ctr)
The ChaCha20 PRF can be composed into the ChaCha20 DRBG (deterministic random bit generator) to produce a keystream by incrementing the counter up to 2^70.
ChaCha20DRBG(key, iv) = ChaCha20PRF(key, iv, ctr=0) || ChaCha20PRF(key, iv, ctr=1) || ...
The ChaCha20 DRBG thus constructed does not provide forward and backward security in case of a compromised key. i.e if an attacker learns the key, they can decrypt all passively collected past and future ciphertext. This proposal outlines a new DRBG construction, the ChaCha20Forward4064 DRBG described below to add forward and backward security.
Poly1305 [8] is a one-time Carter-Wegman MAC that computes a 128 bit integrity tag given a message and a single-use 256 bit secret key.
Assuming a node sends only ping messages (28 bytes in the v2 protocol) every 20 minutes (the timeout interval for post-BIP31 connections) on a connection, the node will transmit 4064 bytes in a little over 2 days. To provide forward and backward security, we propose re-keying the ChaCha20 DRBG every 4064 bytes of keystream. Once 4064 bytes of keystream from the ChaCha20 DRBG has been consumed, the next 32 bytes are used to re-key the cipher instance before continuing to obtain up to another 4064 bytes of keystream. The IV is initialized to 0 and incremented on every re-key event.
k0 = key iv = 0 ks0, k1 = ChaCha20DRBG(k0, iv)[0:4064], ChaCha20DRBG(k0, iv)[4064:4096] iv = iv + 1 ks1, k2 = ChaCha20DRBG(k1, iv)[0:4064], ChaCha20DRBG(k1, iv)[4064:4096] ... ChaCha20Forward4064DRBG(key) = ks0 || ks1 || ks2 || ...
Both peers MUST keep track of the re-keying sequence numbers (uint64) to set as the ChaCha20 IV upon re-keying.
ChaCha20Forward4064-Poly1305@Bitcoin combines the ChaCha20Forward4064 DRBG(using it as a stream cipher by XORing the DRBG keystream with the plaintext or ciphertext) and the Poly1305 MAC into an authenticated encryption mode with addition of encryption of the packet lengths. The detailed construction follows.
The ChaCha20Forward4064-Poly1305@Bitcoin cipher suite requires two 256-bit keys from the key exchange per communication direction. Each key (K1 and K2) is used by two separate instances of a ChaCha20Forward4064 DRBG. We will call the instances F(used for fixed-length purposes, using 35 bytes per message) and V(used for variable length purposes).
F = ChaCha20Forward4064DRBG(key=K1) V = ChaCha20Forward4064DRBG(key=K2)
When encrypting a message M of length len(M) bytes:
- Read 3 bytes from the DRBG instance,
F, XOR them with the 3-byte litte endian encoding forlen(M)and set the result as ciphertextC. - Read
len(M)bytes fromV, XOR them with thelen(M)bytes ofMand append the result toC. - Read 32 bytes from
Fand use it to instantiate a Poly1305 MAC,P. - Compute a 16 byte MAC tag of contents in
CusingP. Append the tag toC.Cis now the content that can be transmit to maintain confidentiality, integrity and optionally, authentication.
C:
- The length MUST be decrypted when 3 bytes of ciphertext have been received. Read 3 bytes from the DRBG instance,
F, XOR them with the first 3 bytes of the ciphertext. Interpret the resulting bytes as the little endian encoding oflen(M) - Read 32 bytes from
Fand use it to instantiate a Poly1305 MAC,P. - Compute a 16 byte MAC tag of contents in
C[0:3 + len(M)]and compare it to the 16 bytes atC[3 + len(M):]. The byte comparison must be done in a constant time manner. If the expected MAC tag differs from the provided MAC tag, the connection MUST be immediately terminated. - Read
len(M)bytes fromVand XOR them withC[3:3 + len(M)]to obtain the plaintext.
F(K1A), V(K2A) to encrypt messages on the send channel, F(K1B), V(K2B) MUST be used to decrypt messages on the receive channel.
The responding peer MUST use F(K1A), V(K2A) to decrypt messages on the receive channel, F(K1B), V(K2B) MUST be used to encrypt messages on the send channel.
Two separate cipher instances are used here so as to keep the packet lengths confidential (best effort; for passive observing) but not create an oracle for the packet payload cipher by decrypting and using the packet length prior to checking the MAC. By using an independently-keyed cipher instance to encrypt the length, an active attacker seeking to exploit the packet input handling as a decryption oracle can learn nothing about the payload contents or its MAC (assuming key derivation, ChaCha20 and Poly1305 are secure). Active observers can still obtain the message length (ex. active ciphertext bit flipping or traffic semantics analysis)
Optimized implementations of ChaCha20Forward4064-Poly1305@bitcoin are relatively fast, therefore it is unlikely that encrypted messages will require additional CPU cycles per byte when compared to the v1 p2p message format (double SHA256).
TODO: Update the test vectors after swapping k1/k2 in code
message 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 k1 (DATA) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 k2 (AAD) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ciphertext 76 b8 e0 76 b8 e0 ad a0 f1 3d 90 40 5d 6a e5 53 86 bd 28 bd d2 19 b8 a0 8d ed 1a a8 36 ef cc 8b MAC df cd 91 c8 75 ee 88 71 8b 63 34 5d 75 0c 68 4a
message 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 k1 (DATA) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 k2 (AAD) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ciphertext 77 b8 e0 76 b8 e0 ad a0 f1 3d 90 40 5d 6a e5 53 86 bd 28 bd d2 19 b8 a0 8d ed 1a a8 36 ef cc 8b MAC fb 6c f9 dc d7 e2 ee 80 7d 5f f9 81 eb 4a 13 5a
message ff 00 00 f1 95 e6 69 82 10 5f fb 64 0b b7 75 7f 57 9d a3 16 02 fc 93 ec 01 ac 56 f8 5a c3 c1 34 a4 54 7b 73 3b 46 41 30 42 c9 44 00 49 17 69 05 d3 be 59 ea 1c 53 f1 59 16 15 5c 2b e8 24 1a 38 00 8b 9a 26 bc 35 94 1e 24 44 17 7c 8a de 66 89 de 95 26 49 86 d9 58 89 fb 60 e8 46 29 c9 bd 9a 5a cb 1c c1 18 be 56 3e b9 b3 a4 a4 72 f8 2e 09 a7 e7 78 49 2b 56 2e f7 13 0e 88 df e0 31 c7 9d b9 d4 f7 c7 a8 99 15 1b 9a 47 50 32 b6 3f c3 85 24 5f e0 54 e3 dd 5a 97 a5 f5 76 fe 06 40 25 d3 ce 04 2c 56 6a b2 c5 07 b1 38 db 85 3e 3d 69 59 66 09 96 54 6c c9 c4 a6 ea fd c7 77 c0 40 d7 0e af 46 f7 6d ad 39 79 e5 c5 36 0c 33 17 16 6a 1c 89 4c 94 a3 71 87 6a 94 df 76 28 fe 4e aa f2 cc b2 7d 5a aa e0 ad 7a d0 f9 d4 b6 ad 3b 54 09 87 46 d4 52 4d 38 40 7a 6d eb 3a b7 8f ab 78 c9 k1 (DATA) 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f k2 (AAD) ff 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f ciphertext 39 40 c1 c8 68 cd 14 5b d5 46 91 e9 b6 b4 02 c7 8b d7 ea 9c 37 24 fc 50 df c6 9a 4a 96 be 8d ec 4e 70 e9 58 18 8a a6 92 22 ea ef 3f 47 f8 00 3f 1b c1 3d cf 9e 66 1b e8 e1 b6 71 e9 cf 46 ba 70 5b ca 96 3e 04 77 a5 b3 c2 e2 c6 6f eb 82 07 26 9d db 01 b1 37 2a ad 68 56 3b b4 aa d1 35 af b0 6f be 40 b3 10 b6 3b ef 57 8f f9 39 f3 a0 0a 6d a9 e7 44 d2 8b a0 70 29 4e 57 46 d2 ca 7b b8 ac 2c 8e 3a 85 5a b4 c9 bc d0 d5 85 5e 11 b5 2c ac aa 2d db 34 c0 a2 6c d0 4f 4b c1 0d e6 dc 15 1d 4e e7 ce d2 c2 b0 de 8d ed 33 ff 11 f3 01 e4 02 75 59 e8 93 8b 69 bc eb 1e 5e 25 9d 41 22 05 6f 6a db d4 8a 06 28 b9 12 f9 0d 72 83 8f 2f 3a af 6b 88 34 2c f5 ba c3 cb 68 8a 9b 0f 7a fc 73 a7 e3 ca d8 e7 12 54 c7 86 ea 00 02 40 ae 7b d1 df 8b cf ca 07 f3 b8 85 72 3a 9d 7f 89 73 64 61 MAC 7a c8 d9 35 a4 1b f9 54 64 32 36 0e 1c 54 37 08
| Field Size | Description | Data type | Comments |
|---|---|---|---|
| 3 | length | 24 bits | Encrypted length of ciphertext payload (not counting the 16 byte MAC tag or the 3 byte encrypted length) in number of bytes |
| 1-13 | encrypted message-type | variable | ASCII message-type (or one byte message-type-ID) |
| ? | encrypted payload | ? | The actual data |
| 16 | MAC tag | ? | 128bit MAC-tag |
The v2 message structure does not need the 4byte network magic. The HKDF key derivation process embeds the network magic bytes into the salt for the extraction step. Using a different network identifier in that step will result in different derived keys per network.
The maximum message size is 2^24 (16’777’216) bytes. Future communication MAY exceed this limit and thus MUST be split into different messages.
The 4 byte sha256 checksum in v1 messages is no longer required because the MAC tag provides integrity and authentication.
The message-type field MUST start with a byte that defines the length of the ASCII message-type string up to 12 chars (1 to 12) or a message-type-ID (see below).
To save valuable bandwidth, the v2 message format supports message-type-IDs. The ID/string mapping is a peer to peer arrangement and MAY be negotiated between the initiating and responding peer. A peer conforming to this proposal MUST support message-type-IDs based on the table below and SHOULD use message-type-IDs for outgoing messages.
| Number | Message Type |
|---|---|
| 13 | ADDR |
| 14 | BLOCK |
| 15 | BLOCKTXN |
| 16 | CMPCTBLOCK |
| 17 | FEEFILTER |
| 18 | FILTERADD |
| 19 | FILTERCLEAR |
| 20 | FILTERLOAD |
| 21 | GETADDR |
| 22 | GETBLOCKS |
| 23 | GETBLOCKTXN |
| 24 | GETDATA |
| 25 | GETHEADERS |
| 26 | HEADERS |
| 27 | INV |
| 28 | MEMPOOL |
| 29 | MERKLEBLOCK |
| 30 | NOTFOUND |
| 31 | PING |
| 32 | PONG |
| 33 | REJECT |
| 34 | SENDCMPCT |
| 35 | SENDHEADERS |
| 36 | TX |
| 37 | VERACK |
| 38 | VERSION |
| 39 | GETCFILTERS |
| 40 | CFILTER |
| 41 | GETCFHEADERS |
| 42 | CFHEADERS |
| 43 | GETCFCHECKPT |
| 44 | CFCHECKPT |
| 45 | WTXIDRELAY |
| 46 | ADDRV2 |
| 47 | SENDADDRV2 |
v1 inv: 4(Magic)+12(Message-Type)+4(MessageSize)+4(Checksum)+37(Payload) == 61 v2 inv: 3(MessageSize)+1(Message-Type)+37(Payload)+16(MAC) == 57 (93.44%)
v1 ping: 4(Magic)+12(Message-Type)+4(MessageSize)+4(Checksum)+8(Payload) == 32 v2 ping: 3(MessageSize)+1(Message-Type)+8(Payload)+16(MAC) == 28 (87.5%)
v1 block: 4(Magic)+12(Message-Type)+4(MessageSize)+4(Checksum)+1’048’576(Payload) = 1’048’600 v2 block: 3(MessageSize)+1(Message-Type)+1’048’576(Payload)+16(MAC) = 1’048’596 (99.9996%)
TODO: Update the test vectors after swapping k1/k2 in code
message verack k1 (DATA) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 k2 (AAD) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 sequence: 0 Message before encryption 01 00 00 25 Ciphertext (3byte AD, 1byte message-type, 16 bytes MAC) 77 b8 e0 53 14 05 09 d3 48 60 7a 07 58 00 77 44 be 48 21 ef
message PING (nonce=123456) k1 (DATA) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 k2 (AAD) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 sequence: 0 Message before encryption (3 bytes packet length, 1byte message-type-ID, 4 byte message payload [ping nonce]) 05 00 00 1f 40 e2 01 00 Ciphertext (3byte AD, 1byte message-type-id, 4 byte message payload [ping nonce], 16 bytes MAC) 73 b8 e0 69 f8 02 ac a0 19 62 2f 1a 1d d3 d3 be de 22 2e d9 ff 17 51 40
The encryption does not include an authentication scheme. This BIP does not cover a proposal to avoid MITM attacks during the encryption initialization. However, peers MUST show the session id to the user on request which allows to identify a MITM by a manual verification on a secure channel.
Optional authentication schemes may be covered by other proposals [3].
An attacker could delay or halt v2 protocol enforcement by providing a reasonable amount of peers not supporting the v2 protocol.
This proposal is backward compatible (as long as not enforced). Non-supporting peers can still use unencrypted communications.
- Complete Bitcoin Core implementation: bitcoin/bitcoin#14032
- Reference implementation of the AEAD in C: https://github.com/jonasschnelli/chacha20poly1305
- ^ Hijacking Bitcoin: Routing Attacks on Cryptocurrencies - M. Apostolaki, A. Zohar, L.Vanbever
- ^ Encrypting traffic between peers is already possible with VPN, tor, I2P, stunnel, curveCP or any other encryption mechanism on a networking layer below the application, however, most of those solutions require significant technical experience in setting up a secure channel and are therefore not widely deployed.
- a b BIP150
- ^ RFC 2119
- ^ This key-exchange is based on the discrete log problem and thus not sufficiently strong against known forms of possible quantum computer algorithms. Adding an additional quantum resistant key exchange like NewHope is possible but out of scope for this proposal.
- ^ HKDF (RFC 5869)
- ^ ChaCha20
- ^ Poly1305
- Pieter Wuille and Gregory Maxwell for most of the ideas in this BIP.
- Tim Ruffing for the review and the hint for the enhancement of the symmetric key derivation.
- Jonathan Cross for re-wording and grammar corrections
This work is placed in the public domain.
@gmaxwell
Seems reasonable. Although, this does leak some information over what noise does. Receiving the signature allows you to convince a third party that you (or someone) did indeed establish a connection with the party.
There is a cool trick I found reading Damgard's notes on witness indistinguishable sigma protocols [1]. Instead of proving I know secret key for X, I can prove that I know the secret key for X OR the DH shared secret. Since the verifier knows I don't know the secret key for the shared secret the verifier is convinced of the same thing (i.e. I know the secret key for X). However they cannot re-use the proof to convince anyone else because no one else could be convinced that the DH key is unknown to them.
I am not sure that this is a practical issue though.
I think I had seen this briefly -- it looks cool. Having authenticated connections covertly probe for interception is a nice idea esp in the case of Bitcoin p2p. I wonder if it achieves the property I mentioned above against a malicious verifier just because of its structure.
This is what I was thinking too. Looking at what is done in the noise XK [2] that lightning uses I don't think there actually would be an extra round of communication doing in session auth with a simple signature (or WI proof).
I haven't seen anything like this. I haven't studied how Poly1305 works at all so I have no idea what's gonna happen when you use it in a non-black-box way :)
[1] https://www.cs.au.dk/~ivan/Sigma.pdf (end of Section 7).
[2] http://www.noiseprotocol.org/noise.html#handshake-patterns