Skip to content

Instantly share code, notes, and snippets.

@gavinandresen
Last active July 3, 2023 11:24
Show Gist options
  • Save gavinandresen/7079034 to your computer and use it in GitHub Desktop.
Save gavinandresen/7079034 to your computer and use it in GitHub Desktop.
Design sketch: "reject" p2p message

Design sketch: "reject" p2p message

Motivation

Mike Hearn has been lobbying for an "error" message in the Bitcoin p2p protocol for years (at least since the "ban peers if they send us garbage" denial-of-service mitigation code was pull-requested). This came up again with my proposed "smartfee" changes, which would drop low-priority or low-fee transactions.

In short, giving peers feedback about why their blocks or transactions are dropped or why they are being banned should help interoperability between different implementations, and will give SPV (simplified payment verification) clients feedback when their transactions are rejected due to insufficient priority or fees.

error" versus "reject"

I've gone with "reject" as the message, because "error" implies that the peer did something wrong-- and doing something wrong is only one of several possible reasons a connection, transaction, or block may be rejected.

##Specification

reject

One new message type "reject" is introduced. It is sent directly to a peer in response to a "version", "tx" or "block" message.

For example, the message flow between two peers for a relayed transaction that is rejected for some reason would be:

--> "inv"
<-- "getdata"
--> "tx"
<-- "reject"

common payload

Every "reject" message begins with the following fields. Some messages append extra, message-specific data.

Field SizeDescriptionData typeComments
?response-to-msgvar_strMessage that triggered the reject
1reject-codeuint8_t
?reasonvar_strHuman-readable message for debugging

The human-readable string is intended only for debugging purposes; in particular, different implementations may use different strings. The string should not be shown to users or used for anthing besides diagnosing interoperability problems.

The following reject code categories are used; in the descriptions below, "server" is the peer generating the reject message, "client" is the peer that will receive the message.

RangeCategory
0x01-0x0fProtocol syntax errors
0x10-0x1fProtocol semantic errors
0x40-0x4fServer policy rule

rejection codes common to all message types

CodeDescription
0x01Message could not be decoded

reject version codes

Codes generated during the intial connection process in response to a "version" message:

CodeDescription
0x11Client is an obsolete, unsupported version
0x12Duplicate version message received

reject tx payload, codes

Transaction rejection messages extend the basic message with the transaction id:

Field SizeDescriptionData typeComments
32txidchar[32]transaction that is rejected

The following codes are used:

Codedescription
0x10Transaction is invalid for some reason (invalid signature, output value greater than input, etc.)
0x40Not mined/relayed because it is "non-standard" (type or version unknown by the server)
0x41One or more output amounts are below the 'dust' threshold
0x42Transaction does not have enough fee/priority to be relayed or mined

payload, reject block

Block rejection messages extend the basic message with the block header hash:

Field SizeDescriptionData typeComments
32hashchar[32]block (hash of block header) that is rejected

Rejection codes:

codedescription
0x10Block is invalid for some reason (invalid proof-of-work, invalid signature, etc)
0x11Block's version is no longer supported
0x43Inconsistent with a compiled-in checkpoint

Note: blocks that are not part of the server's idea of the current best chain, but are otherwise valid, should not trigger "reject" messages.

Implementation notes

Sending "reject" messages to old nodes does no harm-- unknown commands are ignored for extensibility (in the reference implementation, at least-- other implementations should do the same). So there is no need to bump the protocol version.

Implementors should consider what happens if an attacker either sends them "reject" messages for valid transactions/blocks or sends them random "reject" messages, and should beware of possible denial-of-service attacks (e.g. blindly notifying the user of every "reject" message received would definitely be the wrong thing to do, and even blindly writing every "reject" message to a debug.log could open up a fill-up-disk DoS attack).

@petertodd
Copy link

Block rejection strings are a bad idea: whether or not a block is valid isn't a matter of opinion, it's defined by the protocol specification and current state of the blockchain. If a peer isn't willing to accept a block, that's because either you or they are forked; put another way, a block being rejected is an error by one or the other party except in the case of forks. (note how almost all of the rejection messages are things you should have been able to validate yourself) Giving our peers reasons gives opportunities to take action in response, which creates a whole new kind of consensus through rejection messages passing around information that we certainly don't want to make possible.

Having said that, responding to a block that passes the context independent AcceptBlock() tests, but is rejected because it wouldn't be the best blockchain tip, with what we do consider to be that tip is probably reasonable and would let that peer more quickly find out they are out of date.

ACK other reject strings.

@gmaxwell
Copy link

Hm. I'm not a fan of the block messages, I feel like they'll complicate already risky block validation code... and give non-deterministic results due to permissible differences in what order the tests are run in. WRT PT's "what we consider best"... you wouldn't use a reject message there: You'd INV your best block at the peer and let it pull it if its interested.

I'm wondering if the reject messages shouldn't have a some kind of simple field that indicates if the rejection is because the peer considers it invalid vs doesn't like it due to policy reasons... though I suspect that would run into order-non-determinism as its perfectly reasonable to apply stateless policy checks before stateful validity checks.

@petertodd
Copy link

@gmaxwell Quite correct.

Another issue with reject messages is if some alt-implementation maintainer sees blocks it's producing getting rejected, we want them to at bare minimum dig out a Bitcoin node and figure out why on that side, rather than making some hack that seems to work.

After all, if you really do have what is the valid best block, why do you care the other side rejected it? That's their problem. All you want to know is if every peer rejected it, so you can find new peers that aren't broken. You also want to see if your peers actually propagate to other peers, which suggests we shouldn't have any kind of block reject message at all - broadcast to most of your peers, and check if the ones you don't relay the block back to you.

You can apply the same logic to tx reject messages to of course, but unlike with blocks it is reasonable to change your tx's in response to a reject.

@gavinandresen
Copy link
Author

@petertodd: Which block rejection do you object to? I did not propose any reject message response corresponding to "not on the best block chain".

@gmaxwell: Switched to a HTTP-like status code, which should address your concerns (all of the "block is not valid" are same status code; people could still switch on human-readable string, we could discourage that by changing them every once in a while and making it clear that the human readable strings are not standardized.)

@petertodd
Copy link

@gavinandresen "not on the best block chain" would be the only informational rejection I wouldn't object to for now. Again, the issue is that whether or not a block is valid has a strict definition and we don't want people to start relying on their peers in any way about that.

@gavinandresen
Copy link
Author

@petertodd: I'm still not following. Are you worried that people will implement partially-validating nodes, go to all the work of computing a valid proof-of-work, and then rely on their peers to tell them "Nope, not a valid block, try again" ???

Really?

@simondlr
Copy link

If you have a valid block (from your pov of the chain), but a peer sends a reject maliciously that it is invalid (although it is), what does the node do? Send the block to other peers?

@petertodd
Copy link

@gavinandresen As I commented on the pull-request, put it another way: What actions do you expect implementations to take in response to a block rejection message?

@petertodd
Copy link

As for the "it's just for debugging" argument: again, we really don't want to encourage implementations to do anything other than investigate blocks being rejected in any way other than delving into the code deeply. It's critical that we get behavior absolutely exact for consensus, so making it easy to play "whack-a-mole" and fix bugs as they come up with patches that seem to fix the problem just invites behavior that we don't want.

@gavinandresen
Copy link
Author

@simondlr : the reference implementation does nothing by default when it receives "reject" messages. If you turn on debug logging, then it will log the first 111 bytes of the reject message in debug.log. It has no effect on what blocks are or are not relayed to peers.

@petertodd: objection duly noted, we simply disagree about whether this will encourage lazy re-implementors.

@kawalgrover
Copy link

So, with this implementation is it possible now, that if a message is perfectly valid, say the transactions inputs are all good and the fee is included, that there still exists a possibility that the nodes may decide to reject a message.

What if one of the big pools or a few of them got together and decide to reject all the messages that didn't come from one of them?

The motivation for this use case seems to be to avoid spamming (denial of service). I'm curious... has Bitcoin had to worry about DoS in this way in the past. Or are you guys pre-emptively trying to solve for something that has never happened.

PS: https://en.bitcoin.it/wiki/BIP_0061 is blank and I would have added this entire gist to the Wiki but it was locked.

@scalahub
Copy link

scalahub commented Jan 18, 2018

Can someone refer me to examples/test vectors of reject messages. Its a pain to generate them the hard way.

@DevRatnu
Copy link

Still this Protocol Running and work with Virtacoin VTA currency - Protocol 70002 as Sub-version : VirtaCoin:2.9.1
https://github.com/virtacoin/VirtaCoinProject

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment