The Bitcoin P2P network is fundamental to the strength of Bitcoin itself. However, as it stands we know very little about the peers we connect to. This makes it difficult to have data-driven conversations about making changes to the P2P protocol. Additionally, we connect and disconnect from peers in a naïve fashion, simply because we are lacking in the information we would need to make better decisions.
We need more data.
To remedy this problem, I propose the following extensions to Bitcoin Core's per-peer logging abilities.
- Message Capture
This will enable Bitcoin Core to dump all messages from specified peers to a log file. An additional tool will also be built to display and perform analytics on the log, and will be able to replay messages back to the node.
This will be turned on and off by a command-line argument and an RPC.
- Resource Tracking
This will allow us to measure, on a per-peer basis, a variety of metrics. This information will be available over the RPC and can be periodically dumped to a log file. Some ideas:
- Total time spent in ProcessMessages and SendMessages
- Total number of messages of each type processed
- Number of transactions accepted/rejected from the peer
- Number of new blocks this peer has served us
- Ratio between announced transactions and fetched transactions in both directions
- Average time between asking for a transaction and receiving it from the peer
- Frequency of compact blocks usage, and in which direction
- Memory usage of all peer-specific data structures
- Global data structure memory usage assigned proportionally to the peers that contributed to it
- Aggregate "total CPU", "total I/O", "total Network" measures
For both of these aspects, nodes will be able to be filtered flexibly.
- Have nodes react automatically to these metrics by intelligently choosing peers peers:
- Selfishly, to improve personal IBD speed
- Altruistically, to improve the "overall health" of the network
- Ensure that this behavior can not be gamed in some way by malicious nodes.
- Justify a choice for the total number of connections and the ratio of types (inbound to outbound to block-relay-only, etc.)
- ML 🙃
- Can anyone think of conversations (either ongoing, historical, or hypothetical) where the data coming from this project would be useful?
- Is anyone aware of papers/discussions proposing global metrics that could be used to look at our P2P network?
- I'm aware of some of the standard network analysis techniques, but I haven't yet found examples of them being applied to the Bitcoin P2P network.
- Are there other ongoing projects that could benefit from this sort of data? I would like to integrate this as closely as possible to the rest of the ecosystem.
- #6589
- Per-peer usage tracking
- Number of bytes sent and received, grouped by message type
Recently, see bitcoin/bitcoin#18991 (comment). But you can grep for historical big P2P changes, it should be pretty easy to find examples