Skip to content

Instantly share code, notes, and snippets.

@EyeOfPython
Created May 10, 2021 22:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save EyeOfPython/e43716e1c0de4a155f8b26a25b986d5c to your computer and use it in GitHub Desktop.
Save EyeOfPython/e43716e1c0de4a155f8b26a25b986d5c to your computer and use it in GitHub Desktop.
Plugin system for Bitcoin-like blockchains

TCP Plugin system

Allows plugins to connect to nodes.

Message Layout

Messages have the same format as on the P2P network:

Name Data type Description
magic uint32_t Magic message value. Different than the P2P network one for plugin messages
command char[12] ASCII string identifying the packet content, zero-padded.
checksum uint32_t First 4 bytes of sha256(sha256(payload))
payload uint8_t[] Data of the message

Magic values for different networks for plugins:

Network Magic
Lotus Mainnet cc d0 cc c7
Lotus Testnet d4 cc d0 cc
Lotus Regtest d2 cc d0 cc
BCH Mainnet c3 d0 cc c7
BCH Testnet d4 c3 d0 cc
BCH Regtest d2 c3 d0 cc
ABC Mainnet c5 d0 cc c7
ABC Testnet d4 c5 d0 cc
ABC Regtest d2 c5 d0 cc

Rationale: Each mainnet is uniquely identified by the first byte. Each testnet starts with d4, each regtest network starts with d2.

Encryption

Messages are not encrypted. Plugins are meant to be run on the same network or even machine than the node, where encryption would get in the way.

For plugin-to-plugin communication, we use an entirely different protocol based on WebSockets, which has encryption built-in.

Handshake Messages

These messages are required for a plugin to login into a node.

plghello

plghello gets sent by plugins in order to initiate logging into a node.

Name Data type Description
protocol uint32_t Version of the plugin protocol of the plugin
addr_recv net_addr The network address of the node receiving this message
addr_send net_addr The network address of the plugin sending this message

plghelloback

plghelloback gets sent by nodes in reply to a plghello message and has the same encoding as a plghello message.

Name Data type Description
protocol uint32_t Version of the plugin protocol of the plugin
addr_recv net_addr The network address of the plugin receiving this message
addr_send net_addr The network address of the node sending this message

plgchallenge

plgchallenge gets sent by the node after a plghelloback message, if a challenge is required. Otherwise, it sends a plgack message directly.

Name Data type Description
challenges Challenge[] List of challenges this node accepts for authentication

Challenge

Name Data type Description
auth_method char[6] Authentication method, e.g. "userpw", "cookie", "sig" etc.
auth_payloads uint8_t[][] List of byte arrays for this challenge, which all have a special meaning depending on the auth method (e.g. a nonce/public key/etc.)

plglogin

plglogin gets sent by a plugin to a node in reply to a plgchallenge message.

It contains various human-readable info, which is helpful for node maintainers who want to keep track of running plugins.

Name Data type Description
name var_str Name of the plugin
version_str var_str Plugin version string, following semantic versioning for the plugin, e.g. "0.1", "1.1", "3.2-beta"
hash uint8_t[] Hash of the plugin, e.g. git commit hash of the plugin repository, or the SHA256 of the source code, etc.
auth_method char[6] Authentication method picked by the plugin to log in
auth_responses uint8_t[][] List of byte arrays as response to the challenge, which all have a special meaning depending on the auth method (e.g. username/password/signature/etc.)

plgack

plgack gets sent by a node in reply to a successful plglogin message, or directly after a plghelloback message, and provides some data about the node to the plugin.

Name Data type Description
version int32_t Node version, same as in the version message of the P2P protocol
services uint64_t Bitfield of features the node supports, same as in the version message of the P2P protocol
timestamp int64_t UNIX timestamp of the node
user_agent var_str User Agent, same as in the version message of the P2P protocol
best_height int32_t Last block height received by the node, same as in the version message of the P2P protocol
best_blockhash uint8_t[32] Last block hash received by the node

plgbye

Notifies a plugin that it should shut down gracefully. Contains no additional payload. Only sent if the plugin is managed by the node.

Node notifications

These messages are sent by a node to a plugin in order to inform it about an update on the blockchain.

updateblktip

Notifies a plugin when the block chain tip advances. When multiple blocks are connected at once, updateblktip will be sent on the final tip but may not be sent on every intermediate tip. If the latter behavior is desired, subscribe to blkconnected instead.

Name Data type Description
blockhash uint8_t[32] Block hash of the best block

mempooltxadd

Notifies a plugin that a transaction has been added to the mempool.

Encoding is the same as tx messages on the P2P layer.

mempooltxrem

Notifies a plugin that a transaction have been removed from the mempool.

This notification fires for transactions that are removed from the mempool for the following reasons:

  • EXPIRY (expired from mempool after -mempoolexpiry hours)
  • SIZELIMIT (removed in size limiting if the mempool exceeds -maxmempool megabytes)
  • REORG (removed during a reorg)
  • CONFLICT (removed because it conflicts with in-block transaction)

This does not fire for transactions that are removed from the mempool because they have been included in a block. Any client that is interested in transactions removed from the mempool for inclusion in a block can learn about those transactions from the blkconnected notification.

Transactions that are removed from the mempool because they conflict with a transaction in the new block will have mempooltxrem messages sent before the blkconnected event is fired. If multiple blocks are connected in one step, then the ordering could be:

  • mempooltxrem with tx1 from block A
  • mempooltxrem with tx2 from block A
  • mempooltxrem with tx1 from block B
  • mempooltxrem with tx2 from block B
  • blkconnected with block A
  • blkconnected with block B
Name Data type Description
txid TxId ID of the transaction that's being evicted from the mempool

blkconnected

Notifies a plugin that a block has been added to the blockchain.

Provides a vector of transactions evicted from the mempool. Conflicting transactions have been removed before this message with mempooltxrem.

Name Data type Description
block_meta uint8_t[] Block metadata, i.e. header of the connected block + extended metadata (if supported)
txids TxId[] IDs of the transactions included in the block

blkdisconctd

Notifies a plugin that a block has been disconnected from the blockchain.

Name Data type Description
blockhash uint8_t[32] Block hash of the best block
txids TxId[] IDs of the transactions included in the block

blkflushed

Notifies a plugin of the new active block chain on-disk.

Prior to this message, any updates are not guaranteed to persist on disk (ie clients need to handle shutdown/restart safety by being able to understand when some updates were lost due to unclean shutdown).

When this message is sent, the validation changes done by any prior messages are guaranteed to exist on disk and survive a restart, including an unclean shutdown.

Provides a block hash describing the best chain, which is likely useful for storing current state on disk in client DBs.

Name Data type Description
blockhash uint8_t[32] Block hash of the committed block

blkchecked

Notifies a plugin of a block validation result. If the provided validation state is valid, the provided block is guaranteed to be the current best block at the time the message was sent (not necessarily now)

Name Data type Description
blockhash uint8_t[32] Block hash of the committed block
validation_state char[8] Validation state of the block (e.g. "valid", "invalid", etc.)
validation_msg var_str Human-readable message about the validation (e.g. why a block failed etc.)

powvalidblk

Notifies listeners that a block which builds directly on our current tip has been received and connected to the headers tree, though not validated yet.

Name Data type Description
header CBlockHeader Block header of the connected header

Plugin queries & replies

These messages allow plugins to query a node for data, e.g. for initial indexing of the plugin.

getrawblocks

Gets sent by a plugin to a node to get raw block data from a range of block heights.

Name Data type Description
start_height int32_t Height of the beginning of blocks being requested
num_blocks int32_t Number of blocks to be fetched in this call, can be at most 100

rawblock

Gets sent by a node in response to a raw block message.

Name Data type Description
start_height int32_t Height of the beginning of blocks being requested
blocks CBlock[] A list of blocks using CBlock serialization
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment