Skip to content

Instantly share code, notes, and snippets.

@zsfelfoldi

zsfelfoldi/request_mapping.md

Last active February 23, 2024 14:30
Show Gist options
  • Save zsfelfoldi/1a86cd19fc727898053b56e996e8b6b4 to your computer and use it in GitHub Desktop.
Save zsfelfoldi/1a86cd19fc727898053b56e996e8b6b4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
request_mapping.md

Mapping ethereum chain/state access interfaces to trusted and trustless API calls

ethclient (trusted)

Package ethclient implements interface calls by directly calling the respective RPC API call. It requrired a trusted API endpoint because it does not verify the received results.

BlockByHash(ctx context.Context, blockHash common.Hash) (*types.Block, error)
    eth_getBlockByHash(blockHash, fullTxs=true)
BlockByNumber(ctx context.Context, blockNumber *big.Int) (*types.Block, error)
    eth_getBlockByNumber(blockNumber, fullTxs=true)
HeaderByHash(ctx context.Context, blockHash common.Hash) (*types.Header, error)
    eth_getBlockByHash(blockHash, fullTxs=false)
HeaderByNumber(ctx context.Context, blockNumber *big.Int) (*types.Header, error)
    eth_getBlockByNumber(blockNumber, fullTxs=false)
TransactionCount(ctx context.Context, blockHash common.Hash) (uint, error)
    eth_getBlockTransactionCountByHash(blockHash)
TransactionInBlock(ctx context.Context, blockHash common.Hash, index uint) (*types.Transaction, error)
    eth_getTransactionByBlockHashAndIndex(blockHash, txIndex)
SubscribeNewHead(ctx context.Context, ch chan<- *types.Header) (Subscription, error)
    eth_subscribe("newHeads")
TransactionByHash(ctx context.Context, txHash common.Hash) (tx *types.Transaction, isPending bool, err error)
    eth_getTransactionByHash(txHash)
TransactionReceipt(ctx context.Context, txHash common.Hash) (*types.Receipt, error)
    eth_getTransactionReceipt(txHash)
BalanceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (*big.Int, error)
    eth_getBalance(blockNumber)
StorageAt(ctx context.Context, account common.Address, key common.Hash, blockNumber *big.Int) ([]byte, error)
    eth_getStorageAt(blockNumber, account, key)
CodeAt(ctx context.Context, account common.Address, blockNumber *big.Int) ([]byte, error)
    eth_getCode(blockNumber, account)
NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error)
    eth_getTransactionCount(blockNumber, blockHash)
CallContract(ctx context.Context, call CallMsg, blockNumber *big.Int) ([]byte, error)
    eth_call(blockNumber, msg)
BlockNumber(ctx context.Context) (uint64, error)
    eth_blockNumber
FilterLogs(ctx context.Context, q FilterQuery) ([]types.Log, error)
    eth_getLogs(query)

lightclient (trustless)

Package ethclient/lightclient implements interface calls in a trustless way. It requires both an execution layer RPC API endpoint and a consensus layer beacon REST API endpoint, neither of which needs to be trusted. Current chain head is determined by a beacon chain light client. Interface functions are implemented through intermediate structures canonicalChain, blocksByHash, transactions and state. This intermediate layer ensures verification of data retrieved from the API. Additionally, it can also optimize performance by caching retrieved data and ensuring that no request is sent twice simultaneously.

BlockByHash(ctx context.Context, blockHash common.Hash) (*types.Block, error)
    blocksByHash.getBlock(blockHash)
        eth_getBlockByHash(blockHash, fullTxs=true)
BlockByNumber(ctx context.Context, blockNumber *big.Int) (*types.Block, error)
    canonicalChain.getHash(blockNumber)
        eth_getBlockByHash(blockHash, fullTxs=false)
        // Note: see historical_execution_proof below
    blocksByHash.getBlock(blockHash)
        eth_getBlockByHash(blockHash, fullTxs=true)
HeaderByHash(ctx context.Context, blockHash common.Hash) (*types.Header, error)
    blocksByHash.getPartialBlock(blockHash)
        eth_getBlockByHash(blockHash, fullTxs=false)
HeaderByNumber(ctx context.Context, blockNumber *big.Int) (*types.Header, error)
    canonicalChain.getHash(blockNumber)
        eth_getBlockByHash(blockHash, fullTxs=false)
        // Note: see historical_execution_proof below
    blocksByHash.getPartialBlock(blockHash)
        eth_getBlockByHash(blockHash, fullTxs=false)
TransactionCount(ctx context.Context, blockHash common.Hash) (uint, error)
    blocksByHash.getPartialBlock(blockHash)
        eth_getBlockByHash(blockHash, fullTxs=false)
TransactionInBlock(ctx context.Context, blockHash common.Hash, index uint) (*types.Transaction, error)
    blocksByHash.getBlock(blockHash)
        eth_getBlockByHash(blockHash, fullTxs=true)
        // Note: see eth_getTransactionInclusionProof below
SubscribeNewHead(ctx context.Context, ch chan<- *types.Header) (Subscription, error)
    canonicalChain.subscribeNewHead(ch)
        /eth/v1/beacon/light_client/bootstrap
        /eth/v1/beacon/light_client/updates
        /eth/v1/events?topics=light_client_optimistic_update
TransactionByHash(ctx context.Context, txHash common.Hash) (tx *types.Transaction, isPending bool, err error)
    transactions.getTxAndPosition(txHash)
        eth_getTransactionByHash(txHash)
        blocksByHash.getPartialBlock(blockHash)	// can prove inclusion, cannot prove pending state			
            eth_getBlockByHash(blockHash, fullTxs=false)
            // Note: see eth_getTransactionInclusionProof below
TransactionReceipt(ctx context.Context, txHash common.Hash) (*types.Receipt, error)
    transactions.getTxAndPosition(txHash)
        eth_getTransactionByHash(txHash)
        blocksByHash.getPartialBlock(blockHash)	// can prove inclusion, cannot prove pending state			
            eth_getBlockByHash(blockHash, fullTxs=false)
            // Note: see eth_getTransactionInclusionProof below
    transactions.getReceipt(blockHash, txIndex)	
        blocksByHash.getPartialBlock(blockHash)  // header to verify receipts against
        eth_getBlockReceipts(blockHash)
        // Note: see eth_getReceiptWithProof below
BalanceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (*big.Int, error)
    state.getProof(blockNumber, account, {})
        eth_getProof(blockNumber, account, {})
StorageAt(ctx context.Context, account common.Address, key common.Hash, blockNumber *big.Int) ([]byte, error)
    state.getProof(blockNumber, account, {key})
        eth_getProof(blockNumber, account, {key})
CodeAt(ctx context.Context, account common.Address, blockNumber *big.Int) ([]byte, error)
    state.getProof(blockNumber, account, {})
        eth_getProof(blockNumber, account, {})
    state.getCode(blockNumber, account)
NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error)
    state.getProof(blockNumber, account, {})
        eth_getProof(blockNumber, account, {})
CallContract(ctx context.Context, call CallMsg, blockNumber *big.Int) ([]byte, error)
    state.getProof(blockNumber, account, {keys...})
        eth_getProof(blockNumber, account, {keys...})
        // Note: see eth_callWithProof below
    canonicalChain.getHash(blockNumber)
BlockNumber(ctx context.Context) (uint64, error)
    canonicalChain.getHead()
FilterLogs(ctx context.Context, q FilterQuery) ([]types.Log, error)
    logs.filter(query)
        eth_getBlockByHash(blockHash, fullTxs=false)
        eth_getBlockReceipts(blockHash)

Proposed light client-friendly API extensions

  • eth_getTransactionInclusionProof(txHash)
    • returns the header of the canonical block where the transaction is included, along with the position index and the merkle proof of the transaction in the Transactions trie.
  • eth_getReceiptWithProof(blockHash, index)
    • returns the receipt belonging to the canonical transaction at the specified position, along with the merkle proof of the transaction in the Receipts trie.
  • eth_callWithProof(blockNumber/blockHash, callMsg)
    • executes a contract call on the state of the specified block and returns a merkle multiproof of all touched state entries.
  • /eth/v1/beacon/light_client/historical_execution_proof/<blockNumber>
    • returns the CL slot associated with the specified EL block number, along with a Merkle proof rooted in the current head beacon state, through either the current state_roots or historical_summary and a historical state_roots, to the execution block hash referenced in the historical beacon state.
    • Note: safely determining the canonical block by number is currently only possible by reverse syncing headers from the head proven by the latest optimistic_update from the CL API. This is only practically feasible for recent canonical blocks. historical_execution_proof would make it possible to prove older blocks too, at the cost of requiring beacon nodes to store this data (currently it is not required from them to store either the historical state roots or the execution block hash proofs of old blocks).
Finding transactions by hash

Since there is no global consensus registry of transactions by hash, it is impossible to prove in the general case that a transaction does not exist or is not canonical. If eth_getTransactionByHash returns inclusion info, we can verify that but if it does not then all we can do is ask a few other endpoints about is and only believe that it's not canonical if none of them returns valid inclusion info. Note that we can safely verify the status of a transaction with one condition though: if we know when the transaction was published and this moment is still fairly recent. In this case we can get all canonical transaction hashes by calling eth_getBlockByHash(blockHash, fullTxs=false) for all blocks after the given moment. In most cases this could work well for tracking locally created transactions until they become safely confirmed.

types.Receipt non-consensus fields

We can assume that the corresponding transaction and its inclusion position are known (the receipts are fetched based on this info).

  • TxHash, ContractAddress, BlobGasUsed: based on full transaction data
  • GasUsed: derived from CumulativeGasUsed
  • EffectiveGasPrice: based on header.BaseFee and transaction.MaxFee/MaxTip
  • BlobGasPrice: based on header.excess_blob_gas
  • BlockHash, BlockNumber, TransactionIndex: inclusion info is always known when the receipt is known
Searching log events efficiently

This will be the topic of a subsequest proposal. Implementing log search efficiently will require a consensus change that introduces a new helper data structure instead of the bloom filters found in the log headers, which have a fixed size and are overpopulated, giving lots of false positives and not really helping a lot with performance. What's possible right now is syncing all headers in the range (which also requires syncing headers from head to the end of the given range as there is no other way to know the canonical chain), then getting all block receipts for all blocks where there was a bloom filter match (which is a big portion of all blocks in the range). In practice this only works for searching recent and short chain sections.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment