It is desirable to have a way to define valid Test Cases in the specification.
This would allow to:
- fully clarify the specification and detail expected outcome
- improve implementations compatibility
- allow for automated tests of implementations
The following is a proposed syntax allowing this. It builds on top of current usage of example in the specification while being light and OpenAPI compliant.
Proposed syntax consist of:
- defining a top-level
x-test-casesmap property - having matching (i.e. same id)
examplesper component (parameters,requestBodyandresponse)
The combination of parameters, requestBody and associated path properties form the Request. The result of the execution of this Request by a node must match the associated Response definition.
x-test-cases is a Map[string, TestCase object] (defined as an OpenAPI extension) allowing to list high-level details of all Test Cases per path, with TestCase object defined as:
descriptiona high-level description of theTestCaseexpectedResponse
If a response is too big to be added inline, it can be defined as a URI via externalValue.
Here is an example:
post:
tags:
- Beacon
- ValidatorRequiredApi
summary: "Publish a signed block."
operationId: "publishBlindedBlockV2"
description: |
Instructs the beacon node to use the components of the `SignedBlindedBeaconBlock` to construct and publish a
`SignedBeaconBlock` by swapping out the `transactions_root` for the corresponding full list of `transactions`.
The beacon node should broadcast a newly constructed `SignedBeaconBlock` to the beacon network,
to be included in the beacon chain. The beacon node is not required to validate the signed
`BeaconBlock`, and a successful response (20X) only indicates that the broadcast has been
successful. The beacon node is expected to integrate the new block into its state, and
therefore validate the block internally, however blocks which fail the validation are still
broadcast but a different status code is returned (202). Before Bellatrix, this endpoint will accept
a `SignedBeaconBlock`. The broadcast behaviour may be adjusted via the `broadcast_validation`
query parameter.
+ x-test-cases:
+ ValidRequest:
+ description: "A valid request"
+ expectedResponse: "200"
+ InvalidRequest:
+ description: "A request missing proper signature"
+ expectedResponse: "400"
parameters:
- name: broadcast_validation
in: query
required: false
description: |
Level of validation that must be applied to a block before it is broadcast.
Possible values:
- **`gossip`** (default): lightweight gossip checks only
- **`consensus`**: full consensus checks, including validation of all signatures and
blocks fields _except_ for the execution payload transactions.
- **`consensus_and_equivocation`**: the same as `consensus`, with an extra equivocation
check immediately before the block is broadcast. If the block is found to be an
equivocation it fails validation.
If the block fails the requested level of a validation a 400 status MUST be returned
immediately and the block MUST NOT be broadcast to the network.
If validation succeeds, the block must still be fully verified before it is
incorporated into the state and a 20x status is returned to the caller.
schema:
$ref: '../../../beacon-node-oapi.yaml#/components/schemas/BroadcastValidation'
+ examples:
+ ValidRequest:
+ value: gossip
+ InvalidRequest:
+ value: gossip
- in: header
schema:
$ref: '../../../beacon-node-oapi.yaml#/components/schemas/ConsensusVersion'
required: true
name: Eth-Consensus-Version
description: "Version of the block being submitted."
+ examples:
+ ValidRequest:
+ value: altair
+ InvalidRequest:
+ value: altair
requestBody:
description: "The `SignedBlindedBeaconBlock` object composed of `BlindedBeaconBlock` object (produced by beacon node) and validator signature."
required: true
content:
application/json:
schema:
oneOf:
- $ref: '../../../beacon-node-oapi.yaml#/components/schemas/SignedBeaconBlock'
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Altair.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.SignedBlindedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Capella.SignedBlindedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Deneb.SignedBlindedBeaconBlock"
+ examples:
+ ValidRequest:
+ value:
+ message: ...
+ signature: ...
+ InvalidRequest:
+ value:
+ message: ...
application/octet-stream:
schema:
description: "SSZ serialized block bytes. Use content type header to indicate that SSZ data is contained in the request body."
responses:
"200":
description: "The block was validated successfully and has been broadcast. It has also been integrated into the beacon node's database."
"202":
description: "The block failed validation, but was successfully broadcast anyway. It was not integrated into the beacon node's database."
"400":
description: "The `SignedBlindedBeaconBlock` object is invalid or broadcast validation failed"
content:
application/json:
schema:
$ref: "../../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
- code: 400
- message: "Invalid block: missing signature"
+ examples:
+ InvalidRequest:
+ value:
+ code: 400
+ message: "Invalid block: missing signature"
"415":
$ref: '../../../beacon-node-oapi.yaml#/components/responses/UnsupportedMediaType'
"500":
$ref: '../../../beacon-node-oapi.yaml#/components/responses/InternalError'
"503":
$ref: '../../../beacon-node-oapi.yaml#/components/responses/CurrentlySyncing'This proposition takes inspiration from the following works: