eth2-network-testing-strategy.md

Testing strategy

We want to test for different scenario using a list of steps

• Client needs testing setup ability:

  • Pre-load state
  • Pre-load blocks
  • Advanced; pre-load fork-choice

• Scripting needs:

  • Actor model (names, or numbered)
  • Result codes
  • Grep from log
  • Timeouts
  • Detect client crash, extract logs etc.

• rumor options:

  • -i -> REPL
  • pipe in commands (with actors)
  • specify script file

• Python lib to call rumor

• Test for RPC correctness

  • Test for valid responses/requests
  • Test for invalid inputs
  • Test for server errors being returned on requests
  • Out of order responses, other invalid outputs

• Test Sync RPC

  • Regular by-range
  • Regular by-root
  • Sharded by-range
  • Edge-cases for each; e.g. missing chunk, ordering, quick end, bogus extra bytes, no EOF/reset etc.
  • Test all with and without compression
  • Test invalid compression
  • Test message size problems, e.g. 1 MB status

• Test gossipsub;

  • Put messages on network
    • valid
    • invalid

• Test connection, subsequent status req/resp, etc.

• Test obvious connection management issues

  • E.g. sudden re-connects (lots of nodes, short repeats, etc.)

• Test MUX choice

  • support yamux
  • support mplex

• Test security choice

  • support secio
  • support noise

• Test noise

  • Try handshake
  • Try dummy RPC over noise

• Discv5: ask felix