MetropolisTestingGuide.md

Edit 8/14/18: See https://ethereum-tests.readthedocs.io/en/latest/ for more updated information. Much of the links below are still valuable though.

Welcome

This guide is intended to provide resources for those wanting to help test Metropolis EIPs. The CPP team is currently in the middle of a migration from EthDocs to a documentation site that is more dedicated to CPP-Ethereum so the documentation on creating tests for Ethereum using testeth is scattered. Everything you will need to get started should be compiled below.

Suggested skill sets needed to create and run tests

1. Ability to compile/build testeth and LLL compiler or run a docker file.
2. Ability to understand the LLL Ethereum language.
3. Ability to understand EIPs, particuarly those that are going to be going into the Metropolis hard fork.

Now What Do I Do?

1. Create a GitHub account if you do not already have one.
2. Read the resources below to learn about LLL, EVM, testeth, and EIPs.
3. Join the Gitter chat room for Ethereum testers at https://gitter.im/ethereum/tests. Introduce yourself and ask questions if you get confused or need guidance on what to do.
4. Create and/or run consensus tests by following the guide here: Yoichi's Guide to Generate Consensus Tests using testeth. If you create new tests open a pull request in the appropriate repository.
5. Document your learnings and correct/add to current documentation.

Resources to Help You Get Started

LLL and EVM Stack Resources

LLL is a high-level language in Ethereum used to write smart contracts. LLL is considered a more difficult to learn and write than Solidity because it deals directly with a lot of the low level OPCODEs and features of the Ethereum Virtual Machine (EVM). If you have familiarity with stack machines or have ever taken a course on assembly languages you will be able to pick this up.

• Understanding the Ethereum Blockchain Protocol (Video)
• An introduction to LLL for Ethereum Smart Contract Development by Daniel Ellison
• Building and installing Ethereum compilers by Daniel Ellison
• Building and installing Ethereum compilers screencast video by Daniel Ellison
• Compiling an LLL Contract for the First Time by Daniel Ellison
• Deploying your first LLL contract — Part 1
• Deploying your first LLL contract — Part 2
• The Resurrection of LLL blog posts by Daniel Ellison (this has 6 total parts)
• Advanced Solidity Tutorial by Andreas Olofsson (especially "EVM Stack and Assembler" section)
• Ethereum Frontier Guide on state machine and opcodes (possibly outdated)
• Ethereum Frontier Guide on state machine and opcodes (possibly outdated)
• Ethdocs Guide to Estimating Gas
• Reddit thread with links to old LLL code
• Old Ethereum Github Wiki page with links to LLL specs and examples
• Old LLL code by Andreas Olofsson

testeth/cpp-ethereum Resources

• Metropolis Tests - List of Tasks
• Yoichi's Guide to Generate Consensus Tests using testeth
• Testing Ethereum Consensus Video from Devcon1
• @winsvega's testeth Docker image
• ethereum/tests repo
• EthDocs.org Ethereum Testing Section [Note: Outdated]
• ethereum/tests Wiki (Parts may be outdated)

Metropolis EIP Resources

• List of Metropolis EIPs
• Yoichi's Blog Summarizing Metropolis EIPs
• "What happens when Metropolis EIPs change" - blog by Yoichi
• Bi-monthly Ethereum core developer meeting notes/video where devs discuss EIPs

Help/Chat

• Ethereum StackExchange for technical questions about Ethereum
• Gitter chat room for Ethereum testers

We are also looking for contributors to ethereumjs-vm (more info soon)

Video Overview Resources

• Testing Ethereum Consensus

Nice share, @tcsiwula - good addition to @pirapira 's generating_tests.rst documentation.

"Yoichi's Guide to Generate Consensus Tests using testeth" should link to https://github.com/ethereum/cpp-ethereum/blob/develop/doc/generating_tests.rst

@tcsiwula and @cdetrio: Updated guide with your changes. Thanks!

@Souptacular -
Would it be useful to add Yoichi's list of tasks?
Metropolis Tests

Resources that I've found useful to get a high-level understanding of what is being tested, as a programmer without experience in the ethereum protocol (other devs might be able to relate):
Understanding the Ethereum Blockchain Protocol

Docs from Frontier and Homestead, even if possibly outdated:
https://ethereum.gitbooks.io/frontier-guide/opcodes,_costs,_and_gas.html
https://ethereum.gitbooks.io/frontier-guide/contracts_and_transactions_intro.html
http://www.ethdocs.org/en/latest/contracts-and-transactions/account-types-gas-and-transactions.html#estimating-transaction-costs

@sdtsui Added!

I've started reading the yellow paper, and will finish that before I start going through the docs on this page. I've already read the whitepaper and was going through the Solidity docs.

I'm still getting 404 for "Yoichi's Guide to Generate Consensus Tests using testeth".

The yellow paper involves a lot of further reading, e.g. about maths and cryptography, in order to understand it. So I might just jump to learning LLL as it's probably more useful to know how to use that.

I also got a 404 error for compile/build testeth and LLL compiler or run a docker file.

For the latest testing documentation, refer to http://ethereum-tests.readthedocs.io/.
This latest documenation may make other links to testing documentation in this post outdated, which I have been through and provided a reference to the latest site above for articles that discuss testing Ethereum.

As I mentioned before,

I also got a 404 error for compile/build testeth and LLL compiler or run a docker file.

That link can be replaced with the https://ethereum-tests.readthedocs.io/en/latest/ link, like so:

Ability to compile/build testeth.

For:

Create and/or run consensus tests by following the guide here: Yoichi's Guide to Generate Consensus Tests using testeth.

This should be changed to:
Create and/or run consensus tests by following the previously mentioned guide here: https://ethereum-tests.readthedocs.io/en/latest/.

@pirapira, @winsvega, @holgerd77, @Souptacular