Skip to content

Instantly share code, notes, and snippets.

@vindard

vindard/Ethereum-research.md

Last active July 20, 2020 12:18
Show Gist options
  • Save vindard/7983130c473a7ac2b928ec3363987a73 to your computer and use it in GitHub Desktop.
Save vindard/7983130c473a7ac2b928ec3363987a73 to your computer and use it in GitHub Desktop.
Download ZIP
A list of resources I gathered when researching for my post about validating consensus within Ethereum vs. in Bitcoin
Raw
Ethereum-research.md

Ethereum research

The following is a literal dump of resources I collected with associated comments when I was researching how exactly consensus gets built in Ethereum vs. in Bitcoin for this Medium article.

It is a very raw list that I felt could be useful for anyone else who read that article and who may also be researching this topic.

Articles:

  1. Afri’s first attempt at explaining this https://dev.to/5chdn/the-ethereum-blockchain-size-will-not-exceed-1tb-anytime-soon-58a

  2. Afri geth & parity comparison sync cheat sheet: https://dev.to/5chdn/ethereum-node-configuration-modes-cheat-sheet-25l8

  3. Vitalik on state-tree pruning (not as relevant) https://blog.ethereum.org/2015/06/26/state-tree-pruning/

  4. Afri’s check website http://didtheethereumblockchainreach1tbyet.5chdn.co/

  5. Very good ‘latest’ attempt at explaining full nodes vs. archival nodes https://medium.com/coinmonks/how-a-pruned-ethereum-node-can-fully-verify-the-blockchain-bbe9f29663ed

  6. An equally excellent attempt at addressing this entire issue https://www.tokendaily.co/blog/dispelling-myths-about-ethereum-s-disk-space

  7. Getting Deep Into Ethereum: How Data Is Stored In Ethereum? https://hackernoon.com/getting-deep-into-ethereum-how-data-is-stored-in-ethereum-e3f669d96033

  8. Breakdown of an Ethereum block https://ethereum.stackexchange.com/questions/10548/what-does-every-field-in-block-means

  9. Where is the state data stored? https://ethereum.stackexchange.com/questions/359/where-is-the-state-data-stored

  10. Getting Deep Into Geth: Why Syncing Ethereum Node Is Slow https://hackernoon.com/getting-deep-into-geth-why-syncing-ethereum-node-is-slow-1edb04f9dc5

  11. Ethereum design rationale https://ethereumbuilders.gitbooks.io/guide/content/en/design_rationale.html

  12. Chain Reorganisation Depth Expectations https://blog.ethereum.org/2015/08/08/chain-reorganisation-depth-expectations/

  13. On Ethereum proposal to charge state rent https://www.ethnews.com/to-alleviate-ethereum-state-bloat-developers-consider-charging-rent

  14. Deploying a Smart Contract the Hard Way https://medium.com/@gus_tavo_guim/deploying-a-smart-contract-the-hard-way-8aae778d4f2a

  15. Getting Deep Into Ethereum: How Data Is Stored In Ethereum? https://hackernoon.com/getting-deep-into-ethereum-how-data-is-stored-in-ethereum-e3f669d96033

  16. Transaction ordering inside each block https://ethereum.stackexchange.com/questions/6107/what-is-the-default-ordering-of-transactions-during-mining-in-e-g-geth

  17. Transaction & Receipts tries relationship https://ethereum.stackexchange.com/questions/5888/relationship-between-transaction-trie-and-receipts-trie

  18. Inside an Ethereum transaction https://medium.com/@codetractio/inside-an-ethereum-transaction-fa94ffca912f (the incrementing nonce is what ensures transaction intra-block ordering for multiple transactions from same account)

  19. Another on transaction anatomy https://thecoinoffering.com/learn/ethereum-transactions/

  20. Ethereum anatomy of block https://ethereum.stackexchange.com/questions/10548/what-does-every-field-in-block-means (note only transactions explicitly stored. State (storage) and txn receipt data are stored elsewhere and hashed in)

  21. Ethereum block architecture https://ethereum.stackexchange.com/a/31869

  22. Yellow paper for thorough details on block makeup, state & receipt tries https://ethereum.github.io/yellowpaper/paper.pdf

  23. Transaction receipts are derivable (they more contain contract execution data (like token transfers, but necessary for consensus? Can the receipt change on replay of just transactions?) https://ethereum.stackexchange.com/a/1309 && https://ethereum.stackexchange.com/a/8258

  24. Has a good diagram on how the state trie and hashed nodes are made up (storageRoot vs codeHash) https://medium.com/@preethikasireddy/how-does-ethereum-work-anyway-22d1df506369

  25. Very good piece on the tries and merkle structures https://medium.com/shyft-network-media/understanding-trie-databases-in-ethereum-9f03d2c3325d

  26. A question about the nature of the Storage trie https://ethereum.stackexchange.com/questions/55461/how-to-store-storage-trie-in-leveldb

  27. Aha! It looks like storage is separate from code. The immutable code (logic) uses the storage array (trie) to store variable values and other types of data https://programtheblockchain.com/posts/2018/03/09/understanding-ethereum-smart-contract-storage/

  28. Some articles about storage and its costs https://medium.com/@hayeah/diving-into-the-ethereum-vm-part-2-storage-layout-bc5349cb11b7 && https://medium.com/@hayeah/diving-into-the-ethereum-vm-part-2-storage-layout-bc5349cb11b7

  29. Ohhh, 'warp’ in parity means that you warp to the current block by downloading and checking just block headers (like SPV) and then fully validate from there whereas 'no-warp’ means you start validating everything from the start. Also, parity warp == geth fast https://twitter.com/gakonst/status/1084059570560008193 && https://twitter.com/5chdn/status/1084105801889910784

  30. Get sync process, state tries are downloaded/crawled instead of derived??? [see point 33.] ethereum/go-ethereum#16218 (comment)

  31. Extremely good explanation of Patricia Trees and other data tries in Ethereum (read prior episodes as well) https://medium.com/coinmonks/data-structure-in-ethereum-episode-3-patricia-trie-b7b0ccddd32f

  32. On syncing, transaction spam DoS attack that slows validation to a crawl between blocks 2.4 - .2.7 million https://ethereum.stackexchange.com/questions/18334/why-did-the-fast-sync-stop-at-2-4-million-blocks-when-updating-my-geth

  33. With ‘fast sync’ you see this hang where there’s always 64/65 or 100 blocks left because after warp download of blocks & PoW header confirmation the node steps back 100 (or 64) blocks and starts validating the state come forward (has to download as well because it didn’t do the work of deriving from genesis). [see point 30.] If 64/65/100 takes this long, will a ‘full’ sync from genesis take proportionally longer?! https://developerperspective.com/2018/06/07/how-to-unstuck-syncing-ethereum/ && https://hackernoon.com/getting-deep-into-geth-why-syncing-ethereum-node-is-slow-1edb04f9dc5#aad6

    looks like answer is "no" to that "proportionally longer" question

  34. On why “Merkle Patricia Trees” everywhere https://ethereum.stackexchange.com/questions/66729/why-are-transaction-tries-in-ethereum-not-plain-merkle-tries

  35. Transaction Trie vs Transaction Receipt Trie https://ethereum.stackexchange.com/questions/5888/relationship-between-transaction-trie-and-receipts-trie/5959#5959

  36. Deep dive on fast sync (note the “Weakness” section) ethereum/go-ethereum#1889

  37. A breakdown of how state works and how blockchain is constructed https://medium.com/cybermiles/diving-into-ethereums-world-state-c893102030ed

  38. Why you can’t get progress from a state-trie download/sync during --fast sync ethereum/go-ethereum#16202 (comment)

  39. Difference with archival sync vs. normal full sync (why it’s slower) ethereum/go-ethereum#18984 (comment)

  40. An attempt to improve the disk i/o load from 39. above ethereum/go-ethereum#17814

  41. The way fast sync vs. full sync works (header, body, state vs. header + block) ethereum/go-ethereum#17814 (comment)

  42. Why all 3 tries are part of consensus https://blog.ethereum.org/2015/11/15/merkling-in-ethereum/

  43. Question posed by me about what “block download” means ethereum/go-ethereum#16218 (comment)

  44. Ethereum Block Architecture https://ethereum.stackexchange.com/a/31869

  45. Where someone actually creates an archival node solely from a full node (no archive) https://medium.com/@marcandrdumas/are-ethereum-full-nodes-really-full-an-experiment-b77acd086ca7

Helpful/Critical tweets

  • Root Tweet #1:

    • Branch 1a: Afri, full chain is 137GB… that’s it!
    • Branch 1b: Transaction hashing comparison to block PoW hashing is all you need
    • Branch 1c: Same as above in 1b
    • Branch 1d: Whats an archival node good for anyway?

  • Root Tweet #2:

    • Branch 1a: (Good!) comparison of nodes & state to git repository and git diffs
    • Branch 1b: A comment about state size and storage rent (I completely missed this point on first passes. Only got it after ‘storage rent’ article
    • Branch 2a: Afri shares some articles

  • Root Tweet #3: Afri worries about growing influence of Infura

  • Root Tweet #4: a commentary on how simple it is to sync an Ethereum state-pruned node

  • Root Branch #5: a critical take to potentially respond to

  • Root Branch #6: A complaint about poor pruning documentation that I can potentially respond to

  • Root Branch #7: Respond to Brad

  • Root Tweet #8: why does the blockchain size keep fluctuating?

  • Root Tweet #9: another why does the blockchain size keep fluctuating?

Tweets from latest BlockCypher thing:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment