Skip to content

Instantly share code, notes, and snippets.

@gsalgado

gsalgado/gist:d867a03645b9800ee2499e4247534aa9

Last active January 23, 2020 15:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gsalgado/d867a03645b9800ee2499e4247534aa9 to your computer and use it in GitHub Desktop.
Save gsalgado/d867a03645b9800ee2499e4247534aa9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
(trinity-asyncio) salgado@t495~/src/trinity (master) $ pytest -n 4 tests/core/p2p-proto/
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.7.5, pytest-5.3.1, py-1.8.0, pluggy-0.13.1
Using --randomly-seed=1579793872
rootdir: /home/salgado/src/trinity, inifile: pytest.ini
plugins: xdist-1.29.0, cov-2.8.1, web3-5.4.0, mock-1.12.1, hypothesis-4.47.4, forked-1.1.3, asyncio-0.10.0, aiohttp-0.3.0, randomly-3.1.0
gw0 [202] / gw1 [202] / gw2 [202] / gw3 [202]
.........................................................................................................................................................................................................Beam Sync: retrying state data request after timeout. Stats so far: BeamStat: accts=0, a_nodes=0, codes=0, strg=0, s_nodes=0, nodes=0, rtt=0.000s, wait=20s
exception calling callback for <Future at 0x7fbe4c254b50 state=finished raised RuntimeError>
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/batch.py", line 69, in __getitem__
value = self._track_diff[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/diff.py", line 80, in __getitem__
raise DiffMissingError(key, result) # type: ignore # ignore over cast for perf reasons
eth.db.diff.DiffMissingError: (b'\xb4K\x86YjcSX\xe7\xaa`\xb1}2\x86\x0c?\x1e\xfe->S\xfb\x82\xc0\xbb#!;\x9cK\xe3', 'Key is missing because it was never inserted')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 75, in get
return self._get(root_node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 87, in _get
return self._get_branch_node(node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 491, in _get_branch_node
return self._get(sub_node, trie_key[1:])
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 87, in _get
return self._get_branch_node(node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 490, in _get_branch_node
sub_node = self.get_node(node[trie_key[0]])
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 284, in get_node
encoded_node = self.db[node_hash]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/atomic.py", line 37, in __getitem__
return self.wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/backends/memory.py", line 21, in __getitem__
return self.kv_store[key]
KeyError: b'O\x01}\x94\x1a\xfb\xb2\x13\xec\xd1\x84\xe4\x9a\xa1p<f\xf4\xa4\xf0\x05\xbc\xfe\x84\x81\x18\xc1\xd0W\xbavj'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/storage.py", line 88, in __getitem__
return read_trie[hashed_slot]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 515, in __getitem__
return self.get(key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 77, in get
self._raise_missing_node(exc, key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 93, in _raise_missing_node
raise MissingTrieNode(exception.args[0], self.root_hash, key) from exception
trie.exceptions.MissingTrieNode: Trie database is missing hash 0x4f017d941afbb213ecd184e49aa1703c66f4a4f005bcfe848118c1d057ba766a needed to look up key 0xf29d08f797fffc3e57050ea39c8dfc6add0111e41e3f838a31d6c381c6505697 at root hash 0xf3f2a12738a71269c1bc01d29c6b75ea838a7da62f78e2bd5da3c86089f4b2e0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 255, in _request_missing_data
return vm_method(*args, **kwargs) # type: ignore
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/state.py", line 107, in get_storage
return self._account_db.get_storage(address, slot, from_journal)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/account.py", line 146, in get_storage
return account_store.get(slot, from_journal)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/storage.py", line 197, in get
encoded_value = lookup_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/journal.py", line 336, in __getitem__
return self._wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/batch.py", line 74, in __getitem__
return self.wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/cache.py", line 22, in __getitem__
self._cached_values[key] = self._db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/storage.py", line 95, in __getitem__
) from exc
eth.vm.interrupt.MissingStorageTrieNode: Storage trie database is missing hash 0x4f017d941afbb213ecd184e49aa1703c66f4a4f005bcfe848118c1d057ba766a needed to look up key 0xf29d08f797fffc3e57050ea39c8dfc6add0111e41e3f838a31d6c381c6505697 at root hash 0xf3f2a12738a71269c1bc01d29c6b75ea838a7da62f78e2bd5da3c86089f4b2e0 in account address 0x49361e4f811f49542f19d691cf5f79d39983e8e0 -- (b'O\x01}\x94\x1a\xfb\xb2\x13\xec\xd1\x84\xe4\x9a\xa1p<f\xf4\xa4\xf0\x05\xbc\xfe\x84\x81\x18\xc1\xd0W\xbavj', b"\xf3\xf2\xa1'8\xa7\x12i\xc1\xbc\x01\xd2\x9cku\xea\x83\x8a}\xa6/x\xe2\xbd]\xa3\xc8`\x89\xf4\xb2\xe0", b'\xf2\x9d\x08\xf7\x97\xff\xfc>W\x05\x0e\xa3\x9c\x8d\xfcj\xdd\x01\x11\xe4\x1e?\x83\x8a1\xd6\xc3\x81\xc6PV\x97', b'I6\x1eO\x81\x1fIT/\x19\xd6\x91\xcf_y\xd3\x99\x83\xe8\xe0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 390, in _import_block
reorg_info = beam_chain.import_block(block, perform_validation=True)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/chains/base.py", line 469, in import_block
imported_block = self.get_vm(base_header_for_import).import_block(block)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 293, in import_block
new_header, receipts, _ = self.apply_all_transactions(block.transactions, header)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 245, in apply_all_transactions
transaction,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 164, in apply_transaction
computation = self.state.apply_transaction(transaction)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/state.py", line 193, in apply_transaction
return executor(transaction)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/state.py", line 267, in __call__
computation = self.build_computation(message, transaction)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/state.py", line 133, in build_computation
transaction_context).apply_message()
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/computation.py", line 77, in apply_message
self.transaction_context,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 554, in apply_computation
opcode_fn(computation=computation)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/logic/call.py", line 135, in __call__
child_computation = computation.apply_child_computation(child_msg)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 366, in apply_child_computation
child_computation = self.generate_child_computation(child_msg)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 381, in generate_child_computation
self.transaction_context,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/computation.py", line 77, in apply_message
self.transaction_context,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 554, in apply_computation
opcode_fn(computation=computation)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/opcode.py", line 55, in wrapped_logic_fn
return logic_fn(computation)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/logic/storage.py", line 60, in sload
slot=slot,
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 304, in get_storage
return self._pause_on_missing_data(super().get_storage, *args, **kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 231, in _pause_on_missing_data
return self._request_missing_data(vm_method, *args, **kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 290, in _request_missing_data
loop,
File "/usr/lib/python3.7/asyncio/tasks.py", line 846, in run_coroutine_threadsafe
loop.call_soon_threadsafe(callback)
File "uvloop/loop.pyx", line 1265, in uvloop.loop.Loop.call_soon_threadsafe
File "uvloop/loop.pyx", line 635, in uvloop.loop.Loop._call_soon
File "uvloop/loop.pyx", line 639, in uvloop.loop.Loop._call_soon_handle
File "uvloop/loop.pyx", line 668, in uvloop.loop.Loop._check_closed
RuntimeError: Event loop is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/_base.py", line 324, in _invoke_callbacks
callback(self)
File "/usr/lib/python3.7/asyncio/futures.py", line 362, in _call_set_state
dest_loop.call_soon_threadsafe(_set_state, destination, source)
File "uvloop/loop.pyx", line 1265, in uvloop.loop.Loop.call_soon_threadsafe
File "uvloop/loop.pyx", line 635, in uvloop.loop.Loop._call_soon
File "uvloop/loop.pyx", line 639, in uvloop.loop.Loop._call_soon_handle
File "uvloop/loop.pyx", line 668, in uvloop.loop.Loop._check_closed
RuntimeError: Event loop is closed
. [100%]sys:1: RuntimeWarning: coroutine 'pausing_vm_decorator.<locals>.request_missing_storage' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Beam Sync: retrying state data request after timeout. Stats so far: BeamStat: accts=0, a_nodes=0, codes=0, strg=0, s_nodes=0, nodes=0, rtt=0.000s, wait=20s
exception calling callback for <Future at 0x7fc95c0294d0 state=finished raised RuntimeError>
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/batch.py", line 69, in __getitem__
value = self._track_diff[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/diff.py", line 80, in __getitem__
raise DiffMissingError(key, result) # type: ignore # ignore over cast for perf reasons
eth.db.diff.DiffMissingError: (b'\xed=\x8e\xe2\x8c_@\x17\x9e\x0bS\xbf\x87\xb4\x86\xc0\xb1\xa6\xcc\xfb6\xf0;<v,r7\x93\xe0f1', 'Key is missing because it was never inserted')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 75, in get
return self._get(root_node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 87, in _get
return self._get_branch_node(node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 490, in _get_branch_node
sub_node = self.get_node(node[trie_key[0]])
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 284, in get_node
encoded_node = self.db[node_hash]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/atomic.py", line 37, in __getitem__
return self.wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/backends/memory.py", line 21, in __getitem__
return self.kv_store[key]
KeyError: b'\x90!\xd2\x86\x892?\xde\x9d\x15_k\xe9\x1eV-{\xf1\xb4P7\xdd\xe9\xd0ii\x97a\t9\x19\xa7'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/storage.py", line 88, in __getitem__
return read_trie[hashed_slot]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 515, in __getitem__
return self.get(key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 77, in get
self._raise_missing_node(exc, key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 93, in _raise_missing_node
raise MissingTrieNode(exception.args[0], self.root_hash, key) from exception
trie.exceptions.MissingTrieNode: Trie database is missing hash 0x9021d28689323fde9d155f6be91e562d7bf1b45037dde9d069699761093919a7 needed to look up key 0x56aa8ebeb3c941680956d2acd0c9b146a68438e165b9316cb7234168de360045 at root hash 0x3bbe41856e6d377e6f2308cc42e5b6b16e2ee7e0fabfac37839717ef9d647376
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 255, in _request_missing_data
return vm_method(*args, **kwargs) # type: ignore
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/state.py", line 107, in get_storage
return self._account_db.get_storage(address, slot, from_journal)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/account.py", line 146, in get_storage
return account_store.get(slot, from_journal)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/storage.py", line 197, in get
encoded_value = lookup_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/journal.py", line 336, in __getitem__
return self._wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/batch.py", line 74, in __getitem__
return self.wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/cache.py", line 22, in __getitem__
self._cached_values[key] = self._db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/storage.py", line 95, in __getitem__
) from exc
eth.vm.interrupt.MissingStorageTrieNode: Storage trie database is missing hash 0x9021d28689323fde9d155f6be91e562d7bf1b45037dde9d069699761093919a7 needed to look up key 0x56aa8ebeb3c941680956d2acd0c9b146a68438e165b9316cb7234168de360045 at root hash 0x3bbe41856e6d377e6f2308cc42e5b6b16e2ee7e0fabfac37839717ef9d647376 in account address 0x49361e4f811f49542f19d691cf5f79d39983e8e0 -- (b'\x90!\xd2\x86\x892?\xde\x9d\x15_k\xe9\x1eV-{\xf1\xb4P7\xdd\xe9\xd0ii\x97a\t9\x19\xa7', b';\xbeA\x85nm7~o#\x08\xccB\xe5\xb6\xb1n.\xe7\xe0\xfa\xbf\xac7\x83\x97\x17\xef\x9ddsv', b'V\xaa\x8e\xbe\xb3\xc9Ah\tV\xd2\xac\xd0\xc9\xb1F\xa6\x848\xe1e\xb91l\xb7#Ah\xde6\x00E', b'I6\x1eO\x81\x1fIT/\x19\xd6\x91\xcf_y\xd3\x99\x83\xe8\xe0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 390, in _import_block
reorg_info = beam_chain.import_block(block, perform_validation=True)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/chains/base.py", line 469, in import_block
imported_block = self.get_vm(base_header_for_import).import_block(block)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 293, in import_block
new_header, receipts, _ = self.apply_all_transactions(block.transactions, header)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 245, in apply_all_transactions
transaction,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 164, in apply_transaction
computation = self.state.apply_transaction(transaction)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/state.py", line 193, in apply_transaction
return executor(transaction)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/state.py", line 267, in __call__
computation = self.build_computation(message, transaction)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/state.py", line 133, in build_computation
transaction_context).apply_message()
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/computation.py", line 77, in apply_message
self.transaction_context,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 554, in apply_computation
opcode_fn(computation=computation)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/logic/call.py", line 135, in __call__
child_computation = computation.apply_child_computation(child_msg)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 366, in apply_child_computation
child_computation = self.generate_child_computation(child_msg)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 381, in generate_child_computation
self.transaction_context,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/forks/frontier/computation.py", line 77, in apply_message
self.transaction_context,
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/computation.py", line 554, in apply_computation
opcode_fn(computation=computation)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/opcode.py", line 55, in wrapped_logic_fn
return logic_fn(computation)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/logic/storage.py", line 60, in sload
slot=slot,
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 304, in get_storage
return self._pause_on_missing_data(super().get_storage, *args, **kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 231, in _pause_on_missing_data
return self._request_missing_data(vm_method, *args, **kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 290, in _request_missing_data
loop,
File "/usr/lib/python3.7/asyncio/tasks.py", line 846, in run_coroutine_threadsafe
loop.call_soon_threadsafe(callback)
File "uvloop/loop.pyx", line 1265, in uvloop.loop.Loop.call_soon_threadsafe
File "uvloop/loop.pyx", line 635, in uvloop.loop.Loop._call_soon
File "uvloop/loop.pyx", line 639, in uvloop.loop.Loop._call_soon_handle
File "uvloop/loop.pyx", line 668, in uvloop.loop.Loop._check_closed
RuntimeError: Event loop is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/_base.py", line 324, in _invoke_callbacks
callback(self)
File "/usr/lib/python3.7/asyncio/futures.py", line 362, in _call_set_state
dest_loop.call_soon_threadsafe(_set_state, destination, source)
File "uvloop/loop.pyx", line 1265, in uvloop.loop.Loop.call_soon_threadsafe
File "uvloop/loop.pyx", line 635, in uvloop.loop.Loop._call_soon
File "uvloop/loop.pyx", line 639, in uvloop.loop.Loop._call_soon_handle
File "uvloop/loop.pyx", line 668, in uvloop.loop.Loop._check_closed
RuntimeError: Event loop is closed
sys:1: RuntimeWarning: coroutine 'pausing_vm_decorator.<locals>.request_missing_storage' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Beam Sync: retrying state data request after timeout. Stats so far: BeamStat: accts=0, a_nodes=0, codes=0, strg=2, s_nodes=2, nodes=2, rtt=10.015s, wait=20s
exception calling callback for <Future at 0x7fa66c383f50 state=finished raised RuntimeError>
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/batch.py", line 69, in __getitem__
value = self._track_diff[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/diff.py", line 80, in __getitem__
raise DiffMissingError(key, result) # type: ignore # ignore over cast for perf reasons
eth.db.diff.DiffMissingError: (b'_\xfb\x88J\xf6\x9b,\xe7\xf0\x85\x80\xafB\x19Y\xae\xaa\xcc\x94&K\xfa\xe9\x9e\xa7#X\x85\x07\x04\x9b\xc9', 'Key is missing because it was never inserted')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 142, in delete
new_node = self._delete(root_node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 158, in _delete
return self._delete_branch_node(node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 371, in _delete_branch_node
sub_node = self._delete(node_to_delete, trie_key[1:])
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 158, in _delete
return self._delete_branch_node(node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 371, in _delete_branch_node
sub_node = self._delete(node_to_delete, trie_key[1:])
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 158, in _delete
return self._delete_branch_node(node, trie_key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 379, in _delete_branch_node
return self._normalize_branch_node(node)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 342, in _normalize_branch_node
sub_node = self.get_node(sub_node_hash)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 284, in get_node
encoded_node = self.db[node_hash]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/utils/db.py", line 33, in __getitem__
return self.wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/batch.py", line 74, in __getitem__
return self.wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/atomic.py", line 37, in __getitem__
return self.wrapped_db[key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/backends/memory.py", line 21, in __getitem__
return self.kv_store[key]
KeyError: b'_\xfb\x88J\xf6\x9b,\xe7\xf0\x85\x80\xafB\x19Y\xae\xaa\xcc\x94&K\xfa\xe9\x9e\xa7#X\x85\x07\x04\x9b\xc9'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/account.py", line 477, in _apply_account_diff_without_proof
del trie[delete_key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/keymap.py", line 36, in __delitem__
del self._db[mapped_key]
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 521, in __delitem__
return self.delete(key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 144, in delete
self._raise_missing_node(exc, key)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/trie/hexary.py", line 93, in _raise_missing_node
raise MissingTrieNode(exception.args[0], self.root_hash, key) from exception
trie.exceptions.MissingTrieNode: Trie database is missing hash 0x5ffb884af69b2ce7f08580af421959aeaacc94264bfae99ea723588507049bc9 needed to look up key 0x9bea16a2ee7364a038aa6c3b79eb1cabe5e08e1943c6bb8a73950ef54090762b at root hash 0xb90466131e67e9a5afa9b1d0cb63020643211be328a4e36554a3dfaaa0c22130
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 255, in _request_missing_data
return vm_method(*args, **kwargs) # type: ignore
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/state.py", line 182, in persist
self._account_db.persist()
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/account.py", line 403, in persist
self.make_state_root()
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/account.py", line 395, in make_state_root
self._apply_account_diff_without_proof(diff, memory_trie)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/db/account.py", line 488, in _apply_account_diff_without_proof
) from exc
eth.vm.interrupt.MissingAccountTrieNode: State trie database is missing node for hash 0x5ffb884af69b2ce7f08580af421959aeaacc94264bfae99ea723588507049bc9, which is needed to look up account with address hash 0x9bea16a2ee7364a038aa6c3b79eb1cabe5e08e1943c6bb8a73950ef54090762b at root hash 0xb90466131e67e9a5afa9b1d0cb63020643211be328a4e36554a3dfaaa0c22130 -- (b'_\xfb\x88J\xf6\x9b,\xe7\xf0\x85\x80\xafB\x19Y\xae\xaa\xcc\x94&K\xfa\xe9\x9e\xa7#X\x85\x07\x04\x9b\xc9', b'\xb9\x04f\x13\x1eg\xe9\xa5\xaf\xa9\xb1\xd0\xcbc\x02\x06C!\x1b\xe3(\xa4\xe3eT\xa3\xdf\xaa\xa0\xc2!0', b'\x9b\xea\x16\xa2\xeesd\xa08\xaal;y\xeb\x1c\xab\xe5\xe0\x8e\x19C\xc6\xbb\x8as\x95\x0e\xf5@\x90v+')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 390, in _import_block
reorg_info = beam_chain.import_block(block, perform_validation=True)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/chains/base.py", line 469, in import_block
imported_block = self.get_vm(base_header_for_import).import_block(block)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 302, in import_block
return self.mine_block()
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 307, in mine_block
final_block = self.finalize_block(packed_block)
File "/home/salgado/virtualenvs/trinity-asyncio/lib/python3.7/site-packages/eth/vm/base.py", line 378, in finalize_block
self.state.persist()
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 346, in persist
return self._pause_on_missing_data(super().persist)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 231, in _pause_on_missing_data
return self._request_missing_data(vm_method, *args, **kwargs)
File "/home/salgado/src/trinity/trinity/sync/beam/importer.py", line 264, in _request_missing_data
loop,
File "/usr/lib/python3.7/asyncio/tasks.py", line 846, in run_coroutine_threadsafe
loop.call_soon_threadsafe(callback)
File "uvloop/loop.pyx", line 1265, in uvloop.loop.Loop.call_soon_threadsafe
File "uvloop/loop.pyx", line 635, in uvloop.loop.Loop._call_soon
File "uvloop/loop.pyx", line 639, in uvloop.loop.Loop._call_soon_handle
File "uvloop/loop.pyx", line 668, in uvloop.loop.Loop._check_closed
RuntimeError: Event loop is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/_base.py", line 324, in _invoke_callbacks
callback(self)
File "/usr/lib/python3.7/asyncio/futures.py", line 362, in _call_set_state
dest_loop.call_soon_threadsafe(_set_state, destination, source)
File "uvloop/loop.pyx", line 1265, in uvloop.loop.Loop.call_soon_threadsafe
File "uvloop/loop.pyx", line 635, in uvloop.loop.Loop._call_soon
File "uvloop/loop.pyx", line 639, in uvloop.loop.Loop._call_soon_handle
File "uvloop/loop.pyx", line 668, in uvloop.loop.Loop._check_closed
RuntimeError: Event loop is closed
sys:1: RuntimeWarning: coroutine 'pausing_vm_decorator.<locals>.request_missing_account' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending coro=<pausing_vm_decorator.<locals>.request_missing_storage() running at /home/salgado/src/trinity/trinity/sync/beam/importer.py:189> wait_for=<Future pending cb=[AsyncioEndpoint._remove_canc
elled_future(b'PausingEndp...*\x00\x00\x00')(), <TaskWakeupMethWrapper object at 0x7ffb7d7c56d0>()]> cb=[_chain_future.<locals>._call_set_state() at /usr/lib/python3.7/asyncio/futures.py:355]>
Task was destroyed but it is pending!
task: <Task pending coro=<pausing_vm_decorator.<locals>.request_missing_storage() running at /home/salgado/src/trinity/trinity/sync/beam/importer.py:189> wait_for=<Future pending cb=[AsyncioEndpoint._remove_canc
elled_future(b"PausingEndp...'\x00\x00\x00")(), <TaskWakeupMethWrapper object at 0x7ffb9c096610>()]> cb=[_chain_future.<locals>._call_set_state() at /usr/lib/python3.7/asyncio/futures.py:355]>
Task was destroyed but it is pending!
task: <Task pending coro=<pausing_vm_decorator.<locals>.request_missing_storage() running at /home/salgado/src/trinity/trinity/sync/beam/importer.py:189> wait_for=<Future pending cb=[AsyncioEndpoint._remove_canc
elled_future(b'PausingEndp...)\x00\x00\x00')(), <TaskWakeupMethWrapper object at 0x7ffb9c2fe8d0>()]> cb=[_chain_future.<locals>._call_set_state() at /usr/lib/python3.7/asyncio/futures.py:355]>
Task was destroyed but it is pending!
task: <Task pending coro=<pausing_vm_decorator.<locals>.request_missing_storage() running at /home/salgado/src/trinity/trinity/sync/beam/importer.py:189> wait_for=<Future pending cb=[AsyncioEndpoint._remove_canc
elled_future(b'PausingEndp...&\x00\x00\x00')(), <TaskWakeupMethWrapper object at 0x7ffb7dc16d90>()]> cb=[_chain_future.<locals>._call_set_state() at /usr/lib/python3.7/asyncio/futures.py:355]>
Task was destroyed but it is pending!
task: <Task pending coro=<pausing_vm_decorator.<locals>.request_missing_storage() running at /home/salgado/src/trinity/trinity/sync/beam/importer.py:189> wait_for=<Future pending cb=[AsyncioEndpoint._remove_canc
elled_future(b'PausingEndp...+\x00\x00\x00')(), <TaskWakeupMethWrapper object at 0x7ffb7dcb8610>()]> cb=[_chain_future.<locals>._call_set_state() at /usr/lib/python3.7/asyncio/futures.py:355]>
sys:1: RuntimeWarning: coroutine 'pausing_vm_decorator.<locals>.request_missing_storage' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment