Skip to content

Instantly share code, notes, and snippets.

@veox
Last active February 13, 2019 14:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save veox/22f44209bc6e5d8f1d486869e4ad9918 to your computer and use it in GitHub Desktop.
Save veox/22f44209bc6e5d8f1d486869e4ad9918 to your computer and use it in GitHub Desktop.
DEBUG 02-12 23:29:19 HeaderMeatSyncer Requesting 192 headers from ETHPeer <Node(0xebdb@35.163.186.11)>
DEBUG 02-12 23:29:19 HeaderMeatSyncer Requesting 192 headers from ETHPeer <Node(0xcb4a@139.162.39.223)>
DEBUG 02-12 23:29:19 FastChainBodySyncer Got receipts for 44/256 headers from ETHPeer <Node(0xcb4a@139.162.39.223)>, with 0 trivial headers
DEBUG 02-12 23:29:19 FastChainBodySyncer Got receipts for 40/254 headers from ETHPeer <Node(0x6be8@18.188.209.191)>, with 2 trivial headers
DEBUG 02-12 23:29:19 FastChainBodySyncer Requesting receipts for 256 headers from ETHPeer <Node(0xcb4a@139.162.39.223)>
DEBUG 02-12 23:29:19 FastChainBodySyncer Requesting receipts for 256 headers from ETHPeer <Node(0x6be8@18.188.209.191)>
DEBUG 02-12 23:29:19 HeaderMeatSyncer Header Skeleton Gaps: active=3 queued=50 max=50
DEBUG 02-12 23:29:19 FastChainBodySyncer (in progress, queued, max size) of bodies, receipts: [(0, 0, 1024), (1024, 2019, 2048)]
INFO 02-12 23:29:19 FastChainBodySyncer blks=0 txs=51 bps=8 tps=750 elapsed=5.1 head=#6218964 050b..0688 age=5m2w3d
DEBUG 02-12 23:29:19 SkeletonSyncer Skeleton header queue is 711/1536 full
DEBUG 02-12 23:29:24 DiscoveryProtocol Ignoring find_node request from unknown node <Node(0xafbf@39.105.154.225)>
DEBUG 02-12 23:29:26 FastChaiBodySyncer <coroutine object FastChainBodySyncer._launch_prerequisite_tasks at 0x7f1afca777c8> finished while <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f1afc6ae470> is still running, terminating as well
DEBUG 02-12 23:29:26 FastChainBodySyncer Cancelling <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f1afc6ae470>
WARNING 02-12 23:29:26 FastChainBodySyncer Task <coroutine object FastChainBodySyncer._launch_prerequisite_tasks at 0x7f1afca6a448> finished unexpectedly: must not re-add same node twice: b'SM\x9bnx\x1c"X~^T\xfbQl\x15<\xca.\xa9\x14\xa0\xde\xe5\xf8\x04\x93r\xa1\xc3\xf40\x86'
DEBUG 02-12 23:29:26 FastChainBodySyncer Task failure traceback
Traceback (most recent call last):
File "/home/veox/src/trinity/p2p/service.py", line 139, in _run_task_wrapper
await awaitable
File "/home/veox/src/trinity/p2p/service.py", line 158, in _run_daemon_task_wrapper
await awaitable
File "/home/veox/src/trinity/trinity/sync/full/chain.py", line 454, in _launch_prerequisite_tasks
self._block_persist_tracker.register_tasks(headers, ignore_duplicates=True)
File "/home/veox/src/trinity/trinity/_utils/datastructures.py", line 551, in register_tasks
self._roots.add(task_id, dependency_id)
File "/home/veox/src/trinity/trinity/_utils/tree_root.py", line 229, in add
self._tree.add(node_id, parent_id)
File "/home/veox/src/trinity/trinity/_utils/tree_root.py", line 41, in add
raise ValidationError(f"must not re-add same node twice: {node_id}")
eth_utils.exceptions.ValidationError: must not re-add same node twice: b'SM\x9bnx\x1c"X~^T\xfbQl\x15<\xca.\xa9\x14\xa0\xde\xe5\xf8\x04\x93r\xa1\xc3\xf40\x86'
DEBUG 02-12 23:29:26 FastChainBodySyncer <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f1afc6ae470> cancelled, cleaning up...
DEBUG 02-12 23:29:26 FastChainBodySyncer Waiting for tasks: [<Task pending coro=<FastChainBodySyncer._assign_receipt_download_to_peers() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f1a581b8a38>()]>>, <Task pending coro=<FastChainBodySyncer._display_stats() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f189cb31bb8>()]>>, <Task pending coro=<WaitingPeers.put_nowait() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f1a985098b8>()]>>, <Task pending coro=<BaseBodyChainSyncer._assign_body_download_to_peers() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f18b0b6e618>()]>>, <Task pending coro=<FastChainBodySyncer._persist_ready_blocks() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f187e697438>()]>>]
DEBUG 02-12 23:29:26 HeaderMeatSyncer Timed out requesting 192 headers from ETHPeer <Node(0xcb4a@139.162.39.223)>
DEBUG 02-12 23:29:26 HeaderMeatSyncer Timed out requesting 192 headers from ETHPeer <Node(0xebdb@35.163.186.11)>
DEBUG 02-12 23:29:26 HeaderMeatSyncer Pausing ETHPeer <Node(0xcb4a@139.162.39.223)> for 15.0s, for sending 0 headers
DEBUG 02-12 23:29:26 HeaderMeatSyncer Pausing ETHPeer <Node(0xebdb@35.163.186.11)> for 15.0s, for sending 0 headers
DEBUG 02-12 23:29:26 ResponseCandidateStream Got unexpected BlockHeaders payload from ETHPeer <Node(0xebdb@35.163.186.11)>
DEBUG 02-12 23:29:26 FullServer Receiving handshake from Address(206.221.189.74:udp:49251|tcp:49251)
... soon shutdown commences
DEBUG 02-12 22:51:28 HeaderMeatSyncer Pausing ETHPeer <Node(0xa294@45.55.62.36)> for 15.0s, for sending 0 headers
DEBUG 02-12 22:51:28 SkeletonSyncer Skeleton header queue is 414/1536 full
DEBUG 02-12 22:51:28 HeaderMeatSyncer Header Skeleton Gaps: active=4 queued=50 max=50
DEBUG 02-12 22:51:28 FastChainBodySyncer (in progress, queued, max size) of bodies, receipts: [(0, 0, 1024), (1024, 2009, 2048)]
INFO 02-12 22:51:28 FastChainBodySyncer blks=0 txs=0 bps=4 tps=440 elapsed=7.2 head=#6215711 0d20..cd0b age=5m2w4d
DEBUG 02-12 22:51:28 FullServer Receiving handshake from Address(181.119.161.34:udp:60154|tcp:60154)
DEBUG 02-12 22:51:29 FullServer Receiving handshake from Address(89.40.10.150:udp:36610|tcp:36610)
DEBUG 02-12 22:51:29 FullServer Receiving handshake from Address(209.250.255.80:udp:55530|tcp:55530)
DEBUG 02-12 22:51:29 FullServer Receiving handshake from Address(40.83.72.16:udp:56390|tcp:56390)
DEBUG 02-12 22:51:29 FastChainBodySyncer <coroutine object FastChainBodySyncer._launch_prerequisite_tasks at 0x7f9c9c31d7c8> finished while <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f9c96f51240> is still running, terminating as well
DEBUG 02-12 22:51:29 FastChainBodySyncer Cancelling <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f9c96f51240>
WARNING 02-12 22:51:29 FastChainBodySyncer Task <coroutine object FastChainBodySyncer._launch_prerequisite_tasks at 0x7f9c9c310448> finished unexpectedly: No header with hash 0x847ef96126a54290f11db9e6c302d1e85e77dd4705af3488530e1bb663dc10ce found
DEBUG 02-12 22:51:29 FastChainBodySyncer Task failure traceback
trinity._utils.mp.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/veox/src/trinity/.virtualenv/trinity/lib/python3.7/site-packages/eth/db/header.py", line 167, in _get_block_header_by_hash
header_rlp = db[block_hash]
File "/home/veox/src/trinity/.virtualenv/trinity/lib/python3.7/site-packages/eth/db/backends/level.py", line 54, in __getitem__
raise KeyError(key)
KeyError: b'\x84~\xf9a&\xa5B\x90\xf1\x1d\xb9\xe6\xc3\x02\xd1\xe8^w\xddG\x05\xaf4\x88S\x0e\x1b\xb6c\xdc\x10\xce'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/veox/src/trinity/trinity/_utils/mp.py", line 68, in wrapper
return attr(*args, **kwargs)
File "/home/veox/src/trinity/.virtualenv/trinity/lib/python3.7/site-packages/eth/db/header.py", line 156, in get_block_header_by_hash
return self._get_block_header_by_hash(self.db, block_hash)
File "/home/veox/src/trinity/.virtualenv/trinity/lib/python3.7/site-packages/eth/db/header.py", line 170, in _get_block_header_by_hash
encode_hex(block_hash)))
eth.exceptions.HeaderNotFound: No header with hash 0x847ef96126a54290f11db9e6c302d1e85e77dd4705af3488530e1bb663dc10ce found
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/veox/src/trinity/p2p/service.py", line 139, in _run_task_wrapper
await awaitable
File "/home/veox/src/trinity/p2p/service.py", line 158, in _run_daemon_task_wrapper
await awaitable
File "/home/veox/src/trinity/trinity/sync/full/chain.py", line 465, in _launch_prerequisite_tasks
self.db.coro_get_block_header_by_hash(headers[0].parent_hash)
File "/home/veox/src/trinity/p2p/cancellable.py", line 20, in wait
return await self.wait_first(awaitable, token=token, timeout=timeout)
File "/home/veox/src/trinity/p2p/cancellable.py", line 42, in wait_first
return await token_chain.cancellable_wait(*awaitables, timeout=timeout)
File "/home/veox/src/trinity/.virtualenv/trinity/lib/python3.7/site-packages/cancel_token/token.py", line 152, in cancellable_wait
return done.pop().result()
File "/home/veox/src/trinity/trinity/_utils/mp.py", line 33, in method
args,
File "/usr/lib64/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib64/python3.7/multiprocessing/managers.py", line 811, in _callmethod
raise convert_to_error(kind, result)
eth.exceptions.HeaderNotFound: No header with hash 0x847ef96126a54290f11db9e6c302d1e85e77dd4705af3488530e1bb663dc10ce found
DEBUG 02-12 22:51:29 FastChainBodySyncer <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f9c96f51240> cancelled, cleaning up...
DEBUG 02-12 22:51:29 FastChainBodySyncer Waiting for tasks: [<Task pending coro=<FastChainBodySyncer._assign_receipt_download_to_peers() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f9a059c52b8>()]>>, <Task pending coro=<FastChainBodySyncer._display_stats() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f9a14c0c168>()]>>, <Task pending coro=<BaseBodyChainSyncer._assign_body_download_to_peers() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f9a0d987228>()]>>, <Task pending coro=<FastChainBodySyncer._persist_ready_blocks() running at /home/veox/src/trinity/p2p/service.py:139> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f9b8074f7c8>()]>>]
DEBUG 02-12 22:51:29 HeaderMeatSyncer Requesting 192 headers from ETHPeer <Node(0xaec2@47.75.76.23)>
DEBUG 02-12 22:51:29 HeaderMeatSyncer Requesting 192 headers from ETHPeer <Node(0xec25@81.171.3.78)>
DEBUG 02-12 22:51:29 FastChainBodySyncer All tasks finished
DEBUG 02-12 22:51:29 FastChainBodySyncer <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f9c96f51240> halted cleanly
DEBUG 02-12 22:51:29 ETHHeaderChainSyncer Cancelling <trinity.protocol.eth.sync.ETHHeaderChainSyncer object at 0x7f9c9c306208>
DEBUG 02-12 22:51:29 FastChainSyncer <trinity.sync.full.chain.FastChainSyncer object at 0x7f9c9c320eb8> had nothing left to do, ceasing operation...
DEBUG 02-12 22:51:29 FastChainSyncer Waiting for child services: [<trinity.protocol.eth.sync.ETHHeaderChainSyncer object at 0x7f9c9c306208>]
INFO 02-12 22:51:29 ETHHeaderChainSyncer Skeleton sync with ETHPeer <Node(0xcbce@35.174.161.237)> ended
DEBUG 02-12 22:51:29 ResponseCandidateStream Got unexpected BlockHeaders payload from ETHPeer <Node(0xa294@45.55.62.36)>
DEBUG 02-12 22:51:29 ETHPeer Finished P2P handshake with <Node(0x9d45@89.40.10.150)>, using sub-protocol (eth, 63)
DEBUG 02-12 22:51:29 ETHHeaderChainSyncer <trinity.protocol.eth.sync.ETHHeaderChainSyncer object at 0x7f9c9c306208> finished: Cancellation requested by ETHHeaderChainSyncer:FastChainSyncer:FastThenFullChainSyncer:FullNode token
... soon shutdown completes
DEBUG 02-13 15:35:01 SkeletonSyncer Requsting chain of headers from ETHPeer <Node(0xb529@18.214.179.121)> starting at #6268029
DEBUG 02-13 15:35:01 ETHHeaderChainSyncer Skeleton syncer asserts that parent (a725..e968) of the first header (<BlockHeader #6267701 e8ea2267>) is already present
INFO 02-13 15:35:01 ETHHeaderChainSyncer Skeleton sync with ETHPeer <Node(0xb529@18.214.179.121)> ended
ERROR 02-13 15:35:01 ETHHeaderChainSyncer Unexpected error in <trinity.protocol.eth.sync.ETHHeaderChainSyncer object at 0x7f3ac1e242b0>, exiting
Traceback (most recent call last):
File "/home/veox/src/trinity/p2p/service.py", line 101, in run
await self._run()
File "/home/veox/src/trinity/trinity/sync/common/headers.py", line 756, in _run
await self.wait(self._build_skeleton())
File "/home/veox/src/trinity/p2p/cancellable.py", line 20, in wait
return await self.wait_first(awaitable, token=token, timeout=timeout)
File "/home/veox/src/trinity/p2p/cancellable.py", line 42, in wait_first
return await token_chain.cancellable_wait(*awaitables, timeout=timeout)
File "/home/veox/src/trinity/.virtualenv/trinity/lib/python3.7/site-packages/cancel_token/token.py", line 152, in cancellable_wait
return done.pop().result()
File "/home/veox/src/trinity/trinity/sync/common/headers.py", line 770, in _build_skeleton
await self._full_skeleton_sync(syncer)
File "/home/veox/src/trinity/trinity/sync/common/headers.py", line 825, in _full_skeleton_sync
self._stitcher.register_tasks(first_segment, ignore_duplicates=True)
File "/home/veox/src/trinity/trinity/_utils/datastructures.py", line 555, in register_tasks
self._mark_complete(task_id)
File "/home/veox/src/trinity/trinity/_utils/datastructures.py", line 612, in _mark_complete
self._prune_finished(prune_from_leaf)
File "/home/veox/src/trinity/trinity/_utils/datastructures.py", line 641, in _prune_finished
root_task_id, depth = self._roots.get_root(task_id)
File "/home/veox/src/trinity/trinity/_utils/tree_root.py", line 261, in get_root
f"{root_node_id} has parent {parent}, but was going to be returned as a root"
eth_utils.exceptions.ValidationError: b'\x16R\x1e\xe4UK2>\xc2%\xbe\x13yH\x18\x0b\xe8\xeb\xa4\xaf\xd4V\x86\xae\xf5 t\t\x1e\xce\xc7\xdf' has parent b'\x991\xe9\xf9\x07\x85\xefc\x14\xdc\xeb\xdb\xbf\x8d\x1d\x9e\xf7\x03\x12b>\xff0\x7f\x08\x07\xe7\xa2\xf9+\x0f\x1a', but was going to be returned as a root
DEBUG 02-13 15:35:01 ETHHeaderChainSyncer Waiting for child services: [<trinity.sync.common.headers.HeaderMeatSyncer object at 0x7f3ac1e24c50>, <trinity.sync.common.headers.SkeletonSyncer object at 0x7f39a02936a0>, <trinity.protocol.eth.monitors.ETHChainTipMonitor object at 0x7f3ac1e245c0>]
DEBUG 02-13 15:35:01 FastChainBodySyncer <coroutine object FastChainBodySyncer._launch_prerequisite_tasks at 0x7f3ac1e3b7c8> finished while <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f3ac1a722e8> is still running, terminating as well
DEBUG 02-13 15:35:01 FastChainBodySyncer Cancelling <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f3ac1a722e8>
DEBUG 02-13 15:35:01 FastChainBodySyncer <trinity.sync.full.chain.FastChainBodySyncer object at 0x7f3ac1a722e8> cancelled, cleaning up...
... soon shutdown completes
alabaster==0.7.12
apipkg==1.5
argh==0.26.2
asn1crypto==0.24.0
async-generator==1.10
async-lru==0.1.0
asyncio-cancel-token==0.1.0a2
atomicwrites==1.2.1
attrdict==2.0.0
attrs==18.2.0
Babel==2.6.0
backcall==0.1.0
bleach==3.1.0
bloom-filter==1.3
bumpversion==0.5.3
cachetools==2.1.0
certifi==2018.11.29
cffi==1.11.5
chardet==3.0.4
coincurve==10.0.0
colorama==0.4.1
coverage==4.5.2
cryptography==2.4.2
cytoolz==0.9.0.1
decorator==4.3.0
docopt==0.6.2
docutils==0.14
eth-abi==1.3.0
eth-account==0.2.3
eth-bloom==1.0.3
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.1
eth-rlp==0.1.2
eth-typing==2.0.0
eth-utils==1.4.1
execnet==1.5.0
flake8==3.5.0
hexbytes==0.1.0
hypothesis==3.69.5
idna==2.7
ifaddr==0.1.6
imagesize==1.1.0
ipython==6.5.0
ipython-genutils==0.2.0
jedi==0.13.2
Jinja2==2.10
jsonschema==2.6.0
lahja==0.10.2
lru-dict==1.1.6
lxml==4.3.0
MarkupSafe==1.1.0
mccabe==0.6.1
more-itertools==5.0.0
mypy==0.641
mypy-extensions==0.4.1
netdisco==2.3.0
netifaces==0.10.9
packaging==18.0
parsimonious==0.8.1
parso==0.3.1
pathtools==0.1.2
pexpect==4.6.0
pickleshare==0.7.5
pkginfo==1.5.0.1
pluggy==0.7.1
plyvel==1.0.5
prompt-toolkit==1.0.15
ptyprocess==0.6.0
py==1.7.0
py-ecc==1.4.7
py-evm==0.2.0a40
pycodestyle==2.3.1
pycparser==2.19
pycryptodome==3.7.2
pyethash==0.1.27
pyflakes==1.6.0
Pygments==2.3.1
pyparsing==2.3.1
pysha3==1.0.2
pytest==3.6.4
pytest-asyncio==0.9.0
pytest-asyncio-network-simulator==0.1.0a2
pytest-cov==2.5.1
pytest-watch==4.2.0
pytest-xdist==1.18.1
python-dateutil==2.7.5
python-rocksdb==0.6.9
python-snappy==0.5.3
pytz==2018.9
PyYAML==3.13
readme-renderer==24.0
requests==2.21.0
requests-toolbelt==0.8.0
rlp==1.1.0
simplegeneric==0.8.1
six==1.12.0
snowballstemmer==1.2.1
Sphinx==1.7.9
sphinx-rtd-theme==0.4.2
sphinxcontrib-asyncio==0.2.0
sphinxcontrib-websupport==1.1.0
termcolor==1.1.0
toolz==0.9.0
tox==2.7.0
tqdm==4.29.1
traitlets==4.3.2
trie==1.3.8
-e git+git@github-veox:veox/trinity.git@c01424d0cb127af943df290a8b4d8853bd76b891#egg=trinity
twine==1.12.1
typed-ast==1.1.1
uPnPClient==0.0.8
urllib3==1.24.1
uvloop==0.11.2
virtualenv==16.2.0
watchdog==0.9.0
wcwidth==0.1.7
web3==4.4.1
webencodings==0.5.1
websockets==5.0.1
zeroconf==0.21.3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment