Testing https://github.com/bitcoin/bitcoin/pull/6650

testlog.md

Setup:

• regtest mode
• txindex enabled
• database from 0.10
• there are a few blocks and transactions

chainstate:

42 a63a29c6e9d7c03c29fd9b64b758f94f5c0ddf46479f49ff4dfd30883e84aa15
6300934e122e90e3995afffeb9154de426fd109e36de907a3214bea5abc4d26650 010206005ab93563a289b74c355a9b9258b86f12bb84affb8035
63030e372a504e2ea94d86d89e117361c9ca26181ee19670fed99878988080ae46 0103caba36029bba8ecc48f3777868ec77d927dbb3edc8003830d00972baaa0ec2eb04773dca814e
6306739e7055e4ecfec6b0b242f5790b91582f2e76380bb92ebb947ca63887840c 0103c3d40d03dcacd47d0d99bc7e43eba7e5e55d3af8da139356e88e59d43c6cbd52799ff72e8145
63071562eeda0bfa0b44f95ef8b29581fbf005c3f51c48c0a6ff055f0104c2da39 0103806503660d2f03d31344a752ad03fd71c360d4b0be9c607e2ef077ad73f4f9d06e15c88116
63071c48dce1ded71be49a06c97873544d22c1531c5f9515c9516e571dfd93d0fb 010380a6d0d5827702313f346f9973c3bb81ef399ff04b1b65c35c12e4fbba7f31797c44eac859fd1c8123
...

blocks/index:

46077478696e646578 31
62001a290f06cf1eefd8f5b97a7341b131becbafbea8dba883981e622e2965d812 86a75c741d020080cb14a92203000000c8cd46e2dd62e2b2e91021a0dc09b50bbb0bf73f1a947190a3899c1a74fdaf53a17bcba34151c852bf3874c0d7f16d9840991b235b26b5b4a1fca3adc60a0ffdc0160c56ffff7f2002000000
6200367f5fb042c0eae5f70c831f2f01886d3ee5a91ecaef41e2075a5d85d62b06 86a75c371d0100d045902e03000000dd342e25693cccad51ef1f357a03f6d2ebd77f3fb74063fd25f6ab3d76bd5625412e24a927cdfee49331a744695dc5b7c4b970af2558ae377bfbf202a0edae55b5160c56ffff7f2001000000
...

(1) When I started patched Bitcoin Core, then I saw in the logs that 0000000000000000 was used as obfuscation key for blocks/index and chainstate, and no new key entry was written to the database. Everything was as expected.

(2) When I started patched Bitcoin Core with -reindex, then the QT client hung/froze during the splash screen, and the log showed:

init message: Loading block index...
Wiping LevelDB in ~/.bitcoin/regtest/blocks/index
Opening LevelDB in ~/.bitcoin/regtest/blocks/index
Opened LevelDB successfully
Using obfuscation key for ~/.bitcoin/regtest/blocks/index: 0000000000000000
Wiping LevelDB in ~/.bitcoin/regtest/chainstate
Opening LevelDB in ~/.bitcoin/regtest/chainstate
Opened LevelDB successfully
Wrote new obfuscate key for ~/.bitcoin/regtest/chainstate: 3e3f21d3e1e586cd
Using obfuscation key for ~/.bitcoin/regtest/chainstate: 3e3f21d3e1e586cd
Initializing databases...
init message: Verifying blocks...
 block index              79ms
init message: Loading wallet...
nFileVersion = 119900
Keys: 700 plaintext, 0 encrypted, 700 w/ metadata, 700 total
 wallet                   86ms
init message: Activating best chain...
Waiting for genesis block to be imported...
Reindexing block file blk00000.dat...
...
Loaded 495 blocks from external file in 1223ms
Reindexing finished
UPnP: ExternalIPAddress = a.b.c.d
AddLocal(a.b.c.d:18444,3)
AddPortMapping(18444, 18444, a.b.c.d) failed with code 403 (Adding fixed seed nodes as DNS doesn't seem to be available.

Notably, appearingly the chainstate key was written, but no key for blocks/index. Is this intended?

I killed the client.

(3) When I checked the databases, then I noticed obfuscate_key 3e3f21d3e1e586cd was written as expected.

chainstate:

0e006f62667573636174655f6b6579 083e3f21d3e1e586cd

Notably, the database is empty. Likely related to the forced shutdown though.

The blocks/index DB shows the same entries as before, without obfuscate_key entry.

(4) When I started patched Bitcoin Core again (without -reindex), then an error was shown, indicating that the block database is corrupted. Potentially also related to the forced shutdown.

(5) When I chose to rebuild the database, then the client started as expected.

The related log:

init message: Loading block index...
Opening LevelDB in ~/.bitcoin/regtest/blocks/index
Opened LevelDB successfully
Using obfuscation key for ~/.bitcoin/regtest/blocks/index: 0000000000000000
Opening LevelDB in ~/.bitcoin/regtest/chainstate
Opened LevelDB successfully
Using obfuscation key for ~/.bitcoin/regtest/chainstate: 3e3f21d3e1e586cd
LevelDB read failure: Corruption: corrupted compressed block contents
Corruption: corrupted compressed block contents
init message: Loading block index...
Wiping LevelDB in ~/.bitcoin/regtest/blocks/index
Opening LevelDB in ~/.bitcoin/regtest/blocks/index
Opened LevelDB successfully
Using obfuscation key for ~/.bitcoin/regtest/blocks/index: 0000000000000000
Wiping LevelDB in ~/.bitcoin/regtest/chainstate
Opening LevelDB in ~/.bitcoin/regtest/chainstate
Opened LevelDB successfully
Wrote new obfuscate key for ~/.bitcoin/regtest/chainstate: 45ef6c6c24681424
Using obfuscation key for ~/.bitcoin/regtest/chainstate: 45ef6c6c24681424
Initializing databases...
init message: Verifying blocks...
 block index              46ms
init message: Loading wallet...
nFileVersion = 119900
Keys: 700 plaintext, 0 encrypted, 700 w/ metadata, 700 total
 wallet                   53ms
init message: Activating best chain...
Waiting for genesis block to be imported...
Reindexing block file blk00000.dat...

Notably, the blocks/index DB was still unobfuscated, while the chainstate DB was obfuscated with 45ef6c6c24681424:

0e006f62667573636174655f6b6579 0845ef6c6c24681424
42 e3d545aacdbfd4186c12f7089330ed6b19e2b32a63f75ddb08125ce41aecbe31
6300934e122e90e3995afffeb9154de426fd109e36de907a3214bea5abc4d26650 44ed6a6c7ed12147e766db2011328fb61d57037e9fecbbdfc5da
63030e372a504e2ea94d86d89e117361c9ca26181ee19670fed99878988080ae46 44eca6d6126a8f9ecb23249f53107cc832364bb79785dc247ddfbc6556d2be2a8704681b19a2956a
6306739e7055e4ecfec6b0b242f5790b91582f2e76380bb92ebb947ca63887840c 44ecafb8296bc888919261f5981657cfe20a89311e90ce37d6b984e27dbc2848f8bd15f3d3469561
63071562eeda0bfa0b44f95ef8b29581fbf005c3f51c48c0a6ff055f0104c2da39 44ecec09270e190b463c7f28833ab927b89eaf0cf0d8aab82591429c53c567d0bc3f0279ece902
63071c48dce1ded71be49a06c97873544d22c1531c5f9515c9516e571dfd93d0fb 44ececcaf4bd965347de53584bf167e7fe6e8355bb985f3f202c307ec093ae5b74961028cea04dd9596e4f
...

(6) When I started patched Bitcoin Core once more with -reindex, then it went as usual, and without a freeze. blocks/index was still unobfuscated. A new obfuscation key 1af469d6cb7c0a83 was used for the chainstate, which was also expected:

0e006f62667573636174655f6b6579 081af469d6cb7c0a83
42 bcce401022abcabf3309f2b27c24f3cc46f9b6908ce3437c5709595ef5f8a096
6300934e122e90e3995afffeb9154de426fd109e36de907a3214bea5abc4d26650 1bf66fd691c53fe0b87dde9afe269111424c06c470f8a5789ac1
63030e372a504e2ea94d86d89e117361c9ca26181ee19670fed99878988080ae46 1bf7a36cfd7e913994382125bc04626f6d2d4e0d7891c28322c4b9dfb9c6a08dd81f6da1f6b68bcd
6306739e7055e4ecfec6b0b242f5790b91582f2e76380bb92ebb947ca63887840c 1bf7aa02c67fd62fce89644f77024968bd118c8bf184d09089a2815892a836efa7a610493c528bc6
63071562eeda0bfa0b44f95ef8b29581fbf005c3f51c48c0a6ff055f0104c2da39 1bf7e9b3c81a07ac19277a926c2ea780e785aab61fccb41f7a8a4726bcd17977e32407c303fd1c
63071c48dce1ded71be49a06c97873544d22c1531c5f9515c9516e571dfd93d0fb 1bf7e9701ba988f418c556e2a4e57940a17586ef548c41987f3735c42f87b0fc2b8d159221b4537e06754a

(7) When I disabled the txindex, and started patched Bitcoin Core, then an error was shown, indicating that the block database is corrupted.

(8) When I chose to rebuild the database, then the client started as expected.

(9) When I enabled the txindex once again, an error was shown, indicating that changing txindex requires reindexing. This was expected.

(10) When I disabled the txindex a second time, the database was appearingly not corrupted, and the usual "requires reindexing" dialog was shown, as expected, and repeatingly enabling or disabling the txindex was also fine.

(11) When I started Bitcoin Core 0.10 (note: DB is obfuscated) then there was an assertion failure:

main.cpp:1672: bool ConnectBlock(const CBlock &, CValidationState &, CBlockIndex *, CCoinsViewCache &, bool):
Assertion `hashPrevBlock == view.GetBestBlock()' failed.

This was expected.

(12) When I started 0.10 with -reindex, then all was fine.

(13) When I started patched Bitcoin Core with -reindex, then all was fine, too, and the chainstate DB was obfuscated.

Potential issues:

• the block index database isn't obfuscated (is this intended?)
• client hung/froze after the first reindexing (this was not reproducible)
• after that, the DBs were corrupted, potentially due to the freeze + forced shutdown (not not reproducible)
• the first time I disabled the txindex with an obfuscated chainstate, the DB was appearingly in a bad state (not not reproducible)

Side note: I dumped the DBs with https://gist.github.com/dexX7/cbd16b05f1f3870b7afc (requires py-leveldb, used with --binary).