vekexasia/bug.md Secret

## bug.md

      
    Raw
  

              bug.md
            
          
    Vekexasia BUG - a major network disruption bug.

Hello All,
in date 20-07-2017 a major fork occurred in the SHIFT network.
What happened?

PRE Within a forging round a single delegate can only forge 1 block unless tsome other previous delegates skip their block.
In the forging round 8851 (blocks from: 893851 - 893951), the last block in round (893951) was forged by worldcoinpool which has also forged block 896186.
Most of the nodes within the network reacted to 893951 with a fork #3. Any resync from a previous point in time caused fork #3. Why? Cause the delegate suppoused to forge 893951 was shoosh.
So why some nodes accepted worldcoinpool block? and Why worldcoinpool decided to forge that block even if it was not his slot?
Debugging

I've spent almost 10hours trying to figure out why some nodes thought that worldcoinpool had that forging slot.
At first I thought it was due to bad delegates sorting and slot allocation caused by different implementations of the sha256 algorithm that creates the seed used to allocate active delegates.
Then I thought it was some wrong math done by js (which is known to have issues with math).
All of the above resulted in false assumptions.
Then it hit me and I thought a very rare race condition that I've been able to partly certify thanks to the logs that were given to me from a node that accepted worldcoinpool block
The BUG

Preface:

Bob is the suppoused last block forger (shoosh above)
Alice is the last block taker (worldcoinpool above)

This happens only when all of the following conditions are met!:

we are in last block of a round
Bob double forges the block

First block with id #2 (id lowered for the sake of clarity)
Second block with id #3


Alice receives Bobs block #3
Alice receives Bobs block #2
Alice forges a block with id < #2
There were some votes in the round that will lead to ranking changes for both Bob and Alice (that should be applied when next round starts)

Note As you can see this is a very rare (and fairly hard to debug) race condition!
When in 3 Alice accepts Bob block #3, then when block #2 arrives then

Fork 5 happens
since block #2 has an id < than 3 we:
backtick the round (BUGGed)
delete block #3
process block #2 -> fork #3 <--- WHY? read later
forge kicks in and Alice forge the block + broadcast
IF Alice's block has an id < #2 then a major network disruption happens

Meanwhile the network that received both Bobs blocks
Why Fork #3

Fork #3 happens because when the first block from Bob comes, round finishes and delegate votes are reaccounted changing the active delegates chart.
When Bob receives block #2 the chart is changed and not restored correctly by "modules/rounds.js backTick" which does not rollback properly the votes (and other stuff I guess).
For this reason, when bob receives block #2 he thinks that Alice is not the correct delegate -> Fork #3.
Will it change a thing if Fork #3 didn't happen?

Short answer? NO.
Long answer? ->
forge routine triggers every 1s. If the forging routine triggers just after block#3 is deleted (or while block #2 is being processed) then this will happen!
====
Mitigation Ideas

When evaluating a fork #5 we should check if previous accepted block has same delegate than new block in correlation with round position.
(Of course) implement a proper rollback in backwardTick
Evidences

Logs from a node which accepted Bob block:

delegate 9a4d*** => Bob
delegate 20be*** => Alice
[inf] 2017-07-20 14:20:43 | Received new block id: 1951415658472361437 height: 893951 round: 8851 slot: 1350046 reward: 110000000
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:43 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:43 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:43 | Last block stands
[inf] 2017-07-20 14:20:44 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:44 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:44 | Last block stands
[inf] 2017-07-20 14:20:44 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:44 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:44 | Last block stands
[inf] 2017-07-20 14:20:44 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:44 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:44 | Last block stands
[inf] 2017-07-20 14:20:45 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[WRN] 2017-07-20 14:20:45 | Delegate forging on multiple nodes - 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3
[inf] 2017-07-20 14:20:45 | Last block stands
[inf] 2017-07-20 14:20:46 | Fork - {"delegate":"20bef0f43dee2291391c83d100fb77b886aa762e18e96c85663c754e96de5cbe","block":{"id":"15598365189616631313","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[inf] 2017-07-20 14:20:46 | Last block loses
[WRN] 2017-07-20 14:20:46 | Deleting last block - {"version":0,"totalAmount":187139653,"totalFee":3000000,"reward":110000000,"payloadHash":"adca7fe47243803b25d214c9983150913bf14a2a235c4a8e304ceaa9ae54b15e","timestamp":36451242,"numberOfTransactions":3,"payloadLength":543,"previousBlock":"7993208541717327000","generatorPublicKey":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","transactions":[{"type":0,"amount":36865529,"senderPublicKey":"30f34b1f3c13820698e65b60573d10b57d600f9468616e7401ea11df11afb80c","timestamp":36451182,"asset":{},"recipientId":"14675176465578734374S","signature":"075adf2bc46533dc96d6888efef234cb2b67359bc3434da974ec073e741a234e440f1f1605547d2f931d8e74aeb8750fa984ca97cbd8d7cba12dc82472ae9909","signSignature":"bb3c62a3a9f032a9ed68bafb6098e4b2c0d35e880b34b122f4e25c717090533d777e7378f69d094c3232a9d70f8159724ed179bc289dfa84f822956c26cc4d07","id":"15098583552945968542","fee":1000000,"senderId":"7876956936945693367S","relays":2,"receivedAt":"2017-07-20T14:19:55.427Z","blockId":"1951415658472361437"},{"type":0,"amount":67298517,"senderPublicKey":"30f34b1f3c13820698e65b60573d10b57d600f9468616e7401ea11df11afb80c","timestamp":36451213,"asset":{},"recipientId":"18113470329236636101S","signature":"1e12ab50fa80d95c35f23e7cfa8b31ed34c9758f7079b22a4184c3cfd10126d917c12464e891f3240c1f60d65751a4db926ccc4505fdda11d31e73802d1cce0d","signSignature":"80cc093f3bb77d9cd1544d5d47ae695f3fcff9494ba3b3df203aa573ea4d47c095a405c3db7f9ab5471dbbd0dcda12b0cc25c0d7ab500fd51b586c1b608c1304","id":"9290290874924302612","fee":1000000,"senderId":"7876956936945693367S","relays":2,"receivedAt":"2017-07-20T14:20:25.458Z","blockId":"1951415658472361437"},{"type":0,"amount":82975607,"senderPublicKey":"30f34b1f3c13820698e65b60573d10b57d600f9468616e7401ea11df11afb80c","timestamp":36451223,"asset":{},"recipientId":"14982205216512991131S","signature":"3be68da459b9b6eba9b31945fc3478905d28f7b4f39e19ae432368dc4522d37be369fcc4ac790c1d7baf8f93a652b26a5eafa945c8ce4a2a06a0a5501fd16d09","signSignature":"aadd43e588451bb343db0faabaa6f923b2e984cb88f8f02ba0b15c9949af97776ef5b716c4166509e6d3ce5d455e5ba800434aba5a504593144dade06292c900","id":"5444443047466608768","fee":1000000,"senderId":"7876956936945693367S","relays":2,"receivedAt":"2017-07-20T14:20:35.465Z","blockId":"1951415658472361437"}],"blockSignature":"8b715a5f0aed6b6d959c228469ad1cc3eefee7c50bd80d3957b5db310aa329d76848118aa18e97f1a4dcb2c2249524765196b7c8f94cfac81c1f244bf8853f05","id":"1951415658472361437","height":893951,"secondsAgo":4.01,"fresh":true,"relays":2}
[inf] 2017-07-20 14:20:46 | Received new block id: 15598365189616631313 height: 893951 round: 8851 slot: 1350046 reward: 110000000
[inf] 2017-07-20 14:20:46 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"1951415658472361437","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[inf] 2017-07-20 14:20:46 | Last block stands
[inf] 2017-07-20 14:20:46 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"1951415658472361437","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[inf] 2017-07-20 14:20:46 | Last block stands
[inf] 2017-07-20 14:20:46 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"1951415658472361437","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[inf] 2017-07-20 14:20:46 | Last block stands
[inf] 2017-07-20 14:20:46 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[inf] 2017-07-20 14:20:46 | Last block stands
[inf] 2017-07-20 14:20:46 | Fork - {"delegate":"9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3","block":{"id":"858346963379129312","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":5}
[inf] 2017-07-20 14:20:46 | Last block stands
[inf] 2017-07-20 14:21:10 | Received new block id: 9519699560334209750 height: 893952 round: 8852 slot: 1350047 reward: 110000000

Notice how this node received the 2 forged nodes in order (by id) but that caused Alice block to get accepted anyway.
Logs from a node trying to resync. (that will never succeed)

[inf] 2017-07-21 15:47:54 | Found common block: 7993208541717327000 with: 51.15.52.54:9305
[inf] 2017-07-21 15:47:54 | Loading blocks from: 51.15.52.54:9305
[ERR] 2017-07-21 15:47:54 | Expected generator: 9a4d69d5ba637fea3934dafbb613b4ba235b07e569fef5dcf1185debde131dc3 Received generator: 20bef0f43dee2291391c83d100fb77b886aa762e18e96c85663c754e96de5cbe
[inf] 2017-07-21 15:47:54 | Fork - {"delegate":"20bef0f43dee2291391c83d100fb77b886aa762e18e96c85663c754e96de5cbe","block":{"id":"15598365189616631313","timestamp":36451242,"height":893951,"previousBlock":"7993208541717327000"},"cause":3}
[ERR] 2017-07-21 15:47:54 | Error loading blocks: Failed to verify slot: 1350046


===
PoC Code

I've set up a PoC code.
It basically does the following:

run lisk like app.js do (copied and pasted the code there)
when ready:

generate blocks till lastBlockInRound - 2
find last block delegate
create a vote/unvote tx for such delegate (so that its ranking changes in the next round)
forge block "lastBlockInRound - 1"
forge block "lastBlockInRound"
craft a new block with lower id (changing version)
call modules.blocks.submodules.process.onReceiveBlock to simulate an incoming block
here happens the following:

fork 5
previously forged block "lastBlockInRound" gets:

deleted since new one has an higher id (not always -> see last line in this doc)
rounds.backtick! (where the bug is)


new incoming block gets discarded due to invalid version
lastBlock is now back at lastBlockInRound - 1


I verify that the next delegate changed since round votes were not rolled back.


It requires the following:

running redis (-> test/config.json)
running postgres (w/ login details -> test/config.json)
node 7
reinstall node_modules after node 7 switch
change line 90 of logic/transaction.js to accept timestamp from input (just for the test)

	var trs = {
   	type: data.type,
   	amount: 0,
   	senderPublicKey: data.sender.publicKey,
   	requesterPublicKey: data.requester ? data.requester.publicKey.toString('hex') : null,
 timestamp: data.timestamp || slots.getTime(),
   	asset: {}
   };

place vekexasiabug.js in the root folder of lisk
run it at least twice. (Every time you launch it it will generate 101 blocks manually forging the blocks)

Note Relevant code starts at line 715 till end of file. (onInit is being called after app initialization. (copied from app.js) )
If the code ends with DIDNt rollback properly! WTF? it's because there is another minor bug here. Comment says: .... keep one with lower id but id comparison is lexicographical not "numeric" since block.id is a string (or could be a string in my tests)

  
## vekexasiabug.js
// const { spawn } = require('child_process');
const fs                     = require('fs');
const configContent          = require('./test/config.json');
configContent.forging.secret = [];
fs.writeFileSync('./config.vekexasiabug.json', JSON.stringify(configContent, null, 2));
if (!fs.existsSync('./genesisBlock.backup.json')) {
  fs.renameSync('./genesisBlock.json', './genesisBlock.backup.json');
  fs.writeFileSync('./genesisBlock.json', fs.readFileSync('./test/genesisBlock.json'));
}
// const app = spawn('/home/abaccega/.nvm/versions/node/v6.11.0/bin/node', ['./app.js', '-c', './config.vekexasiabug.json' ])
//
// app.stdout.on('data', data => console.log(data.toString()));
// app.on('exit', data => console.log('exit', data));
// app.stderr.on('data', (data) => {
//   console.log(data.toString());
// });

//docker run -p 5432:5432 -e POSTGRES_USER=coin -e POSTGRES_DB=db -e POSTGRES_PASSWORD=password --name pg postgres:9.6.3-alpine


'use strict';
/**
 * A node-style callback as used by {@link logic} and {@link modules}.
 * @see {@link https://nodejs.org/api/errors.html#errors_node_js_style_callbacks}
 * @callback nodeStyleCallback
 * @param {?Error} error - Error, if any, otherwise `null`.
 * @param {Data} data - Data, if there hasn't been an error.
 */
/**
 * A triggered by setImmediate callback as used by {@link logic}, {@link modules} and {@link helpers}.
 * Parameters formats: (cb, error, data), (cb, error), (cb).
 * @see {@link https://nodejs.org/api/timers.html#timers_setimmediate_callback_args}
 * @callback setImmediateCallback
 * @param {function} cb - Callback function.
 * @param {?Error} [error] - Error, if any, otherwise `null`.
 * @param {Data} [data] - Data, if there hasn't been an error and the function should return data.
 */

/**
 * Main entry point.
 * Loads the lisk modules, the lisk api and run the express server as Domain master.
 * CLI options available.
 * @module app
 */

var async         = require('async');
var checkIpInList = require('./helpers/checkIpInList.js');
var extend        = require('extend');
// var fs = require('fs');

var genesisblock = require('./genesisBlock.json');
var git          = require('./helpers/git.js');
var https        = require('https');
var Logger       = require('./logger.js');
var packageJson  = require('./package.json');
var path         = require('path');
var program      = require('commander');
var httpApi      = require('./helpers/httpApi.js');
var Sequence     = require('./helpers/sequence.js');
var util         = require('util');
var z_schema     = require('./helpers/z_schema.js');

process.stdin.resume();

var versionBuild = fs.readFileSync(path.join(__dirname, 'build'), 'utf8');

/**
 * @property {string} - Hash of last git commit.
 */
var lastCommit = '';

if (typeof gc !== 'undefined') {
  setInterval(function () {
    gc();
  }, 60000);
}

program
  .version(packageJson.version)
  .option('-c, --config <path>', 'config file path')
  .option('-p, --port <port>', 'listening port number')
  .option('-a, --address <ip>', 'listening host name or ip')
  .option('-x, --peers [peers...]', 'peers list')
  .option('-l, --log <level>', 'log level')
  .option('-s, --snapshot <round>', 'verify snapshot')
  .parse(process.argv);

/**
 * @property {object} - The default list of configuration options. Can be updated by CLI.
 * @default 'config.json'
 */
var appConfig = require('./helpers/config.js')(program.config || './config.vekexasiabug.json');

if (program.port) {
  appConfig.port = program.port;
}
appConfig.port = appConfig.port + 1;

if (program.address) {
  appConfig.address = program.address;
}

if (program.peers) {
  if (typeof program.peers === 'string') {
    appConfig.peers.list = program.peers.split(',').map(function (peer) {
      peer = peer.split(':');
      return {
        ip  : peer.shift(),
        port: peer.shift() || appConfig.port
      };
    });
  } else {
    appConfig.peers.list = [];
  }
}

if (program.log) {
  appConfig.consoleLogLevel = program.log;
}
appConfig.consoleLogLevel = 'warn';


if (program.snapshot) {
  appConfig.loading.snapshot = Math.abs(
    Math.floor(program.snapshot)
  );
}

if (process.env.NODE_ENV === 'test') {
  appConfig.coverage = true;
}

// Define top endpoint availability
process.env.TOP = appConfig.topAccounts;

/**
 * The config object to handle lisk modules and lisk api.
 * It loads `modules` and `api` folders content.
 * Also contains db configuration from config.json.
 * @property {object} db - Config values for database.
 * @property {object} modules - `modules` folder content.
 * @property {object} api - `api/http` folder content.
 */
var config = {
  db          : appConfig.db,
  cache       : appConfig.redis,
  cacheEnabled: appConfig.cacheEnabled,
  modules     : {
    accounts       : './modules/accounts.js',
    transactions   : './modules/transactions.js',
    blocks         : './modules/blocks.js',
    signatures     : './modules/signatures.js',
    transport      : './modules/transport.js',
    loader         : './modules/loader.js',
    system         : './modules/system.js',
    peers          : './modules/peers.js',
    delegates      : './modules/delegates.js',
    rounds         : './modules/rounds.js',
    multisignatures: './modules/multisignatures.js',
    dapps          : './modules/dapps.js',
    crypto         : './modules/crypto.js',
    cache          : './modules/cache.js'
  },
  api         : {
    accounts       : { http: './api/http/accounts.js' },
    blocks         : { http: './api/http/blocks.js' },
    dapps          : { http: './api/http/dapps.js' },
    delegates      : { http: './api/http/delegates.js' },
    loader         : { http: './api/http/loader.js' },
    multisignatures: { http: './api/http/multisignatures.js' },
    peers          : { http: './api/http/peers.js' },
    signatures     : { http: './api/http/signatures.js' },
    transactions   : { http: './api/http/transactions.js' },
    transport      : { http: './api/http/transport.js' }
  }
};

/**
 * Logger holder so we can log with custom functionality.
 * The Object is initialized here and pass to others as parameter.
 * @property {object} - Logger instance.
 */
var logger = new Logger({
  echo    : appConfig.consoleLogLevel, errorLevel: appConfig.fileLogLevel,
  filename: appConfig.logFileName
});

// Trying to get last git commit
try {
  lastCommit = git.getLastCommit();
} catch (err) {
  logger.debug('Cannot get last git commit', err.message);
}

/**
 * Creates the express server and loads all the Modules and logic.
 * @property {object} - Domain instance.
 */
var d = require('domain').create();

d.on('error', function (err) {
  logger.fatal('Domain master', { message: err.message, stack: err.stack });
  process.exit(0);
});

// runs domain
d.run(function () {
  var modules = [];
  async.auto({
    /**
     * Loads `payloadHash`.
     * Then updates config.json with new random  password.
     * @method config
     * @param {nodeStyleCallback} cb - Callback function with the mutated `appConfig`.
     * @throws {Error} If failed to assign nethash from genesis block.
     */
    config: function (cb) {
      try {
        appConfig.nethash = Buffer.from(genesisblock.payloadHash, 'hex').toString('hex');
      } catch (e) {
        logger.error('Failed to assign nethash from genesis block');
        throw Error(e);
      }
      cb(null, appConfig);
    },

    logger: function (cb) {
      cb(null, logger);
    },

    build: function (cb) {
      cb(null, versionBuild);
    },

    /**
     * Returns hash of last git commit.
     * @method lastCommit
     * @param {nodeStyleCallback} cb - Callback function with Hash of last git commit.
     */
    lastCommit: function (cb) {
      cb(null, lastCommit);
    },

    genesisblock: function (cb) {
      cb(null, {
        block: genesisblock
      });
    },

    schema: function (cb) {
      cb(null, new z_schema());
    },

    /**
     * Once config is completed, creates app, http & https servers & sockets with express.
     * @method network
     * @param {object} scope - The results from current execution,
     * at leats will contain the required elements.
     * @param {nodeStyleCallback} cb - Callback function with created Object:
     * `{express, app, server, io, https, https_io}`.
     */
    network: ['config', function (scope, cb) {
      var express     = require('express');
      var compression = require('compression');
      var cors        = require('cors');
      var app         = express();

      if (appConfig.coverage) {
        var im = require('istanbul-middleware');
        logger.debug('Hook loader for coverage - do not use in production environment!');
        im.hookLoader(__dirname);
        app.use('/coverage', im.createHandler());
      }

      require('./helpers/request-limiter')(app, appConfig);

      app.use(compression({ level: 9 }));
      app.use(cors());
      app.options('*', cors());

      var server = require('http').createServer(app);
      var io     = require('socket.io')(server);

      var privateKey, certificate, https, https_io;

      if (scope.config.ssl.enabled) {
        privateKey  = fs.readFileSync(scope.config.ssl.options.key);
        certificate = fs.readFileSync(scope.config.ssl.options.cert);

        https = require('https').createServer({
          key    : privateKey,
          cert   : certificate,
          ciphers: 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:' + 'ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA256:HIGH:' + '!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!SRP:!CAMELLIA'
        }, app);

        https_io = require('socket.io')(https);
      }

      cb(null, {
        express : express,
        app     : app,
        server  : server,
        io      : io,
        https   : https,
        https_io: https_io
      });
    }],

    dbSequence: ['logger', function (scope, cb) {
      var sequence = new Sequence({
        onWarning: function (current, limit) {
          scope.logger.warn('DB queue', current);
        }
      });
      cb(null, sequence);
    }],

    sequence: ['logger', function (scope, cb) {
      var sequence = new Sequence({
        onWarning: function (current, limit) {
          scope.logger.warn('Main queue', current);
        }
      });
      cb(null, sequence);
    }],

    balancesSequence: ['logger', function (scope, cb) {
      var sequence = new Sequence({
        onWarning: function (current, limit) {
          scope.logger.warn('Balance queue', current);
        }
      });
      cb(null, sequence);
    }],

    /**
     * Once config, genesisblock, logger, build and network are completed,
     * adds configuration to `network.app`.
     * @method connect
     * @param {object} scope - The results from current execution,
     * at leats will contain the required elements.
     * @param {function} cb - Callback function.
     */
    connect: ['config', 'genesisblock', 'logger', 'build', 'network', function (scope, cb) {
      var path           = require('path');
      var bodyParser     = require('body-parser');
      var methodOverride = require('method-override');
      var queryParser    = require('express-query-int');
      var randomString   = require('randomstring');

      scope.nonce = randomString.generate(16);
      scope.network.app.use(require('express-domain-middleware'));
      scope.network.app.use(bodyParser.raw({ limit: '2mb' }));
      scope.network.app.use(bodyParser.urlencoded({ extended: true, limit: '2mb', parameterLimit: 5000 }));
      scope.network.app.use(bodyParser.json({ limit: '2mb' }));
      scope.network.app.use(methodOverride());

      var ignore = ['id', 'name', 'lastBlockId', 'blockId', 'transactionId', 'address', 'recipientId', 'senderId', 'previousBlock'];

      scope.network.app.use(queryParser({
        parser: function (value, radix, name) {
          if (ignore.indexOf(name) >= 0) {
            return value;
          }

          // Ignore conditional fields for transactions list
          if (/^.+?:(blockId|recipientId|senderId)$/.test(name)) {
            return value;
          }

          /*eslint-disable eqeqeq */
          if (isNaN(value) || parseInt(value) != value || isNaN(parseInt(value, radix))) {
            return value;
          }
          /*eslint-enable eqeqeq */
          return parseInt(value);
        }
      }));

      scope.network.app.use(require('./helpers/z_schema-express.js')(scope.schema));

      scope.network.app.use(httpApi.middleware.logClientConnections.bind(null, scope.logger));

      /* Instruct browser to deny display of <frame>, <iframe> regardless of origin.
       *
       * RFC -> https://tools.ietf.org/html/rfc7034
       */
      scope.network.app.use(httpApi.middleware.attachResponseHeader.bind(null, 'X-Frame-Options', 'DENY'));
      /* Set Content-Security-Policy headers.
       *
       * frame-ancestors - Defines valid sources for <frame>, <iframe>, <object>, <embed> or <applet>.
       *
       * W3C Candidate Recommendation -> https://www.w3.org/TR/CSP/
       */
      scope.network.app.use(httpApi.middleware.attachResponseHeader.bind(null, 'Content-Security-Policy', 'frame-ancestors \'none\''));

      scope.network.app.use(httpApi.middleware.applyAPIAccessRules.bind(null, scope.config));

      cb();
    }],

    ed: function (cb) {
      cb(null, require('./helpers/ed.js'));
    },

    bus    : ['ed', function (scope, cb) {
      var changeCase = require('change-case');
      var bus        = function () {
        this.message = function () {
          var args = [];
          Array.prototype.push.apply(args, arguments);
          var topic     = args.shift();
          var eventName = 'on' + changeCase.pascalCase(topic);

          // executes the each module onBind function
          modules.forEach(function (module) {
            if (typeof(module[eventName]) === 'function') {
              module[eventName].apply(module[eventName], args);
            }
            if (module.submodules) {
              async.each(module.submodules, function (submodule) {
                if (submodule && typeof(submodule[eventName]) === 'function') {
                  submodule[eventName].apply(submodule[eventName], args);
                }
              });
            }
          });
        };
      };
      cb(null, new bus());
    }],
    db     : function (cb) {
      var db = require('./helpers/database.js');
      db.connect(config.db, logger, cb);
    },
    /**
     * It tries to connect with redis server based on config. provided in config.json file
     * @param {function} cb
     */
    cache  : function (cb) {
      var cache = require('./helpers/cache.js');
      cache.connect(config.cacheEnabled, config.cache, logger, cb);
    },
    /**
     * Once db, bus, schema and genesisblock are completed,
     * loads transaction, block, account and peers from logic folder.
     * @method logic
     * @param {object} scope - The results from current execution,
     * at leats will contain the required elements.
     * @param {function} cb - Callback function.
     */
    logic  : ['db', 'bus', 'schema', 'genesisblock', function (scope, cb) {
      var Transaction = require('./logic/transaction.js');
      var Block       = require('./logic/block.js');
      var Account     = require('./logic/account.js');
      var Peers       = require('./logic/peers.js');

      async.auto({
        bus         : function (cb) {
          cb(null, scope.bus);
        },
        db          : function (cb) {
          cb(null, scope.db);
        },
        ed          : function (cb) {
          cb(null, scope.ed);
        },
        logger      : function (cb) {
          cb(null, logger);
        },
        schema      : function (cb) {
          cb(null, scope.schema);
        },
        genesisblock: function (cb) {
          cb(null, {
            block: genesisblock
          });
        },
        account     : ['db', 'bus', 'ed', 'schema', 'genesisblock', 'logger', function (scope, cb) {
          new Account(scope.db, scope.schema, scope.logger, cb);
        }],
        transaction : ['db', 'bus', 'ed', 'schema', 'genesisblock', 'account', 'logger', function (scope, cb) {
          new Transaction(scope.db, scope.ed, scope.schema, scope.genesisblock, scope.account, scope.logger, cb);
        }],
        block       : ['db', 'bus', 'ed', 'schema', 'genesisblock', 'account', 'transaction', function (scope, cb) {
          new Block(scope.ed, scope.schema, scope.transaction, cb);
        }],
        peers       : ['logger', function (scope, cb) {
          new Peers(scope.logger, cb);
        }]
      }, cb);
    }],
    /**
     * Once network, connect, config, logger, bus, sequence,
     * dbSequence, balancesSequence, db and logic are completed,
     * loads modules from `modules` folder using `config.modules`.
     * @method modules
     * @param {object} scope - The results from current execution,
     * at leats will contain the required elements.
     * @param {nodeStyleCallback} cb - Callback function with resulted load.
     */
    modules: ['network', 'connect', 'config', 'logger', 'bus', 'sequence', 'dbSequence', 'balancesSequence', 'db', 'logic', 'cache', function (scope, cb) {

      var tasks = {};

      Object.keys(config.modules).forEach(function (name) {
        tasks[name] = function (cb) {
          var d = require('domain').create();

          d.on('error', function (err) {
            scope.logger.fatal('Domain ' + name, { message: err.message, stack: err.stack });
          });

          d.run(function () {
            logger.debug('Loading module', name);
            var Klass = require(config.modules[name]);
            var obj   = new Klass(cb, scope);
            modules.push(obj);
          });
        };
      });

      async.parallel(tasks, function (err, results) {
        cb(err, results);
      });
    }],

    /**
     * Loads api from `api` folder using `config.api`, once modules, logger and
     * network are completed.
     * @method api
     * @param {object} scope - The results from current execution,
     * at leats will contain the required elements.
     * @param {function} cb - Callback function.
     */
    api: ['modules', 'logger', 'network', function (scope, cb) {
      Object.keys(config.api).forEach(function (moduleName) {
        Object.keys(config.api[moduleName]).forEach(function (protocol) {
          var apiEndpointPath = config.api[moduleName][protocol];
          try {
            var ApiEndpoint = require(apiEndpointPath);
            new ApiEndpoint(scope.modules[moduleName], scope.network.app, scope.logger, scope.modules.cache);
          } catch (e) {
            scope.logger.error('Unable to load API endpoint for ' + moduleName + ' of ' + protocol, e);
          }
        });
      });

      scope.network.app.use(httpApi.middleware.errorLogger.bind(null, scope.logger));
      cb();
    }],

    ready: ['modules', 'bus', 'logic', function (scope, cb) {
      scope.bus.message('bind', scope.modules);
      scope.logic.transaction.bindModules(scope.modules);
      scope.logic.peers.bindModules(scope.modules);
      cb();
    }],

    /**
     * Once 'ready' is completed, binds and listens for connections on the
     * specified host and port for `scope.network.server`.
     * @method listen
     * @param {object} scope - The results from current execution,
     * at leats will contain the required elements.
     * @param {nodeStyleCallback} cb - Callback function with `scope.network`.
     */
    listen: ['ready', function (scope, cb) {
      scope.network.server.listen(scope.config.port, scope.config.address, function (err) {
        scope.logger.info('Lisk started: ' + scope.config.address + ':' + scope.config.port);

        if (!err) {
          if (scope.config.ssl.enabled) {
            scope.network.https.listen(scope.config.ssl.options.port, scope.config.ssl.options.address, function (err) {
              scope.logger.info('Lisk https started: ' + scope.config.ssl.options.address + ':' + scope.config.ssl.options.port);

              cb(err, scope.network);
            });
          } else {
            cb(null, scope.network);
          }
        } else {
          cb(err, scope.network);
        }
      });
    }]
  }, function (err, scope) {
    if (err) {
      logger.fatal(err);
    } else {
      onInit(scope, modules);
      /**
       * Handles app instance (acts as global variable, passed as parameter).
       * @global
       * @typedef {Object} scope
       * @property {Object} api - Undefined.
       * @property {undefined} balancesSequence - Sequence function, sequence Array.
       * @property {string} build - Empty.
       * @property {Object} bus - Message function, bus constructor.
       * @property {Object} config - Configuration.
       * @property {undefined} connect - Undefined.
       * @property {Object} db - Database constructor, database functions.
       * @property {function} dbSequence - Database function.
       * @property {Object} ed - Crypto functions from lisk node-sodium.
       * @property {Object} genesisblock - Block information.
       * @property {string} lastCommit - Hash transaction.
       * @property {Object} listen - Network information.
       * @property {Object} logger - Log functions.
       * @property {Object} logic - several logic functions and objects.
       * @property {Object} modules - Several modules functions.
       * @property {Object} network - Several network functions.
       * @property {string} nonce
       * @property {undefined} ready
       * @property {Object} schema - ZSchema with objects.
       * @property {Object} sequence - Sequence function, sequence Array.
       * @todo logic repeats: bus, ed, genesisblock, logger, schema.
       * @todo description for nonce and ready
       */
      scope.logger.info('Modules ready and launched');
      /**
       * Event reporting a cleanup.
       * @event cleanup
       */
      /**
       * Receives a 'cleanup' signal and cleans all modules.
       * @listens cleanup
       */
      process.once('cleanup', function () {
        scope.logger.info('Cleaning up...');
        async.eachSeries(modules, function (module, cb) {
          if (typeof(module.cleanup) === 'function') {
            module.cleanup(cb);
          } else {
            setImmediate(cb);
          }
        }, function (err) {
          if (err) {
            scope.logger.error(err);
          } else {
            scope.logger.info('Cleaned up successfully');
          }
          process.exit(1);
        });
      });

      /**
       * Event reporting a SIGTERM.
       * @event SIGTERM
       */
      /**
       * Receives a 'SIGTERM' signal and emits a cleanup.
       * @listens SIGTERM
       */
      process.once('SIGTERM', function () {
        /**
         * emits cleanup once 'SIGTERM'.
         * @emits cleanup
         */
        process.emit('cleanup');
      });

      /**
       * Event reporting an exit.
       * @event exit
       */
      /**
       * Receives an 'exit' signal and emits a cleanup.
       * @listens exit
       */
      process.once('exit', function () {
        /**
         * emits cleanup once 'exit'.
         * @emits cleanup
         */
        process.emit('cleanup');
      });

      /**
       * Event reporting a SIGINT.
       * @event SIGINT
       */
      /**
       * Receives a 'SIGINT' signal and emits a cleanup.
       * @listens SIGINT
       */
      process.once('SIGINT', function () {
        /**
         * emits cleanup once 'SIGINT'.
         * @emits cleanup
         */
        process.emit('cleanup');
      });
    }
  });
});

/**
 * Event reporting an uncaughtException.
 * @event uncaughtException
 */
/**
 * Receives a 'uncaughtException' signal and emits a cleanup.
 * @listens uncaughtException
 */
process.on('uncaughtException', function (err) {
  // Handle error safely
  logger.fatal('System error', { message: err.message, stack: err.stack });
  /**
   * emits cleanup once 'uncaughtException'.
   * @emits cleanup
   */
  process.emit('cleanup');
});

const genesisDelegates = require('./test/genesisDelegates.json');
const crypto           = require('crypto');
const slots            = require('./helpers/slots');
const BigNumber        = require('bignumber.js');
const blockTime        = 10;

async function onInit(scope, modules) {
  try {
    await wait(3000);

    console.log('generating...');

    // I manually forge N blocks till endOfRound - 2
    await genBlocksTill2BeforeEndOfRound(scope);

    // Fetch the list of delegates in the chart.
    // Will use that to compute (before it happens) the suppoused delegate taker
    // and decide who to vote/unvote to change the ranking of the last block forger
    const { delegates }     = await toPromise(cback => scope.modules.delegates.getDelegates({}, cback));

    // get The last block delegate
    const lastBlockDelegate = await getDelegateInFuture(scope, 2);

    // Find its current ranking.
    const delegateRank      = delegates.map(d => d.publicKey).indexOf(lastBlockDelegate);
    let votes               = []; // tx asset

    // will hold the publicKey of the taker. The taker is the delegate that
    // Will be identified (after the bug happens) as the last block in round forger.
    let replacingDelegate;
    if (delegates[delegateRank].approval === 100) {
      // unvote
      votes.push(`-${delegates[delegateRank].publicKey}`);
      replacingDelegate = delegates[(delegateRank + 1) % delegates.length].publicKey;
    } else {
      // vote
      votes.push(`+${delegates[delegateRank].publicKey}`);
      replacingDelegate = delegates[(delegateRank + delegates.length - 1) % delegates.length].publicKey;
    }

    // Create tx
    const secret  = 'wagon stock borrow episode laundry kitten salute link globe zero feed marble';
    const keypair = scope.ed.makeKeypair(crypto.createHash('sha256')
      .update(secret, 'utf8').digest());
    const tx = scope.logic.transaction.create({
      type     : 3, // Vote
      sender   : {
        publicKey: 'c094ebee7ec0c50ebee32918655e089f6e1a604b83bcaa760293c61e0f18ab6f',
        address  : '16313739661670634666L'
      },
      timestamp: scope.modules.blocks.lastBlock.get().timestamp,
      keypair,
      votes
    });

    // Process tx
    await toPromise(cback => scope.modules.transactions.receiveTransactions([tx], true, cback));
    await toPromise(cback => scope.modules.transactions.fillPool(cback));

    // forge round - 1
    await genBlocks(1, scope);

    // Keep block round-1 in memory
    const previousBlock = scope.modules.blocks.lastBlock.get();

    // Get the last (REAL) delegate - the one that should always forge the last block in round
    // For this given timestamp.
    let lastDel = await getDelegateInFuture(scope, 1);
    // Next is last in round.
    await genBlocks(1, scope);

    const lastInRoundBlock = scope.modules.blocks.lastBlock.get();
    // Iterates over version to change blockid so that it's < than lastInRoundBlock.id
    const newBlock = findFork5Block(
      scope,
      delegateKeypair(lastDel, scope.ed),
      lastInRoundBlock,
      previousBlock
    );

    // block won't pass verification but it will cause backTick
    scope.modules.blocks.submodules.process.onReceiveBlock(newBlock);
    await wait(5000);

    // Verify that now lastblock is (round-1)
    if (previousBlock.id !== scope.modules.blocks.lastBlock.get().id) {
      console.log('DIDNt rollback properly! WTF?');
      process.exit(1);
    }

    let newlastDel = await getDelegateInFuture(scope, 1);
    if (newlastDel === replacingDelegate) {
      console.log(`BUG! Last round block forger is now ${newlastDel} while should've been ${lastDel}`);
      process.exit(1);
    } else {
      console.log('NO BUG');
    }


  } catch (e) {
    console.log(e);
    console.log(e.stack);
  }

}

async function genBlocksTill2BeforeEndOfRound(scope) {
  // scope.modules.blocks.lastBlock.get()
  const curHeight   = scope.modules.blocks.lastBlock.get().height;
  const curRound    = scope.modules.rounds.calc(curHeight);
  const nextRoundAt = curRound * 101 + 1;
  let todo          = nextRoundAt - curHeight - 3;
  if (todo <= 0) {
    // We're already on the end of the round.
    await genBlocks(10, scope);
    return await genBlocksTill2BeforeEndOfRound(scope);
  }
  return genBlocks(todo, scope);
}

async function getDelegateInFuture(scope, blocksInFuture = 1) {
  const lastBlock = scope.modules.blocks.lastBlock.get();
  const slot      = slots.getSlotNumber(lastBlock.timestamp + blockTime * blocksInFuture);
  const height    = lastBlock.height + blocksInFuture;
  const delegates = await toPromise(cb => scope.modules.delegates.generateDelegateList(height, cb));
  return delegates[slot % slots.delegates];
}

function findFork5Block(scope, keypair, ofBlock, previousBlock) {
  let block = scope.logic.block.create({
    keypair,
    timestamp   : ofBlock.timestamp,
    previousBlock,
    transactions: []
  });

  for (let i = 0; ; i++) {
    block.version        = i; // hacky way to get a different id
    block.blockSignature = null;
    block.blockSignature = scope.logic.block.sign(block, keypair);
    block                = scope.logic.block.objectNormalize(block);
    console.log(i, scope.logic.block.getId(block));
    if (new BigNumber(scope.logic.block.getId(block)).lt(new BigNumber(ofBlock.id))){
      scope.modules.blocks.submodules.verify.verifyBlock(block);
      block.height= ofBlock.height;
      return block;
    }
  }


}

/**
 * Generates n blocks as they would've been forged from this node.
 * @param n
 * @returns {Promise.<*>}
 */
async function genBlocks(n, scope) {
  for (let i = 0; i < n; i++) {
    const lastBlock = scope.modules.blocks.lastBlock.get();
    const slot      = slots.getSlotNumber(lastBlock.timestamp + blockTime);
    const delegate  = await getDelegateInFuture(scope);
    const keyPair   = delegateKeypair(delegate, scope.ed);
    await toPromise(cback => scope.modules.blocks.submodules.process
      .generateBlock(
        keyPair,
        slots.getSlotTime(slot),
        cback
      )
    );
  }
  return scope.modules.blocks.lastBlock.get();
}

/**
 * Gets delegate secret from a pubKEy
 * @param pubKey
 */
function delegateSecret(pubKey) {
  const delegate = genesisDelegates.delegates.filter(d => d.publicKey === pubKey)[0];
  return delegate.secret;
}

/**
 * get Delegate keypair from pub/priv Key
 * @param pubKey
 * @param ed
 * @returns {{publicKey, privateKey}}
 */
function delegateKeypair(pubKey, ed) {
  return ed.makeKeypair(crypto.createHash('sha256').update(delegateSecret(pubKey), 'utf8').digest());
}

/**
 * Just waits for some time.
 * @param ms
 * @returns {Promise}
 */
function wait(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

/**
 * A simple (stupid) way to change callback-style fns to promise style.
 * @param callbackkable
 * @returns {Promise}
 */
function toPromise(callbackkable) {
  return new Promise((resolve, reject) => {
    callbackkable((err, res) => {
      if (err) {
        return reject(err);
      }
      return resolve(res);
    });
  });
}