proper crash at midnight and corrupted database data files

gistfile1.txt

2014-01-02 18:52:54 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 18:53:54 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 18:53:55 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:32:0
5 (UTC) to 2014-01-02 23:53:55 (UTC)
2014-01-02 18:53:55 (DEBUG):    minimum trade treshold: 2014-01-02 23:52:06 UTC
2014-01-02 18:53:55 (DEBUG):    processing 44 trade(s)
2014-01-02 18:53:55 (DEBUG):    inserting candle 1432 (2014-01-02 23:52:00 UTC)
volume: 0.48313090999999997
2014-01-02 18:53:55 (DEBUG):    Leftovers: 1433
2014-01-02 18:53:55 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 18:54:55 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 18:55:00 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:32:4
7 (UTC) to 2014-01-02 23:54:56 (UTC)
2014-01-02 18:55:00 (DEBUG):    minimum trade treshold: 2014-01-02 23:53:55 UTC
2014-01-02 18:55:00 (DEBUG):    processing 58 trade(s)
2014-01-02 18:55:00 (DEBUG):    inserting candle 1433 (2014-01-02 23:53:00 UTC)
volume: 2.72093051
2014-01-02 18:55:00 (DEBUG):    Leftovers: 1434
2014-01-02 18:55:00 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 18:56:00 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 18:56:01 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:33:5
5 (UTC) to 2014-01-02 23:56:00 (UTC)
2014-01-02 18:56:01 (DEBUG):    minimum trade treshold: 2014-01-02 23:54:56 UTC
2014-01-02 18:56:01 (DEBUG):    processing 71 trade(s)
2014-01-02 18:56:01 (DEBUG):    inserting candle 1434 (2014-01-02 23:54:00 UTC)
volume: 21.227133229999993
2014-01-02 18:56:01 (DEBUG):    inserting candle 1435 (2014-01-02 23:55:00 UTC)
volume: 40.66658424
2014-01-02 18:56:01 (DEBUG):    Leftovers: 1436
2014-01-02 18:56:01 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 18:57:01 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 18:57:02 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:35:3
3 (UTC) to 2014-01-02 23:57:01 (UTC)
2014-01-02 18:57:02 (DEBUG):    minimum trade treshold: 2014-01-02 23:56:00 UTC
2014-01-02 18:57:02 (DEBUG):    processing 87 trade(s)
2014-01-02 18:57:02 (DEBUG):    inserting candle 1436 (2014-01-02 23:56:00 UTC)
volume: 31.415805940000002
2014-01-02 18:57:02 (DEBUG):    Leftovers: 1437
2014-01-02 18:57:02 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 18:58:02 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 18:58:03 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:57:5
8 (UTC) to 2014-01-02 23:58:00 (UTC)
2014-01-02 18:58:03 (DEBUG):    minimum trade treshold: 2014-01-02 23:57:01 UTC
2014-01-02 18:58:03 (DEBUG):    processing 1001 trade(s)
2014-01-02 18:58:03 (DEBUG):    inserting candle 1437 (2014-01-02 23:57:00 UTC)
volume: 14.856789460000234
2014-01-02 18:58:03 (DEBUG):    Leftovers: 1438
2014-01-02 18:58:03 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 18:59:03 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 18:59:07 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:57:5
8 (UTC) to 2014-01-02 23:59:05 (UTC)
2014-01-02 18:59:07 (DEBUG):    minimum trade treshold: 2014-01-02 23:58:00 UTC
2014-01-02 18:59:07 (DEBUG):    processing 39 trade(s)
2014-01-02 18:59:07 (DEBUG):    inserting candle 1438 (2014-01-02 23:58:00 UTC)
volume: 55.3791489
2014-01-02 18:59:07 (DEBUG):    Leftovers: 1439
2014-01-02 18:59:07 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:00:07 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 19:00:08 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:57:5
8 (UTC) to 2014-01-03 00:00:07 (UTC)
2014-01-02 19:00:08 (DEBUG):    minimum trade treshold: 2014-01-02 23:59:05 UTC
2014-01-02 19:00:08 (DEBUG):    processing 35 trade(s)
2014-01-02 19:00:08 (DEBUG):    This batch includes trades for a new day.
2014-01-02 19:00:08 (DEBUG):    Creating a new daily database for day 2014-01-03

2014-01-02 19:00:08 (DEBUG):    inserting candle 1439 (2014-01-02 23:59:00 UTC)
volume: 0.2918300999999999
2014-01-02 19:00:08 (DEBUG):    Leftovers: 0
2014-01-02 19:00:08 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:00:08 (WARN):     Found a corrupted database ( 2014-01-03 ), going
 to clean it up

c:\gekko\core\databaseManager.js:509
      throw 'Gekko expected the history to be complete, however it was not :('
      ^
Gekko expected the history to be complete, however it was not :(

[Thu 01/02/2014 19:00:08.73] gekko seems to have stopped, waiting 30 seconds bef
ore restarting
[Thu 01/02/2014 19:00:12.79] Press Ctrl + C now if you REALLY want to stop gekko

[Thu 01/02/2014 19:00:31.84] attempting to (re)start gekko...
[Thu 01/02/2014 19:00:35.89] launching node.js gekko bot NOW

2014-01-02 19:00:36 (INFO):     I'm gonna make you rich, Bud Fox.


2014-01-02 19:00:36 (INFO):     Starting to watch the market: CEX.io BTC/GHS
2014-01-02 19:00:36 (DEBUG):    Requested trade data from CEX.io ...

2014-01-02 19:00:37 (INFO):     Seting up a new price actor:
2014-01-02 19:00:37 (INFO):              Trade advisor
2014-01-02 19:00:37 (INFO):              Calculate trading advice based on the E
xponential Moving Average Crossovers

2014-01-02 19:00:38 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:57:5
8 (UTC) to 2014-01-03 00:00:37 (UTC)
2014-01-02 19:00:38 (INFO):     No history found, starting to build one now
2014-01-02 19:00:38 (INFO):     Expected to start giving advice in 16 hours (201
4-01-03 16:09:38 UTC)
2014-01-02 19:00:38 (DEBUG):    minimum trade treshold: 2014-01-02 00:00:00 UTC
2014-01-02 19:00:38 (DEBUG):    processing 1001 trade(s)
2014-01-02 19:00:38 (DEBUG):    This batch includes trades for a new day.
2014-01-02 19:00:38 (DEBUG):    Creating a new daily database for day 2014-01-03

2014-01-02 19:00:38 (DEBUG):    inserting candle 1437 (2014-01-02 23:57:00 UTC)
volume: 14.725603370000233
2014-01-02 19:00:38 (DEBUG):    inserting candle 1438 (2014-01-02 23:58:00 UTC)
volume: 55.3791489
2014-01-02 19:00:38 (DEBUG):    inserting candle 1439 (2014-01-02 23:59:00 UTC)
volume: 0.2918300999999999
2014-01-02 19:00:38 (DEBUG):    Leftovers: 0
2014-01-02 19:00:38 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:00:38 (WARN):     Found a corrupted database ( 2014-01-03 ), going
 to clean it up

c:\gekko\core\databaseManager.js:509
      throw 'Gekko expected the history to be complete, however it was not :('
      ^
Gekko expected the history to be complete, however it was not :(

[Thu 01/02/2014 19:00:38.92] gekko seems to have stopped, waiting 30 seconds bef
ore restarting
[Thu 01/02/2014 19:00:42.99] Press Ctrl + C now if you REALLY want to stop gekko

[Thu 01/02/2014 19:01:02.08] attempting to (re)start gekko...
[Thu 01/02/2014 19:01:06.14] launching node.js gekko bot NOW

2014-01-02 19:01:06 (INFO):     I'm gonna make you rich, Bud Fox.


2014-01-02 19:01:06 (INFO):     Starting to watch the market: CEX.io BTC/GHS
2014-01-02 19:01:06 (DEBUG):    Requested trade data from CEX.io ...

2014-01-02 19:01:07 (INFO):     Seting up a new price actor:
2014-01-02 19:01:07 (INFO):              Trade advisor
2014-01-02 19:01:07 (INFO):              Calculate trading advice based on the E
xponential Moving Average Crossovers

2014-01-02 19:01:08 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:57:5
8 (UTC) to 2014-01-03 00:01:08 (UTC)
2014-01-02 19:01:08 (INFO):     No history found, starting to build one now
2014-01-02 19:01:08 (INFO):     Expected to start giving advice in 16 hours (201
4-01-03 16:10:08 UTC)
2014-01-02 19:01:08 (DEBUG):    minimum trade treshold: 2014-01-02 00:00:00 UTC
2014-01-02 19:01:08 (DEBUG):    processing 1001 trade(s)
2014-01-02 19:01:08 (DEBUG):    This batch includes trades for a new day.
2014-01-02 19:01:09 (DEBUG):    Creating a new daily database for day 2014-01-03

2014-01-02 19:01:09 (DEBUG):    inserting candle 1437 (2014-01-02 23:57:00 UTC)
volume: 14.720807170000231
2014-01-02 19:01:09 (DEBUG):    inserting candle 0 (2014-01-03 00:00:00 UTC) vol
ume: 0.50354414
2014-01-02 19:01:09 (DEBUG):    Leftovers: 1
2014-01-02 19:01:09 (DEBUG):    inserting candle 1438 (2014-01-02 23:58:00 UTC)
volume: 55.3791489
2014-01-02 19:01:09 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:01:09 (DEBUG):    inserting candle 1439 (2014-01-02 23:59:00 UTC)
volume: 0.2918300999999999
2014-01-02 19:01:09 (DEBUG):    Leftovers: 1
2014-01-02 19:01:09 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:01:09 (WARN):     DOUBLE UNIQUE INSERT, this should never happen.
Please post details  here: https://github.com/askmike/gekko/issues/90 2014-01-02
 1437 { message: 'Can\'t insert key 1437, it violates the unique constraint',
  key: 1437,
  errorType: 'uniqueViolated' }

c:\gekko\core\databaseManager.js:485
      throw err;
            ^
[object Object]

[Thu 01/02/2014 19:01:09.27] gekko seems to have stopped, waiting 30 seconds bef
ore restarting
[Thu 01/02/2014 19:01:13.32] Press Ctrl + C now if you REALLY want to stop gekko

[Thu 01/02/2014 19:01:32.40] attempting to (re)start gekko...
[Thu 01/02/2014 19:01:36.46] launching node.js gekko bot NOW

2014-01-02 19:01:36 (INFO):     I'm gonna make you rich, Bud Fox.


2014-01-02 19:01:36 (INFO):     Starting to watch the market: CEX.io BTC/GHS
2014-01-02 19:01:36 (DEBUG):    Requested trade data from CEX.io ...

2014-01-02 19:01:37 (INFO):     Seting up a new price actor:
2014-01-02 19:01:37 (INFO):              Trade advisor
2014-01-02 19:01:37 (INFO):              Calculate trading advice based on the E
xponential Moving Average Crossovers

2014-01-02 19:01:38 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:57:5
8 (UTC) to 2014-01-03 00:01:36 (UTC)
2014-01-02 19:01:38 (WARN):     Found a corrupted database ( 2014-01-03 ), going
 to clean it up
2014-01-02 19:01:38 (DEBUG):    This should not happen, please post details here
: https://github.com/askmike/gekko/issues/90
2014-01-02 19:01:38 (INFO):     No history found, starting to build one now
2014-01-02 19:01:38 (INFO):     Expected to start giving advice in 16 hours (201
4-01-03 16:10:38 UTC)
2014-01-02 19:01:38 (DEBUG):    minimum trade treshold: 2014-01-02 00:00:00 UTC
2014-01-02 19:01:38 (DEBUG):    processing 1001 trade(s)
2014-01-02 19:01:39 (DEBUG):    This batch includes trades for a new day.
2014-01-02 19:01:39 (DEBUG):    Creating a new daily database for day 2014-01-03

2014-01-02 19:01:39 (DEBUG):    inserting candle 1437 (2014-01-02 23:57:00 UTC)
volume: 14.710214770000231
2014-01-02 19:01:39 (DEBUG):    inserting candle 0 (2014-01-03 00:00:00 UTC) vol
ume: 0.50354414
2014-01-02 19:01:39 (DEBUG):    Leftovers: 1
2014-01-02 19:01:39 (DEBUG):    inserting candle 1438 (2014-01-02 23:58:00 UTC)
volume: 55.3791489
2014-01-02 19:01:39 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:01:39 (DEBUG):    inserting candle 1439 (2014-01-02 23:59:00 UTC)
volume: 0.2918300999999999
2014-01-02 19:01:39 (DEBUG):    Leftovers: 1
2014-01-02 19:01:39 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:01:39 (WARN):     DOUBLE UNIQUE INSERT, this should never happen.
Please post details  here: https://github.com/askmike/gekko/issues/90 2014-01-02
 1437 { message: 'Can\'t insert key 1437, it violates the unique constraint',
  key: 1437,
  errorType: 'uniqueViolated' }

c:\gekko\core\databaseManager.js:485
      throw err;
            ^
[object Object]

[Thu 01/02/2014 19:01:39.71] gekko seems to have stopped, waiting 30 seconds bef
ore restarting
[Thu 01/02/2014 19:01:43.76] Press Ctrl + C now if you REALLY want to stop gekko

[Thu 01/02/2014 19:02:02.82] attempting to (re)start gekko...
[Thu 01/02/2014 19:02:06.87] launching node.js gekko bot NOW

2014-01-02 19:02:07 (INFO):     I'm gonna make you rich, Bud Fox.


2014-01-02 19:02:07 (INFO):     Starting to watch the market: CEX.io BTC/GHS
2014-01-02 19:02:07 (DEBUG):    Requested trade data from CEX.io ...

2014-01-02 19:02:08 (INFO):     Seting up a new price actor:
2014-01-02 19:02:08 (INFO):              Trade advisor
2014-01-02 19:02:08 (INFO):              Calculate trading advice based on the E
xponential Moving Average Crossovers

2014-01-02 19:02:09 (DEBUG):    Fetched 1001 new trades, from 2014-01-02 23:57:5
8 (UTC) to 2014-01-03 00:02:04 (UTC)
2014-01-02 19:02:09 (WARN):     Found a corrupted database ( 2014-01-03 ), going
 to clean it up
2014-01-02 19:02:09 (DEBUG):    This should not happen, please post details here
: https://github.com/askmike/gekko/issues/90
2014-01-02 19:02:09 (INFO):     No history found, starting to build one now
2014-01-02 19:02:09 (INFO):     Expected to start giving advice in 16 hours (201
4-01-03 16:11:09 UTC)
2014-01-02 19:02:09 (DEBUG):    minimum trade treshold: 2014-01-02 00:00:00 UTC
2014-01-02 19:02:09 (DEBUG):    processing 1001 trade(s)
2014-01-02 19:02:10 (DEBUG):    This batch includes trades for a new day.
2014-01-02 19:02:10 (DEBUG):    Creating a new daily database for day 2014-01-03

2014-01-02 19:02:10 (DEBUG):    inserting candle 1437 (2014-01-02 23:57:00 UTC)
volume: 14.683116240000231
2014-01-02 19:02:10 (DEBUG):    inserting candle 0 (2014-01-03 00:00:00 UTC) vol
ume: 0.50354414
2014-01-02 19:02:10 (DEBUG):    inserting candle 1438 (2014-01-02 23:58:00 UTC)
volume: 55.3791489
2014-01-02 19:02:10 (DEBUG):    inserting candle 1 (2014-01-03 00:01:00 UTC) vol
ume: 40.03484713999976
2014-01-02 19:02:10 (DEBUG):    Leftovers: 2
2014-01-02 19:02:10 (DEBUG):    inserting candle 1439 (2014-01-02 23:59:00 UTC)
volume: 0.2918300999999999
2014-01-02 19:02:10 (DEBUG):    Leftovers: 2
2014-01-02 19:02:10 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:02:10 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:02:10 (WARN):     DOUBLE UNIQUE INSERT, this should never happen.
Please post details  here: https://github.com/askmike/gekko/issues/90 2014-01-02
 1437 { message: 'Can\'t insert key 1437, it violates the unique constraint',
  key: 1437,
  errorType: 'uniqueViolated' }

c:\gekko\core\databaseManager.js:485
      throw err;
            ^
[object Object]

[Thu 01/02/2014 19:02:10.64] gekko seems to have stopped, waiting 30 seconds bef
ore restarting
[Thu 01/02/2014 19:02:14.68] Press Ctrl + C now if you REALLY want to stop gekko

[Thu 01/02/2014 19:02:33.74] attempting to (re)start gekko...
[Thu 01/02/2014 19:02:37.79] launching node.js gekko bot NOW

2014-01-02 19:02:38 (INFO):     I'm gonna make you rich, Bud Fox.


2014-01-02 19:02:38 (INFO):     Starting to watch the market: CEX.io BTC/GHS
2014-01-02 19:02:38 (DEBUG):    Requested trade data from CEX.io ...

2014-01-02 19:02:39 (INFO):     Seting up a new price actor:
2014-01-02 19:02:39 (INFO):              Trade advisor
2014-01-02 19:02:39 (INFO):              Calculate trading advice based on the E
xponential Moving Average Crossovers

2014-01-02 19:02:40 (DEBUG):    Fetched 1001 new trades, from 2014-01-03 00:01:5
9 (UTC) to 2014-01-03 00:02:36 (UTC)
2014-01-02 19:02:40 (WARN):     Found a corrupted database ( 2014-01-03 ), going
 to clean it up
2014-01-02 19:02:40 (DEBUG):    This should not happen, please post details here
: https://github.com/askmike/gekko/issues/90
2014-01-02 19:02:40 (INFO):     No history found, starting to build one now
2014-01-02 19:02:40 (INFO):     Expected to start giving advice in 16 hours (201
4-01-03 16:11:40 UTC)
2014-01-02 19:02:40 (DEBUG):    Creating a new daily database for day 2014-01-03

2014-01-02 19:02:40 (DEBUG):    minimum trade treshold: 2014-01-03 00:00:00 UTC
2014-01-02 19:02:40 (DEBUG):    processing 1001 trade(s)
2014-01-02 19:02:40 (DEBUG):    inserting candle 1 (2014-01-03 00:01:00 UTC) vol
ume: 0.014378399999999984
2014-01-02 19:02:40 (DEBUG):    Leftovers: 2
2014-01-02 19:02:40 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:03:40 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 19:03:41 (DEBUG):    Fetched 1001 new trades, from 2014-01-03 00:01:5
9 (UTC) to 2014-01-03 00:03:40 (UTC)
2014-01-02 19:03:41 (DEBUG):    minimum trade treshold: 2014-01-03 00:02:36 UTC
2014-01-02 19:03:41 (DEBUG):    processing 56 trade(s)
2014-01-02 19:03:41 (DEBUG):    inserting candle 2 (2014-01-03 00:02:00 UTC) vol
ume: 18.748692839999908
2014-01-02 19:03:41 (DEBUG):    Leftovers: 3
2014-01-02 19:03:41 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:04:41 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 19:04:43 (DEBUG):    Fetched 1001 new trades, from 2014-01-03 00:02:1
6 (UTC) to 2014-01-03 00:04:41 (UTC)
2014-01-02 19:04:43 (DEBUG):    minimum trade treshold: 2014-01-03 00:03:40 UTC
2014-01-02 19:04:43 (DEBUG):    processing 23 trade(s)
2014-01-02 19:04:43 (DEBUG):    inserting candle 3 (2014-01-03 00:03:00 UTC) vol
ume: 4.054342140000002
2014-01-02 19:04:43 (DEBUG):    Leftovers: 4
2014-01-02 19:04:43 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:05:43 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 19:05:44 (DEBUG):    Fetched 1001 new trades, from 2014-01-03 00:02:1
6 (UTC) to 2014-01-03 00:05:44 (UTC)
2014-01-02 19:05:44 (DEBUG):    minimum trade treshold: 2014-01-03 00:04:41 UTC
2014-01-02 19:05:44 (DEBUG):    processing 67 trade(s)
2014-01-02 19:05:44 (DEBUG):    inserting candle 4 (2014-01-03 00:04:00 UTC) vol
ume: 1.39061999
2014-01-02 19:05:44 (DEBUG):    Leftovers: 5
2014-01-02 19:05:44 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:06:44 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 19:06:45 (DEBUG):    Fetched 1001 new trades, from 2014-01-03 00:02:1
6 (UTC) to 2014-01-03 00:06:44 (UTC)
2014-01-02 19:06:45 (DEBUG):    minimum trade treshold: 2014-01-03 00:05:44 UTC
2014-01-02 19:06:45 (DEBUG):    processing 61 trade(s)
2014-01-02 19:06:45 (DEBUG):    inserting candle 5 (2014-01-03 00:05:00 UTC) vol
ume: 84.66667759000006
2014-01-02 19:06:45 (DEBUG):    Leftovers: 6
2014-01-02 19:06:45 (INFO):     Processed trades, sleeping for a minute ...
2014-01-02 19:07:45 (DEBUG):    Requested trade data from CEX.io ...
2014-01-02 19:07:47 (DEBUG):    Fetched 1001 new trades, from 2014-01-03 00:02:1
6 (UTC) to 2014-01-03 00:07:41 (UTC)
2014-01-02 19:07:47 (DEBUG):    minimum trade treshold: 2014-01-03 00:06:44 UTC
2014-01-02 19:07:47 (DEBUG):    processing 33 trade(s)
2014-01-02 19:07:47 (DEBUG):    inserting candle 6 (2014-01-03 00:06:00 UTC) vol
ume: 17.41847424
2014-01-02 19:07:47 (DEBUG):    Leftovers: 7
2014-01-02 19:07:47 (INFO):     Processed trades, sleeping for a minute ...