Skip to content

Instantly share code, notes, and snippets.

@dontlaugh
Created August 4, 2020 19:42
Show Gist options
  • Save dontlaugh/0d4948754e7cf08239b72d3d027d5df5 to your computer and use it in GitHub Desktop.
Save dontlaugh/0d4948754e7cf08239b72d3d027d5df5 to your computer and use it in GitHub Desktop.
From 8/4/2020, 3:32:06 PM to 8/4/2020, 3:35:28 PM
2020-08-04T19:32:06.139381959Z
2020-08-04T19:32:06.139431798Z > novelcovid-api@1.0.1 start:scraper /home/container
2020-08-04T19:32:06.139436761Z > node serverScraper.js
2020-08-04T19:32:06.139439244Z
2020-08-04T19:32:06.85437932Z {
2020-08-04T19:32:06.854403359Z message: '[2020-08-04T19:32:06.851Z] ERROR: Failed to load environment variables',
2020-08-04T19:32:06.854407177Z error: "ENOENT: no such file or directory, open '/home/container/.env'",
2020-08-04T19:32:06.854409963Z stack: "Error: ENOENT: no such file or directory, open '/home/container/.env'\n" +
2020-08-04T19:32:06.854413012Z ' at Object.openSync (fs.js:465:3)\n' +
2020-08-04T19:32:06.854415464Z ' at Object.readFileSync (fs.js:368:35)\n' +
2020-08-04T19:32:06.854418118Z ' at Object.config (/home/container/node_modules/dotenv/lib/main.js:96:29)\n' +
2020-08-04T19:32:06.854420543Z ' at Object.<anonymous> (/home/container/config/index.js:3:34)\n' +
2020-08-04T19:32:06.854423492Z ' at Module._compile (internal/modules/cjs/loader.js:1256:30)\n' +
2020-08-04T19:32:06.854425916Z ' at Object.Module._extensions..js (internal/modules/cjs/loader.js:1277:10)\n' +
2020-08-04T19:32:06.854428296Z ' at Module.load (internal/modules/cjs/loader.js:1105:32)\n' +
2020-08-04T19:32:06.854430607Z ' at Function.Module._load (internal/modules/cjs/loader.js:967:14)\n' +
2020-08-04T19:32:06.854433018Z ' at Module.require (internal/modules/cjs/loader.js:1145:19)\n' +
2020-08-04T19:32:06.854435374Z ' at require (internal/modules/cjs/helpers.js:75:18)\n' +
2020-08-04T19:32:06.854437674Z ' at Object.<anonymous> (/home/container/routes/instances.js:17:32)\n' +
2020-08-04T19:32:06.854441499Z ' at Module._compile (internal/modules/cjs/loader.js:1256:30)\n' +
2020-08-04T19:32:06.85444394Z ' at Object.Module._extensions..js (internal/modules/cjs/loader.js:1277:10)\n' +
2020-08-04T19:32:06.854446346Z ' at Module.load (internal/modules/cjs/loader.js:1105:32)\n' +
2020-08-04T19:32:06.854448659Z ' at Function.Module._load (internal/modules/cjs/loader.js:967:14)\n' +
2020-08-04T19:32:06.854451006Z ' at Module.require (internal/modules/cjs/loader.js:1145:19)'
2020-08-04T19:32:06.854453348Z }
2020-08-04T19:32:06.854896068Z [2020-08-04T19:32:06.854Z]: Using default settings
2020-08-04T19:32:06.875325061Z [2020-08-04T19:32:06.875Z]: USING 08-03-2020.csv CSSEGISandData
2020-08-04T19:32:07.648079903Z [2020-08-04T19:32:07.647Z]: Updated states: 63 states
2020-08-04T19:32:07.734626776Z [2020-08-04T19:32:07.734Z]: Updated yesterday states: 63 states
2020-08-04T19:32:08.950363524Z [2020-08-04T19:32:08.950Z]: Updated influenza:ILINET data: 43 weeks
2020-08-04T19:32:10.771136005Z [2020-08-04T19:32:10.770Z]: Updated today countries statistics: 216
2020-08-04T19:32:10.77126017Z [2020-08-04T19:32:10.771Z]: Updated today continents statistics: 6
2020-08-04T19:32:11.445573814Z [2020-08-04T19:32:11.445Z]: Updated yesterday countries statistics: 216
2020-08-04T19:32:11.445690044Z [2020-08-04T19:32:11.445Z]: Updated yesterday continents statistics: 6
2020-08-04T19:32:11.857049009Z [2020-08-04T19:32:11.856Z]: Updated yesterday2 countries statistics: 216
2020-08-04T19:32:11.857139742Z [2020-08-04T19:32:11.857Z]: Updated yesterday2 continents statistics: 6
2020-08-04T19:32:12.138264049Z [2020-08-04T19:32:12.138Z]: Updated JHU CSSE: 3935 locations
2020-08-04T19:32:12.236747105Z [2020-08-04T19:32:12.236Z]: Updated influenza:USPHL data: 43 weeks
2020-08-04T19:32:19.244831167Z [2020-08-04T19:32:19.244Z]: Finished Vaccine scraping!
2020-08-04T19:32:22.87626394Z [2020-08-04T19:32:22.876Z]: Apple Data successfully retrieved
2020-08-04T19:32:23.145741602Z [2020-08-04T19:32:23.145Z]: Updated influenza:USCL data: 43 weeks
2020-08-04T19:32:23.145773427Z [2020-08-04T19:32:23.145Z]: Finished CDC Influenza scraping!
2020-08-04T19:32:45.077929709Z [2020-08-04T19:32:45.077Z]: Updated JHU CSSE Historical USA: 3340 locations
2020-08-04T19:32:45.139035167Z {
2020-08-04T19:32:45.139069905Z message: '[2020-08-04T19:32:45.136Z] ERROR: Error: Requesting Austria Gov Data failed!',
2020-08-04T19:32:45.139073346Z error: 'connect ECONNRESET 194.48.237.240:443',
2020-08-04T19:32:45.139075826Z stack: 'Error: connect ECONNRESET 194.48.237.240:443\n' +
2020-08-04T19:32:45.139078366Z ' at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1144:16)'
2020-08-04T19:32:45.139080842Z }
2020-08-04T19:32:47.053351133Z [2020-08-04T19:32:47.053Z]: NYT Data successfully retrieved
2020-08-04T19:32:47.568536668Z [2020-08-04T19:32:47.568Z]: Updated JHU CSSE Historical: 266 locations
2020-08-04T19:32:47.568579745Z [2020-08-04T19:32:47.568Z]: Finished JHU and worldometers scraping!
2020-08-04T19:33:01.238737785Z
2020-08-04T19:33:01.238761369Z <--- Last few GCs --->
2020-08-04T19:33:01.238765953Z a[24:0x556c9b315c20] 53210 ms: Mark-sweep (reduce) 513.5 (515.2) -> 513.5 (517.2) MB, 815.7 / 0.0 ms (+ 11.6 ms in 16 steps since start of marking, biggest step 9.8 ms, walltime since start of marking 908 ms) (average mu = 0.136, current mu = 0.089) al[24:0x556c9b315c20] 54184 ms: Mark-sweep (reduce) 514.6 (519.2) -> 514.6 (520.2) MB, 690.6 / 0.0 ms (+ 186.3 ms in 16 steps since start of marking, biggest step 184.3 ms, walltime since start of marking 974 ms) (average mu = 0.118, current mu = 0.100)
2020-08-04T19:33:01.238773727Z
2020-08-04T19:33:01.238776859Z <--- JS stacktrace --->
2020-08-04T19:33:01.238780185Z
2020-08-04T19:33:01.238783147Z FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
2020-08-04T19:33:22.146192077Z
2020-08-04T19:33:22.146234077Z > novelcovid-api@1.0.1 start:scraper /home/container
2020-08-04T19:33:22.146240111Z > node serverScraper.js
2020-08-04T19:33:22.146243566Z
2020-08-04T19:33:22.954865865Z {
2020-08-04T19:33:22.954881653Z message: '[2020-08-04T19:33:22.952Z] ERROR: Failed to load environment variables',
2020-08-04T19:33:22.954885889Z error: "ENOENT: no such file or directory, open '/home/container/.env'",
2020-08-04T19:33:22.95488973Z stack: "Error: ENOENT: no such file or directory, open '/home/container/.env'\n" +
2020-08-04T19:33:22.9548939Z ' at Object.openSync (fs.js:465:3)\n' +
2020-08-04T19:33:22.954896469Z ' at Object.readFileSync (fs.js:368:35)\n' +
2020-08-04T19:33:22.954899072Z ' at Object.config (/home/container/node_modules/dotenv/lib/main.js:96:29)\n' +
2020-08-04T19:33:22.954901464Z ' at Object.<anonymous> (/home/container/config/index.js:3:34)\n' +
2020-08-04T19:33:22.954904139Z ' at Module._compile (internal/modules/cjs/loader.js:1256:30)\n' +
2020-08-04T19:33:22.954906522Z ' at Object.Module._extensions..js (internal/modules/cjs/loader.js:1277:10)\n' +
2020-08-04T19:33:22.954908848Z ' at Module.load (internal/modules/cjs/loader.js:1105:32)\n' +
2020-08-04T19:33:22.954911178Z ' at Function.Module._load (internal/modules/cjs/loader.js:967:14)\n' +
2020-08-04T19:33:22.95491348Z ' at Module.require (internal/modules/cjs/loader.js:1145:19)\n' +
2020-08-04T19:33:22.954915747Z ' at require (internal/modules/cjs/helpers.js:75:18)\n' +
2020-08-04T19:33:22.954918019Z ' at Object.<anonymous> (/home/container/routes/instances.js:17:32)\n' +
2020-08-04T19:33:22.954921604Z ' at Module._compile (internal/modules/cjs/loader.js:1256:30)\n' +
2020-08-04T19:33:22.954923998Z ' at Object.Module._extensions..js (internal/modules/cjs/loader.js:1277:10)\n' +
2020-08-04T19:33:22.954926288Z ' at Module.load (internal/modules/cjs/loader.js:1105:32)\n' +
2020-08-04T19:33:22.954928562Z ' at Function.Module._load (internal/modules/cjs/loader.js:967:14)\n' +
2020-08-04T19:33:22.954930866Z ' at Module.require (internal/modules/cjs/loader.js:1145:19)'
2020-08-04T19:33:22.954933172Z }
2020-08-04T19:33:22.955395009Z [2020-08-04T19:33:22.955Z]: Using default settings
2020-08-04T19:33:22.975839823Z [2020-08-04T19:33:22.975Z]: USING 08-03-2020.csv CSSEGISandData
2020-08-04T19:33:23.63793194Z [2020-08-04T19:33:23.637Z]: Updated states: 63 states
2020-08-04T19:33:23.643622464Z [2020-08-04T19:33:23.643Z]: Updated yesterday states: 63 states
2020-08-04T19:33:25.035780812Z [2020-08-04T19:33:25.035Z]: Updated influenza:ILINET data: 43 weeks
2020-08-04T19:33:26.935528671Z [2020-08-04T19:33:26.935Z]: Updated today countries statistics: 216
2020-08-04T19:33:26.935710753Z [2020-08-04T19:33:26.935Z]: Updated today continents statistics: 6
2020-08-04T19:33:27.36479322Z [2020-08-04T19:33:27.364Z]: Updated yesterday countries statistics: 216
2020-08-04T19:33:27.364885199Z [2020-08-04T19:33:27.364Z]: Updated yesterday continents statistics: 6
2020-08-04T19:33:27.942806478Z [2020-08-04T19:33:27.942Z]: Updated yesterday2 countries statistics: 216
2020-08-04T19:33:27.942899223Z [2020-08-04T19:33:27.942Z]: Updated yesterday2 continents statistics: 6
2020-08-04T19:33:28.356502468Z [2020-08-04T19:33:28.356Z]: Updated JHU CSSE: 3935 locations
2020-08-04T19:33:28.466651394Z [2020-08-04T19:33:28.466Z]: Updated influenza:USPHL data: 43 weeks
2020-08-04T19:33:39.1397522Z [2020-08-04T19:33:39.139Z]: Apple Data successfully retrieved
2020-08-04T19:33:59.047800969Z [2020-08-04T19:33:59.047Z]: Updated influenza:USCL data: 43 weeks
2020-08-04T19:33:59.047858829Z [2020-08-04T19:33:59.047Z]: Finished CDC Influenza scraping!
2020-08-04T19:33:59.051605443Z {
2020-08-04T19:33:59.051618999Z message: '[2020-08-04T19:33:59.050Z] ERROR: Error: Requesting Austria Gov Data failed!',
2020-08-04T19:33:59.051622389Z error: 'write ECONNRESET',
2020-08-04T19:33:59.051624952Z stack: 'Error: write ECONNRESET\n' +
2020-08-04T19:33:59.051627518Z ' at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16)'
2020-08-04T19:33:59.051630078Z }
2020-08-04T19:34:01.277565174Z [2020-08-04T19:34:01.277Z]: Finished Vaccine scraping!
2020-08-04T19:34:04.036899928Z [2020-08-04T19:34:04.036Z]: Updated JHU CSSE Historical USA: 3340 locations
2020-08-04T19:34:04.036929337Z [2020-08-04T19:34:04.036Z]: NYT Data successfully retrieved
2020-08-04T19:34:04.444333303Z [2020-08-04T19:34:04.444Z]: Updated JHU CSSE Historical: 266 locations
2020-08-04T19:34:04.44438671Z [2020-08-04T19:34:04.444Z]: Finished JHU and worldometers scraping!
2020-08-04T19:34:14.435994749Z
2020-08-04T19:34:14.436028265Z <--- Last few GCs --->
2020-08-04T19:34:14.436034575Z ll[24:0x5624c8d3ec20] 51087 ms: Mark-sweep (reduce) 513.8 (515.7) -> 513.8 (517.7) MB, 595.1 / 0.0 ms (+ 6.2 ms in 16 steps since start of marking, biggest step 4.7 ms, walltime since start of marking 670 ms) (average mu = 0.152, current mu = 0.103) all[24:0x5624c8d3ec20] 51689 ms: Mark-sweep (reduce) 514.8 (519.7) -> 514.8 (520.7) MB, 509.9 / 0.0 ms (+ 10.7 ms in 16 steps since start of marking, biggest step 8.8 ms, walltime since start of marking 602 ms) (average mu = 0.144, current mu = 0.135) al
2020-08-04T19:34:14.436044578Z
2020-08-04T19:34:14.436046948Z <--- JS stacktrace --->
2020-08-04T19:34:14.436049386Z
2020-08-04T19:34:14.436051678Z FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
2020-08-04T19:34:47.148828899Z
2020-08-04T19:34:47.148863926Z > novelcovid-api@1.0.1 start:scraper /home/container
2020-08-04T19:34:47.148867832Z > node serverScraper.js
2020-08-04T19:34:47.148870288Z
2020-08-04T19:34:48.049242553Z {
2020-08-04T19:34:48.049267456Z message: '[2020-08-04T19:34:48.045Z] ERROR: Failed to load environment variables',
2020-08-04T19:34:48.049272049Z error: "ENOENT: no such file or directory, open '/home/container/.env'",
2020-08-04T19:34:48.049275718Z stack: "Error: ENOENT: no such file or directory, open '/home/container/.env'\n" +
2020-08-04T19:34:48.049278551Z ' at Object.openSync (fs.js:465:3)\n' +
2020-08-04T19:34:48.049281002Z ' at Object.readFileSync (fs.js:368:35)\n' +
2020-08-04T19:34:48.049283306Z ' at Object.config (/home/container/node_modules/dotenv/lib/main.js:96:29)\n' +
2020-08-04T19:34:48.049285676Z ' at Object.<anonymous> (/home/container/config/index.js:3:34)\n' +
2020-08-04T19:34:48.04928834Z ' at Module._compile (internal/modules/cjs/loader.js:1256:30)\n' +
2020-08-04T19:34:48.049290816Z ' at Object.Module._extensions..js (internal/modules/cjs/loader.js:1277:10)\n' +
2020-08-04T19:34:48.049293175Z ' at Module.load (internal/modules/cjs/loader.js:1105:32)\n' +
2020-08-04T19:34:48.049295656Z ' at Function.Module._load (internal/modules/cjs/loader.js:967:14)\n' +
2020-08-04T19:34:48.049297952Z ' at Module.require (internal/modules/cjs/loader.js:1145:19)\n' +
2020-08-04T19:34:48.049300209Z ' at require (internal/modules/cjs/helpers.js:75:18)\n' +
2020-08-04T19:34:48.049302473Z ' at Object.<anonymous> (/home/container/routes/instances.js:17:32)\n' +
2020-08-04T19:34:48.049311694Z ' at Module._compile (internal/modules/cjs/loader.js:1256:30)\n' +
2020-08-04T19:34:48.049314163Z ' at Object.Module._extensions..js (internal/modules/cjs/loader.js:1277:10)\n' +
2020-08-04T19:34:48.049317657Z ' at Module.load (internal/modules/cjs/loader.js:1105:32)\n' +
2020-08-04T19:34:48.049320051Z ' at Function.Module._load (internal/modules/cjs/loader.js:967:14)\n' +
2020-08-04T19:34:48.049322545Z ' at Module.require (internal/modules/cjs/loader.js:1145:19)'
2020-08-04T19:34:48.049324952Z }
2020-08-04T19:34:48.049965952Z [2020-08-04T19:34:48.049Z]: Using default settings
2020-08-04T19:34:48.135612741Z [2020-08-04T19:34:48.135Z]: USING 08-03-2020.csv CSSEGISandData
2020-08-04T19:34:49.649492311Z [2020-08-04T19:34:49.649Z]: Updated states: 63 states
2020-08-04T19:34:49.734217331Z [2020-08-04T19:34:49.734Z]: Updated yesterday states: 63 states
2020-08-04T19:34:50.250144085Z [2020-08-04T19:34:50.249Z]: Updated influenza:ILINET data: 43 weeks
2020-08-04T19:34:52.152118436Z [2020-08-04T19:34:52.151Z]: Updated today countries statistics: 216
2020-08-04T19:34:52.152245204Z [2020-08-04T19:34:52.152Z]: Updated today continents statistics: 6
2020-08-04T19:34:52.641380671Z [2020-08-04T19:34:52.641Z]: Updated yesterday countries statistics: 216
2020-08-04T19:34:52.641663115Z [2020-08-04T19:34:52.641Z]: Updated yesterday continents statistics: 6
2020-08-04T19:34:53.241223553Z [2020-08-04T19:34:53.241Z]: Updated yesterday2 countries statistics: 216
2020-08-04T19:34:53.241349286Z [2020-08-04T19:34:53.241Z]: Updated yesterday2 continents statistics: 6
2020-08-04T19:34:53.451043165Z [2020-08-04T19:34:53.450Z]: Updated JHU CSSE: 3935 locations
2020-08-04T19:34:53.567644642Z [2020-08-04T19:34:53.567Z]: Updated influenza:USPHL data: 43 weeks
2020-08-04T19:35:14.158055691Z [2020-08-04T19:35:14.157Z]: Finished Vaccine scraping!
2020-08-04T19:35:14.239941158Z [2020-08-04T19:35:14.239Z]: Updated influenza:USCL data: 43 weeks
2020-08-04T19:35:14.239960966Z [2020-08-04T19:35:14.239Z]: Finished CDC Influenza scraping!
2020-08-04T19:35:14.269173079Z {
2020-08-04T19:35:14.269202004Z message: '[2020-08-04T19:35:14.267Z] ERROR: Error: Requesting Austria Gov Data failed!',
2020-08-04T19:35:14.269206216Z error: 'write ECONNRESET',
2020-08-04T19:35:14.269209555Z stack: 'Error: write ECONNRESET\n' +
2020-08-04T19:35:14.269212368Z ' at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16)'
2020-08-04T19:35:14.269215065Z }
2020-08-04T19:35:15.949322936Z [2020-08-04T19:35:15.949Z]: NYT Data successfully retrieved
2020-08-04T19:35:24.145565527Z [2020-08-04T19:35:24.144Z]: Apple Data successfully retrieved
2020-08-04T19:35:26.170152998Z [2020-08-04T19:35:26.170Z]: Updated JHU CSSE Historical: 266 locations
2020-08-04T19:35:28.351663237Z [2020-08-04T19:35:28.351Z]: Updated JHU CSSE Historical USA: 3340 locations
2020-08-04T19:35:28.35169472Z [2020-08-04T19:35:28.351Z]: Finished JHU and worldometers scraping!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment