Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lucasgautheron/3de308787a7ef52976e9792971009779 to your computer and use it in GitHub Desktop.
Save lucasgautheron/3de308787a7ef52976e9792971009779 to your computer and use it in GitHub Desktop.
import { parse } from 'node-html-parser';
class dataFetcher {
constructor() {
// initialize cURL
this.curl = new (require( 'curl-request' ))();
curl.setHeaders([
'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36'
]);
//
this.entries = [];
}
fetchPage(date, pageNumber) {
}
fetch(from, to) {
for (date = from; date <= to; date.setDate(date.getDate() + 1)) {
let i = 1;
while(true) {
let moreEntries = fetchPage(date, i);
if (moreEntries) {
this.entries += moreEntries
} else {
break;
}
i++;
}
}
}
}
let fetcher = new dataFetcher();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment