Skip to content

Instantly share code, notes, and snippets.

@NeuronQ
Created February 5, 2019 23:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save NeuronQ/0a362687ef24b95452745b9365d13af5 to your computer and use it in GitHub Desktop.
Save NeuronQ/0a362687ef24b95452745b9365d13af5 to your computer and use it in GitHub Desktop.
// async_scrape.js (tested with node 11.3)
const sleep = ts => new Promise(resolve => setTimeout(resolve, ts * 1000));
async function fetchUrl(url) {
console.log(`~ executing fetchUrl(${url})`);
console.time(`fetchUrl(${url})`);
await sleep(1 + Math.random() * 4);
console.timeEnd(`fetchUrl(${url})`);
return `<em>fake</em> page html for ${url}`;
}
async function analyzeSentiment(html) {
console.log(`~ analyzeSentiment("${html}")`);
console.time(`analyzeSentiment("${html}")`);
await sleep(1 + Math.random() * 4);
const r = {
positive: Math.random()
}
console.timeEnd(`analyzeSentiment("${html}")`);
return r;
}
// ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment