Skip to content

Instantly share code, notes, and snippets.

@barhoring
Created December 20, 2020 11:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save barhoring/cc6cbf7329c7efc92a46581b7a977dbb to your computer and use it in GitHub Desktop.
Save barhoring/cc6cbf7329c7efc92a46581b7a977dbb to your computer and use it in GitHub Desktop.
create promise array and use axios to wait for all to return
{/*
LIVE EXAMPLE:
https://repl.it/@BarHoring/multiple-http#index.js
*/}
const axios = require('axios');
const fs = require('fs');
const dependencies = [
"@babel/polyfill",
"@blueprintjs/select",
"@emotion/core",
"@fortawesome/fontawesome-svg-core",
"@fortawesome/free-brands-svg-icons",
"@fortawesome/free-regular-svg-icons",]
const baseUri = "https://www.npmjs.com/package/"
const getNpmURI = (dep) => {
return baseUri + dep
}
{/* will be filled with desired data */}
const result = Array(dependencies.length).fill({})
const linksArray = dependencies.map(dep => getNpmURI(dep))
{/* EXAMPLE:
linksArray =
[
'https://www.npmjs.com/package/@babel/polyfill',
'https://www.npmjs.com/package/@blueprintjs/select',
'https://www.npmjs.com/package/@emotion/core',
'https://www.npmjs.com/package/@fortawesome/fontawesome-svg-core',
'https://www.npmjs.com/package/@fortawesome/free-brands-svg-icons',
'https://www.npmjs.com/package/@fortawesome/free-regular-svg-icons'
]
*/}
const promisesArray = linksArray.map(link => axios.get(link))
const getLicense = (htmlString) => {
const preAnchor = `<h3 class="c84e15be f5 mt2 pt2 mb0 black-50">License</h3>`;
const anchorStart = `<p class="f2874b88 fw6 mb3 mt2 truncate black-80 f4">`;
const anchorEnd = `</p>`
const indexPreAnchor = htmlString.indexOf(preAnchor);
const indexStart = htmlString.indexOf(anchorStart, indexPreAnchor);
const indexEnd = htmlString.indexOf(anchorEnd, indexStart);
// EXAMPLE: substring ~ '<p class="f2874b88 fw6 mb3 mt2 truncate black-80 f4">MIT'
const markup = htmlString.substring(indexStart, indexEnd)
const license = markup.substring(markup.indexOf(">") + 1)
return license;
}
{/* wait for all to finish */}
axios.all(promisesArray).then(axios.spread((...responses) => {
for(let i = 0; i < responses.length; i++) {
const { status, statusText, headers, config, request, data } = responses[i];
const license = getLicense(data);
const name = dependencies[i]
const link = linksArray[i]
result[i] = {name, license, link }
// ERROR
if(status != 200) {
console.log("error!");
console.log(JSON.stringify(responses[0]))
}
}
})).catch(errors => {
// react on errors.
console.log(errors)
})
.finally(() => {
// save as a file
const data = JSON.stringify(result, null, 4)
console
fs.writeFileSync('./deps.txt', data);
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment