Skip to content

Instantly share code, notes, and snippets.

@suissa
Created February 13, 2017 21:07
Show Gist options
  • Save suissa/3dc15d97981d93edd5f23ac879a9358b to your computer and use it in GitHub Desktop.
Save suissa/3dc15d97981d93edd5f23ac879a9358b to your computer and use it in GitHub Desktop.
Pega e salva os conteúdos da MDN
const cheerio = require('cheerio')
const rp = require('request-promise')
const links = require('./links')
const createFile = require('./createFile')
const options = {
uri: `https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/`,
transform: (body) => cheerio.load(body),
}
const BASE = `https://developer.mozilla.org/`
rp(options)
.then( ($) => {
$(`#wikiArticle ul li a`)
.map(( index, element ) => element.attribs.href )
.get() //
.map( ( el, i ) => {
const _name = el.split('Reference/')[1]
const name = (_name) ? _name.replace('/', '_') : i
const link = BASE+el
return { name, link }
})
.map( content =>
rp( {
uri: content.link,
transform: (body) => cheerio.load(body),
})
.then( ($) =>
createFile(`conteudos/${content.name}.html`,
$(`#wikiArticle`).html()) )
.catch( (err) =>
console.log('err no request', err) )
)
}
)
.catch( (err) => console.log('err no request', err) )
// } )
@suissa
Copy link
Author

suissa commented Feb 13, 2017

const fs = require('fs')

module.exports = (PATH, data) => fs.writeFile(PATH, data, (err) =>
(err)
? console.log('err em createFile', err)
: console.log('\n Salvei o conteúdo em : ', PATH))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment