Skip to content

Instantly share code, notes, and snippets.

@matheus-rossi
Created October 1, 2017 23:32
Show Gist options
  • Save matheus-rossi/bc4c688264be072ded4ff7ee3f933bc2 to your computer and use it in GitHub Desktop.
Save matheus-rossi/bc4c688264be072ded4ff7ee3f933bc2 to your computer and use it in GitHub Desktop.
node-horseman crawl
var Horseman = require('node-horseman');
var horseman = new Horseman();
horseman
.open('http://www.angeloni.com.br/super/index')
.status()
.evaluate(function(){
const descNode = document.querySelectorAll('.descr a')
const desc = Array.prototype.map.call(descNode, function (t) { return t.textContent })
const valueNode = document.querySelectorAll('.price a')
const value = Array.prototype.map.call(valueNode, function (t) { return t.textContent })
const finalData = []
for (let i=0; i < desc.length; i ++) {
let item = {}
item['desc'] = desc[i]
item['value'] = value[i]
finalData.push(item)
}
return finalData
})
.then(function(result){
console.log(result)
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment