Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
// Phantombuster configuration {
"phantombuster command: casperjs"
"phantombuster package: 3"
"phantombuster transform: babel"
"phantombuster flags: save-folder"
import "babel-polyfill"
import Buster from "phantombuster"
const buster = new Buster()
import Nick from "nickjs"
const nick = new Nick()
// }
/*
This demo script returns two things:
1) JSON of all the Hacker News homepage links
2) Screenshot of the Hacker News homepage
Edit it as you wish, have a look at our documentation on https://hub.phantombuster.com/ and do get in touch!
*/
nick.newTab().then(async (tab) => {
await tab.open("news.ycombinator.com")
await tab.untilVisible("#hnmain") // We make sure we have loaded the page
await tab.inject("../injectables/jquery-3.0.0.min.js") // We're going to use jQuery to scrape
// Evaluate a function in the current page DOM context. Execution is sandboxed: page has no access to the Nick context
// In other words: Open the browser inspector to execute this function in the console
const hackerNewsLinks = await tab.evaluate((arg, callback) => {
const data = []
$(".athing").each((index, element) => {
data.push({
title: $(element).find(".storylink").text(),
url: $(element).find(".storylink").attr("href")
})
})
callback(null, data)
})
await buster.setResultObject(hackerNewsLinks) // Send the result back to Phantombuster
await tab.screenshot("hacker-news.jpg") // Why not take a screenshot while we're at it?
})
.then(() => {
console.log("Job done!")
nick.exit()
})
.catch((err) => {
console.log(`Something went wrong: ${err}`)
nick.exit(1)
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.