Skip to content

Instantly share code, notes, and snippets.

@jancurn
Created September 4, 2019 06:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jancurn/40d04eeff567db2f637bd6b74bd1bed4 to your computer and use it in GitHub Desktop.
Save jancurn/40d04eeff567db2f637bd6b74bd1bed4 to your computer and use it in GitHub Desktop.
Scrapy Executor code example
import scrapy
import apify
class MySpider(scrapy.Spider):
name = 'apifySpider'
def start_requests(self):
urls = [
'https://apify.com',
'https://apify.com/store',
]
for url in urls:
yield scrapy.Request(url=url, callback=self.parse)
def parse(self, response):
url = response.url
title = response.css('title::text').get()
output = {
'url': url,
'title': title
}
apify.pushData(output)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment