Skip to content

Instantly share code, notes, and snippets.

@velotiotech

velotiotech/custom_spider.sh

Last active July 3, 2020 15:23
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save velotiotech/c40bf01c94a7558eb5dec156e5752b17 to your computer and use it in GitHub Desktop.
Save velotiotech/c40bf01c94a7558eb5dec156e5752b17 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
custom_spider.sh
$cat > myspider.py <import scrapy
class BlogSpider(scrapy.Spider):
name = 'blogspider'
start_urls = ['https://blog.scrapinghub.com']
def parse(self, response):
for title in response.css('h2.entry-title'):
yield {'title': title.css('a ::text').extract_first()}
EOF
scrapy runspider myspider.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment