Skip to content

Instantly share code, notes, and snippets.

@shubham-singh-ss

shubham-singh-ss/scrapy.py

Last active July 3, 2019 12:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shubham-singh-ss/5477d719c92d6e7eb97e32ac91f7029b to your computer and use it in GitHub Desktop.
Save shubham-singh-ss/5477d719c92d6e7eb97e32ac91f7029b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
scrapy.py
import scrapy
class Spider(scrapy.Spider):
name = 'NAME'
start_urls = ['LINK']
def parse(self, response):
for title in response.css('.post-header>h2'):
yield {'title': title.css('a ::text').get()}
for next_page in response.css('a.next-posts-link'):
yield response.follow(next_page, self.parse)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment