Skip to content

Instantly share code, notes, and snippets.

@shubham-singh-ss
Last active July 3, 2019 12:45
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Embed
What would you like to do?
import scrapy
class Spider(scrapy.Spider):
name = 'NAME'
start_urls = ['LINK']
def parse(self, response):
for title in response.css('.post-header>h2'):
yield {'title': title.css('a ::text').get()}
for next_page in response.css('a.next-posts-link'):
yield response.follow(next_page, self.parse)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment