Skip to content

Instantly share code, notes, and snippets.

@ThibaudLamothe
Created February 1, 2021 10:43
Show Gist options
  • Save ThibaudLamothe/48802824b5e0107cfe8958244c69c558 to your computer and use it in GitHub Desktop.
Save ThibaudLamothe/48802824b5e0107cfe8958244c69c558 to your computer and use it in GitHub Desktop.
def parse(self, response):
# Getting the hotels list
hotels = response.css('hotels_selector')
# Following hotels pages
for hotel in hotels:
info_1 = hotel.css('info_1_selector')
info_2 = hotel.css('info_2_selector')
yield {
"info_1":info_1,
"info_2":info_2
}
# Get Next Page of hotels
next_page = response.css('next_page_selector').extract()
yield scrapy.Request(url=url, callback=self.parse)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment