Skip to content

Instantly share code, notes, and snippets.

@ThibaudLamothe
Last active July 12, 2022 12:25
Show Gist options
  • Save ThibaudLamothe/f13ad71183a124216c0455ce75767f9b to your computer and use it in GitHub Desktop.
Save ThibaudLamothe/f13ad71183a124216c0455ce75767f9b to your computer and use it in GitHub Desktop.
import requests
from scrapy.selector import Selector
# Prepare url
city = 'Toronto'
main_url = 'https://www.airbnb.com'
city_url = f'{main_url}/s/{city}/homes/'
# Create selector
html = requests.get(city_url).content
sel = Selector(text=html)
# Get hotels
hotels = sel.css('div._8ssblpx')
print('Number of hotels:', len(hotels))
# Find the next page url
next_page = sel.css('a._za9j7e ::attr(href)').extract_first()
print('Next page:', '\n', next_page)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment