Skip to content

Instantly share code, notes, and snippets.


eparikh/ Secret

Created Feb 19, 2017
What would you like to do?
A sample of my Wikipedia TV show scraper
class TVSpider(Spider):
name = "tv_spider"
allowed_urls = [""]
start_urls = [""]
def parse(self, response):
# TV show URLs scraped from the list of TV shows
urls = response.xpath("//i//a/@href").extract()
# follow URL to get show details
for url in urls:
yield (scrapy.Request(response.urljoin(url),
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.