Skip to content

Instantly share code, notes, and snippets.

@whalebot-helmsman

whalebot-helmsman/redirector.py

Created March 12, 2019 07:38
Show Gist options
  • Save whalebot-helmsman/4d6a8aac5d42ba6b842feb00030e3037 to your computer and use it in GitHub Desktop.
Save whalebot-helmsman/4d6a8aac5d42ba6b842feb00030e3037 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
redirector.py
import scrapy
class Redirect(scrapy.Spider):
name = 'redirect'
start_urls = ['http://www.wikipedia.net/wiki/Hello']
def parse(self, response):
self.logger.info(response.meta.get('redirect_urls', []))
self.logger.info(response.url)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment