Skip to content

Instantly share code, notes, and snippets.

@serhii73

serhii73/scrapy shell

Created June 7, 2017 07:09
Show Gist options
  • Save serhii73/6069201cc096d48b1e6080faa97b7eb6 to your computer and use it in GitHub Desktop.
Save serhii73/6069201cc096d48b1e6080faa97b7eb6 to your computer and use it in GitHub Desktop.
Download ZIP
scrapy shell and user-agent
Raw
scrapy shell
from scrapy import Request
req = Request("url.com", headers={"USER-AGENT" : "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 OPR/45.0.2552.888"})
fetch(req)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment