Skip to content

Instantly share code, notes, and snippets.

@killertilapia

killertilapia/runner.py

Created July 15, 2019 05:46
Show Gist options
  • Save killertilapia/c959a072cbf342cbb3cb1924406e275a to your computer and use it in GitHub Desktop.
Save killertilapia/c959a072cbf342cbb3cb1924406e275a to your computer and use it in GitHub Desktop.
Download ZIP
To help debug scrapy scrapers in vscode
Raw
runner.py
# runner.py
# to help debug scrapy scrapers in vscode
from scrapy.cmdline import execute
execute(['scrapy', 'crawl', 'radiall_spider', '-o', 'test.json'])
# same thing just with extra params
from scrapy.cmdline import execute
execute(['scrapy', 'crawl', 'coilcraft', '-o', 'test.json', '-a', 'filepath_urls=parts_urls.json'])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment