Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
User ipdb to debug a running spider in scrapyd
# The scrapyd daemon adds a lot of power and flexibility to scrapy, but it also adds a layer of indirection
# (another process) that can make debugging a running spider troublesome. Use this code to launch scrapyd
# in ipdb (or pdb). You'll be able to breakpoint and catch exceptions inside the spiders it launches.
import scrapyd.runner
import ipdb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.