Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
[获取scrapy所有spider name] #python #scrapy
from scrapy import spiderloader
from scrapy.utils import project
def get_all_spider_names() -> typing.List[str]:
os.environ.setdefault('SCRAPY_SETTINGS_MODULE', 'darkweb.settings')
settings = project.get_project_settings()
spider_loader = spiderloader.SpiderLoader.from_settings(settings)
return spider_loader.list()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment