Skip to content

Instantly share code, notes, and snippets.

View jnario's full-sized avatar
🦉
Intermittently Available

Jose Nario jnario

🦉
Intermittently Available
View GitHub Profile
@jnario
jnario / 0readme.md
Created February 7, 2018 17:48 — forked from mdamien/0readme.md
404 link detector with scrapy

List all the broken links on your website

Requirements:

python3 and scrapy (pip install scrapy)

Usage

  • scrapy runspider -o items.csv -a site="https://yoursite.org" 1spider.py
  • python3 2format_results.py