Skip to content

Instantly share code, notes, and snippets.

artforlife /
Created July 23, 2017 05:30 — forked from typehorror/
Simple Website Crawler (in python)

Simple Website Crawler

The following gist is an extract of the article Building a simple crawler. It allows crawling from a URL and for a given number of bounce.

Basic Usage

from crawler import Crawler
crawler = Crawler()

displays the urls