Skip to content

Instantly share code, notes, and snippets.

View zhanghongquan's full-sized avatar

ZhangHongquan zhanghongquan

View GitHub Profile
@jmoiron
jmoiron / crawler.py
Created May 27, 2011 20:37
Simple gevent/httplib2 web crawler.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Simple async crawler/callback queue based on gevent."""
import traceback
import logging
import httplib2
import gevent