Rotten domain detector

gistfile1.py

#!/usr/bin/env python

import socket
import sys
import re

DOMAIN = re.compile("https?://([a-zA-Z0-9.-_]*\.[\w]{2,})")

dejavue = set()
for line in sys.stdin:
    domains = DOMAIN.findall(line)
    for domain in domains:
        if domain in dejavue:
            continue
        q = "%s.multi.surbl.org" % domain
        try:
            host = socket.gethostbyname(q)
        except socket.gaierror:
            pass
        else:
            dejavue.add(domain)
            print host, domain

Use it threw a pipe:

cat test.txt | ./rotten.py