Python script for checking robots.txt's 'Disallow' property presence with urllib2

disallow.py

#!/usr/bin/python

import urllib2

class DisallowPresent(Exception):

	def __init__(self, path) :
		self.disallowed = path

	def __str__(self) :
		# repr is the same shit as str -> string representation on objects
		return repr(self.disallowed)

# urllib file
link = urllib2.urlopen('http://davidlares.now.sh/robots.txt')

for line in link.readlines():
	try:
		#checking for disallow
		if line.lower().find('disallow') != -1:
			print(line.strip())
			# raising custom error
			raise DisallowPresent(line.split(':')[1].strip())
	except DisallowPresent as e:
		print("Exeption occurred for path: %s" % e.disallowed) 

Since Gist does not handle indentation for raw python string very well. You'll need to check it previously for correct possible indentation errors on implementation.

Happy coding :)