ryarwood/robots.txt

Created December 17, 2012 18:39

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/ryarwood/4320747.js"></script>
Save ryarwood/4320747 to your computer and use it in GitHub Desktop.

Download ZIP

Simple Robots.txt

Raw

	User-agent: *
	Disallow: /
	Disallow: /directory/
	Disallow: /directory/file.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment