awavering/Wordpress_Recommended_Robots.txt

Created January 10, 2019 20:26

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/awavering/c0e067ff310027e83b820d5e33ad1130.js"></script>
Save awavering/c0e067ff310027e83b820d5e33ad1130 to your computer and use it in GitHub Desktop.

Download ZIP

For details on why this is so minimal, see the Yoast blog post at https://yoast.com/wordpress-robots-txt-example/

Raw

Wordpress_Recommended_Robots.txt

	# This space intentionally left blank
	User-Agent: *
	# Exclude directories which generate new pages based on parameters (site search, for example)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment