Skip to content

Instantly share code, notes, and snippets.

@lukecav

lukecav/robots.txt

Created June 2, 2022 20:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lukecav/24503f61009c8569387fc2afe4278365 to your computer and use it in GitHub Desktop.
Save lukecav/24503f61009c8569387fc2afe4278365 to your computer and use it in GitHub Desktop.
Download ZIP
Disallow recommends for a robots.txt file for WordPress
Raw
robots.txt
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-admin/admin-ajax.php
Disallow: /wp-login.php
Disallow: /wp-json/
Disallow: /?s=
Disallow: /search/
@lukecav
Copy link
Author

lukecav commented Jun 2, 2022

https://developers.google.com/search/docs/advanced/robots/intro
https://www.cloudflare.com/learning/bots/what-is-robots.txt/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment