Skip to content

Instantly share code, notes, and snippets.

@heyitsritesh

heyitsritesh/Fix Google AdSense Crawl Errors for WordPress using Robots.txt file

Last active September 14, 2018 15:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save heyitsritesh/b64c6115bed1df1a03f293aa36270490 to your computer and use it in GitHub Desktop.
Save heyitsritesh/b64c6115bed1df1a03f293aa36270490 to your computer and use it in GitHub Desktop.
Download ZIP
Fix Google AdSense Crawl Errors for WordPress using Robots.txt file | https://www.digitalkube.com/fix-google-adsense-crawl-errors/
Raw
Fix Google AdSense Crawl Errors for WordPress using Robots.txt file
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-includes/
Disallow: /archives/
Disallow: /comments/feed/
Disallow: /trackback/
Disallow: /index.php
Disallow: /xmlrpc.php
Disallow: *?wptheme
Disallow: ?comments=*
Disallow: /search?
Disallow: /?p=*
Disallow: /feeds
User-agent: Mediapartners-Google*
Allow: /
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment