Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save vladimirlukyanov/247f9417b4b53ae1e1c447d40a560da6 to your computer and use it in GitHub Desktop.
Save vladimirlukyanov/247f9417b4b53ae1e1c447d40a560da6 to your computer and use it in GitHub Desktop.
Fix_Internal_Site_Search_Spam__robots+meta.txt
User-agent: *
Disallow: /wp-admin/*
Disallow: /wp-login.php
Disallow: /wp-content/themes/*
Disallow: /wp-content/plugins/*
Disallow: /trackback
Disallow: /cgi-bin
Disallow: /users/
Disallow: */trackback
Disallow: */rss
Disallow: */embed
Disallow: /xmlrpc.php
Disallow: *utm=
Disallow: *openstat=
Disallow: /readme.html
Disallow: *?replytocom
# This is comments ( do NOT include )
# STEP 1 :
# Remove those two lines, if you have such in robots.txt file :
# Disallow: /search/
# Disallow: /?s=
# STEP 2 :
# Add/Update following meta at search page :
# <meta name="robots" content="noindex, nofollow">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment