Skip to content

Instantly share code, notes, and snippets.

@amorkovin

amorkovin/robots.txt

Last active October 13, 2019 04:14
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save amorkovin/b40d62160457d40bb980c1d0f710ce04 to your computer and use it in GitHub Desktop.
Save amorkovin/b40d62160457d40bb980c1d0f710ce04 to your computer and use it in GitHub Desktop.
Download ZIP
robots.txt
Raw
robots.txt
User-agent: Yandex
Disallow: /wp-admin
Disallow: /wp-includes
Allow: /wp-includes/js/*
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /xmlrpc.php
Disallow: *?s*
Disallow: *?preview=true*
Disallow: */trackback/
Disallow: */feed
Disallow: */attachment/*
Disallow: */print/
Disallow: *?print=*
Disallow: /wp-json*
Allow: /wp-content/uploads/
User-agent: Googlebot
Disallow: /wp-admin
Disallow: /wp-includes
Allow: /wp-includes/js/*
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /xmlrpc.php
Disallow: *?s*
Disallow: *?preview=true*
Disallow: */trackback/
Disallow: */feed
Disallow: */attachment/*
Disallow: */print/
Disallow: *?print=*
Disallow: /wp-json*
Allow: /wp-content/uploads/
User-agent: Mail.Ru
Disallow: /wp-admin
Disallow: /wp-includes
Allow: /wp-includes/js/*
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /xmlrpc.php
Disallow: *?s*
Disallow: *?preview=true*
Disallow: */trackback/
Disallow: */feed
Disallow: */attachment/*
Disallow: */print/
Disallow: *?print=*
Disallow: /wp-json*
Allow: /wp-content/uploads/
User-agent: *
Disallow: /wp-admin
Disallow: /wp-includes
Allow: /wp-includes/js/*
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /xmlrpc.php
Disallow: *?s*
Disallow: *?preview=true*
Disallow: */trackback/
Disallow: */feed
Disallow: */attachment/*
Disallow: */print/
Disallow: *?print=*
Disallow: /wp-json*
Allow: /wp-content/uploads/
User-agent: Googlebot-Image
Allow: /wp-content/uploads/
User-agent: YandexImages
Allow: /wp-content/uploads/
User-agent: Mediapartners-Google
Disallow:
User-Agent: YaDirectBot
Disallow:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment