Skip to content

Instantly share code, notes, and snippets.

@JamesHusband
Created December 23, 2019 14:59
Show Gist options
  • Save JamesHusband/4a06d02c3a6b6213cba927dc30b8ee2a to your computer and use it in GitHub Desktop.
Save JamesHusband/4a06d02c3a6b6213cba927dc30b8ee2a to your computer and use it in GitHub Desktop.
# Test Sites
# User-agent: *
# Disallow: /
# Live Sites
User-agent: *
Allow: /
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /comments
Disallow: /go/*
Disallow: /free-slots-news-t/*
Disallow: */trackback
Disallow: */commentsa
#Disallow: /*?*
#Disallow: /*?
User-agent: Mediapartners-Google
Allow: /
User-agent: Adsbot-Google
Allow: /
User-agent: Googlebot-Image
Allow: /
User-agent: Googlebot-Mobile
Allow: /
User-agent: ia_archiver-web.archive.org
Disallow: /
Sitemap: http://www.superfreeslotgames.com/sitemap.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment