Skip to content

Instantly share code, notes, and snippets.

@simbus82
Created January 14, 2019 14:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save simbus82/36f96b23572a52bcebd16515f9945af8 to your computer and use it in GitHub Desktop.
Save simbus82/36f96b23572a52bcebd16515f9945af8 to your computer and use it in GitHub Desktop.
Good Robots.txt for Joomla 3.x
# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://tool.motoricerca.info/robots-checker.phtml
User-agent: *
Allow: /*.js*
Allow: /*.css*
Allow: /*.png*
Allow: /*.jpg*
Allow: /*.gif*
Allow: /cache/*.js
Allow: /cache/*.css
Allow: /components/*.js
Allow: /components/*.css
Allow: /modules/*.js
Allow: /modules/*.css
Allow: /plugins/*.js
Allow: /plugins/*.css
Allow: /templates/*.js
Allow: /templates/*.css
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/
#Custom rules
Disallow: /news/author/
# Sitemaps (jsitemap example)
Sitemap: https://www.mysite.com/index.php?option=com_jmap&view=sitemap&format=xml&lang=it
Sitemap: https://www.mysite.com/index.php?option=com_jmap&view=sitemap&format=xml&lang=en
Sitemap: https://www.mysite.com/index.php?option=com_jmap&view=sitemap&format=xml&lang=ru
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment