Created
January 14, 2019 14:17
-
-
Save simbus82/36f96b23572a52bcebd16515f9945af8 to your computer and use it in GitHub Desktop.
Good Robots.txt for Joomla 3.x
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# If the Joomla site is installed within a folder | |
# eg www.example.com/joomla/ then the robots.txt file | |
# MUST be moved to the site root | |
# eg www.example.com/robots.txt | |
# AND the joomla folder name MUST be prefixed to all of the | |
# paths. | |
# eg the Disallow rule for the /administrator/ folder MUST | |
# be changed to read | |
# Disallow: /joomla/administrator/ | |
# | |
# For more information about the robots.txt standard, see: | |
# http://www.robotstxt.org/orig.html | |
# | |
# For syntax checking, see: | |
# http://tool.motoricerca.info/robots-checker.phtml | |
User-agent: * | |
Allow: /*.js* | |
Allow: /*.css* | |
Allow: /*.png* | |
Allow: /*.jpg* | |
Allow: /*.gif* | |
Allow: /cache/*.js | |
Allow: /cache/*.css | |
Allow: /components/*.js | |
Allow: /components/*.css | |
Allow: /modules/*.js | |
Allow: /modules/*.css | |
Allow: /plugins/*.js | |
Allow: /plugins/*.css | |
Allow: /templates/*.js | |
Allow: /templates/*.css | |
Disallow: /administrator/ | |
Disallow: /bin/ | |
Disallow: /cache/ | |
Disallow: /cli/ | |
Disallow: /components/ | |
Disallow: /includes/ | |
Disallow: /installation/ | |
Disallow: /language/ | |
Disallow: /layouts/ | |
Disallow: /libraries/ | |
Disallow: /logs/ | |
Disallow: /modules/ | |
Disallow: /plugins/ | |
Disallow: /tmp/ | |
#Custom rules | |
Disallow: /news/author/ | |
# Sitemaps (jsitemap example) | |
Sitemap: https://www.mysite.com/index.php?option=com_jmap&view=sitemap&format=xml&lang=it | |
Sitemap: https://www.mysite.com/index.php?option=com_jmap&view=sitemap&format=xml&lang=en | |
Sitemap: https://www.mysite.com/index.php?option=com_jmap&view=sitemap&format=xml&lang=ru |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment