Skip to content

Instantly share code, notes, and snippets.

@ADoebeling

ADoebeling/contao.robots.txt

Created September 4, 2015 12:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ADoebeling/7a3e9940697a46cba231 to your computer and use it in GitHub Desktop.
Save ADoebeling/7a3e9940697a46cba231 to your computer and use it in GitHub Desktop.
Download ZIP
Contao robots.txt
Raw
contao.robots.txt
User-agent: *
Disallow: /check/
Disallow: /contao/
Disallow: /system/
Disallow: /templates/
Disallow: /vendor/
Disallow: /share/index.php
Disallow: /build.xml
Disallow: /composer.json
Disallow: /composer.lock
Disallow: /README.md
Allow: /system/cron/cron.txt
Allow: /system/modules/*/assets/
Allow: /system/*.js
Allow: /system/*.css
Allow: /plugins/*.js
Allow: /plugins/*.css
Allow: /assets/*.js
Allow: /assets/*.css
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment