Skip to content

Instantly share code, notes, and snippets.

@benytocarlo

benytocarlo/robots.txt

Created June 25, 2015 20:10
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save benytocarlo/75017de37b1f8bbb0cc0 to your computer and use it in GitHub Desktop.
Save benytocarlo/75017de37b1f8bbb0cc0 to your computer and use it in GitHub Desktop.
Download ZIP
Robots.txt for Magento
Raw
robots.txt
User-agent: *
Disallow: /index.php/
Disallow: /*?
Disallow: /*.js$
Disallow: /*.css$
Disallow: /customer/
Disallow: /checkout/
Disallow: /js/
Disallow: /lib/
Disallow: /media/
Allow: /media/catalog/product/
Disallow: /*.php$
Disallow: /skin/
Disallow: /catalog/product/view/
User-agent: Googlebot-Image
Disallow: /
Allow: /media/catalog/product/
Sitemap: http://example.com/sitemap.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment