Skip to content

Instantly share code, notes, and snippets.

@benytocarlo
Created June 25, 2015 20:10
Show Gist options
  • Save benytocarlo/75017de37b1f8bbb0cc0 to your computer and use it in GitHub Desktop.
Save benytocarlo/75017de37b1f8bbb0cc0 to your computer and use it in GitHub Desktop.
Robots.txt for Magento
User-agent: *
Disallow: /index.php/
Disallow: /*?
Disallow: /*.js$
Disallow: /*.css$
Disallow: /customer/
Disallow: /checkout/
Disallow: /js/
Disallow: /lib/
Disallow: /media/
Allow: /media/catalog/product/
Disallow: /*.php$
Disallow: /skin/
Disallow: /catalog/product/view/
User-agent: Googlebot-Image
Disallow: /
Allow: /media/catalog/product/
Sitemap: http://example.com/sitemap.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment