Skip to content

Instantly share code, notes, and snippets.

@moaiandin

moaiandin/sample-demandware-robots.txt Secret

Last active July 27, 2016 11:28
Show Gist options
  • Save moaiandin/b2dddda8cd18c564fe51bbc2cd27c019 to your computer and use it in GitHub Desktop.
Save moaiandin/b2dddda8cd18c564fe51bbc2cd27c019 to your computer and use it in GitHub Desktop.
Download ZIP
Blocking hygiene pages from crawling and indexation in robots.txt for demandware
Raw
sample-demandware-robots.txt
User-Agent: *
Disallow: /on/demandware.store/Sites-yoursite-Site/default/Account-StartRegister
Disallow: /on/demandware.store/Sites-yoursite-Site/default/Account-PasswordResetDialog
Disallow: /on/demandware.store/Sites-yoursite-Site/default/GiftCert-Purchase
Disallow: /on/demandware.store/Sites-yoursite-Site/default/GiftRegistry-Start
Disallow: /on/demandware.store/Sites-yoursite-Site/default/Page-Show
Disallow: /on/demandware.store/Sites-yoursite-Site/default/Wishlist-SetShippingAddress
Disallow: /on/demandware.static/-/Sites-yoursite-Library/default
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment