Pages should not show in search results, links should not be followed and images should not be indexed.

X-Robots-Tag

<!--
Avoid search engines (Google, Yahoo, etc) indexing website's content
This method is preferred to using robots.txt, which may still cause your site to
show in search engine result pages even if you have set to disallow.

http://yoast.com/prevent-site-being-indexed/
http://code.google.com/web/controlcrawlindex/docs/robots_meta_tag.html
http://www.youtube.com/watch?v=KBdEwpRQRD0
-->

<!-- 
Web.config
By default, pages should not be indexed e.g. dev, staging sites 
-->
<httpProtocol>
    <customHeaders>
        <clear />
        <add name="X-Robots-Tag" value="noindex, nofollow, noimageindex" />
    </customHeaders>
</httpProtocol>

<!-- 
Web.Production.config
Index pages when in production 
-->
<httpProtocol>
    <customHeaders>
        <add name="X-Robots-Tag" xdt:Transform="Remove" xdt:Locator="Match(name)" />
    </customHeaders>
</httpProtocol>