Skip to content

Instantly share code, notes, and snippets.

@carlin-q-scott

carlin-q-scott/robots.cshtml

Last active June 29, 2022 19:49
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save carlin-q-scott/08a8c3341b0ee85a9ffd350ba51ae056 to your computer and use it in GitHub Desktop.
Save carlin-q-scott/08a8c3341b0ee85a9ffd350ba51ae056 to your computer and use it in GitHub Desktop.
Download ZIP
robots.txt Razor Page
Raw
robots.cshtml
@page "/robots.txt"
@{
Layout = null;
Response.ContentType = "text/plain";
}
@using Microsoft.AspNetCore.Hosting
@inject IHostingEnvironment env
User-agent: *
@if (env.IsProduction())
{
@Html.Raw("Disallow: /hidden/path");
}
else
{
@Html.Raw("Disallow: /");
}
@carlin-q-scott
Copy link
Author

This Razor Page will generate a robots.txt for your ASP.NET website. By default it will hide the whole site from web crawlers if it's not production. To use this file, put it in the Pages folder inside your ASP.NET project, and modify the Html.Raws accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment