I struggled to find the answer so I hope this helps someone else. This discussion on Google Groups helped point me in the right direction.
I have a website hosted on Microsoft Azure. Each website you create is assigned a subdomain to azurewebsites.net
to which Google and Bing can and do index. My website is using a custom domain that I would prefer to use instead of the azurewebsites.net domain. To do this I needed to block Google and Bing from indexing my site on that domain.
I added the following code to my app.js
file. INSERT BEFORE YOU CALL express.static
.
/**************
* Robots.txt redirect
**************/
app.use(function(req, res, next){