You probably don't want Google crawling your development staging app. Here's how to fix that.
$ mv public/robots.txt config/robots.production.txt
$ cp config/robots.production.txt config/robots.development.txt
Now edit config/routes.rb
to add a route for /robots.txt
, and add the controller code.
Instead of modifying the robots.txt file, we can simply use something like in the head section of application.html.erb. This will ban spiders from crawling the entire staging site.