Skip to content

Instantly share code, notes, and snippets.

@sergejmueller

sergejmueller/nginx.conf

Last active August 18, 2022 08:44
Show Gist options
  • Star 16 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save sergejmueller/0e508f3a4d92fd7ec110 to your computer and use it in GitHub Desktop.
Save sergejmueller/0e508f3a4d92fd7ec110 to your computer and use it in GitHub Desktop.
Download ZIP
Stop search engines from indexing .txt, .log, .xml, .css and .js files in Nginx
Raw
nginx.conf
location ~* \.(txt|log|xml|css|js)$ {
add_header X-Robots-Tag noindex;
}
@jessuppi
Copy link

Thanks @sergejmueller

I've expanded this to noindex non-programming files that Google claims to index.

## allowed: html, xml (also css, js, php, etc)
## Ref: https://support.google.com/webmasters/answer/35287?hl=en

location ~* \.(asp|aspx|doc|docx|dwf|gpx|htm|hwp|jsp|kml|kmz|log|odp|ods|odt|pdf|ppt|pptx|ps|rtf|tex|text|txt|wap|wml|xls|xlsx)$ {
    set $robots "noindex, nofollow, nosnippet, noarchive";
}

Ref: https://github.com/littlebizzy/slickstack/edit/master/modules/nginx/sites/production.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment