Skip to content

Instantly share code, notes, and snippets.

@budparr
budparr / robots.txt
Last active August 13, 2019 14:21
Jekyll Robots page to exclude from robots pages that are excluded from the sitemap
---
layout: null
permalink: robots.txt
---
# filter pages and documents for the noindex key
{% assign noindexPages = site.pages | where: 'sitemap', false %}
{% assign noindexDocuments = site.documents | where: 'sitemap', false %}
User-agent: *
# robotstxt.org - if _config production variable is false robots will be disallowed.
{% if site.production != true %}