Last active
May 1, 2021 21:44
-
-
Save ilyahoilik/c6d70d828f42aae429e3e84283fb85f7 to your computer and use it in GitHub Desktop.
Robots.txt for Invision Community: https://hoilik.com/robots-txt-for-invision-community-473d8aa32984
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This robots.txt file is created by Ilya Hoilik. | |
# You can contribute by visiting the link bellow. | |
# https://gist.github.com/ilyahoilik/c6d70d828f42aae429e3e84283fb85f7 | |
User-agent: * | |
# Folders and files | |
Disallow: /admin | |
Disallow: /api | |
Disallow: /applications | |
Disallow: /datastore | |
Disallow: /dev | |
Disallow: /oauth | |
Disallow: /plugins | |
Disallow: /system | |
Disallow: /uploads | |
Disallow: /vendor | |
Disallow: /404error.php | |
Disallow: /Credits.txt | |
Disallow: /error.php | |
# Authentication | |
Disallow: /login | |
Disallow: /logout | |
Disallow: /register | |
Disallow: /lostpassword | |
# Legal information | |
Disallow: /privacy | |
Disallow: /cookies | |
Disallow: /guidelines | |
Disallow: /terms | |
# Staff and users | |
Disallow: /online | |
Disallow: /staff | |
# Contact page | |
Disallow: /contact | |
# Announcements | |
Disallow: /announcement* | |
# Content discovery | |
Disallow: /search | |
Disallow: /discover* | |
Disallow: /rss* | |
Disallow: /tags* | |
Disallow: /activity | |
Disallow: /new-content | |
# Content promotion | |
Disallow: /promote* | |
Disallow: /ourpicks | |
# Leaderboard | |
Disallow: /leaderboard | |
Disallow: /pastleaders | |
Disallow: /topmembers | |
# With extra parameters | |
Disallow: /*do=* | |
Disallow: /*sort=* | |
Disallow: /*sortby=* | |
Disallow: /*csrf=* | |
Disallow: /*csrfKey=* | |
Disallow: */?tab=* | |
Disallow: */?_fromLogin=* | |
Disallow: */?_fromLogout=* | |
# Create and edit pages | |
Disallow: */submit | |
Disallow: */create | |
Disallow: */edit | |
# Allow attachments in the uploads folder | |
Allow: /uploads/monthly_*_*/* | |
Host: https://example.com | |
Sitemap: https://example.com/sitemap.php |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment