Skip to content

Instantly share code, notes, and snippets.

@npgenx

npgenx/.htaccess

Last active April 30, 2022 00:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save npgenx/7614fdfac81b6360af8b to your computer and use it in GitHub Desktop.
Save npgenx/7614fdfac81b6360af8b to your computer and use it in GitHub Desktop.
Download ZIP
apache: Allow Facebook and twitter crawlers behind a password protected site
Raw
.htaccess
AuthUserFile #Location of htpasswd file#
AuthType Basic
AuthName #<Site Name>#
Require valid-user
Order deny,allow
Deny from all
# Update the fb and twitter crawler IP lists by running the following commands, then copy & replace the IPs below.
#
# Facebook
# whois -h whois.radb.net -- '-i origin AS32934' | grep ^route
# Twitter
# whois -h whois.radb.net -- '-i origin AS13414' | grep ^route
#
#
# Details can be found at:
# http://www.askapache.com/htaccess/301-redirect-with-mod_rewrite-or-redirectmatch.html
# fb routes on 01/10/2016
Allow from 204.15.20.0/22
Allow from 69.63.176.0/20
Allow from 66.220.144.0/20
Allow from 66.220.144.0/21
Allow from 69.63.184.0/21
Allow from 69.63.176.0/21
Allow from 74.119.76.0/22
Allow from 69.171.255.0/24
Allow from 173.252.64.0/18
Allow from 69.171.224.0/19
Allow from 69.171.224.0/20
Allow from 103.4.96.0/22
Allow from 69.63.176.0/24
Allow from 173.252.64.0/19
Allow from 173.252.70.0/24
Allow from 31.13.64.0/18
Allow from 31.13.24.0/21
Allow from 66.220.152.0/21
Allow from 66.220.159.0/24
Allow from 69.171.239.0/24
Allow from 69.171.240.0/20
Allow from 31.13.64.0/19
Allow from 31.13.64.0/24
Allow from 31.13.65.0/24
Allow from 31.13.67.0/24
Allow from 31.13.68.0/24
Allow from 31.13.69.0/24
Allow from 31.13.70.0/24
Allow from 31.13.71.0/24
Allow from 31.13.72.0/24
Allow from 31.13.73.0/24
Allow from 31.13.74.0/24
Allow from 31.13.75.0/24
Allow from 31.13.76.0/24
Allow from 31.13.77.0/24
Allow from 31.13.96.0/19
Allow from 31.13.66.0/24
Allow from 173.252.96.0/19
Allow from 69.63.178.0/24
Allow from 31.13.78.0/24
Allow from 31.13.79.0/24
Allow from 31.13.80.0/24
Allow from 31.13.82.0/24
Allow from 31.13.83.0/24
Allow from 31.13.84.0/24
Allow from 31.13.85.0/24
Allow from 31.13.86.0/24
Allow from 31.13.87.0/24
Allow from 31.13.88.0/24
Allow from 31.13.89.0/24
Allow from 31.13.90.0/24
Allow from 31.13.91.0/24
Allow from 31.13.92.0/24
Allow from 31.13.93.0/24
Allow from 31.13.94.0/24
Allow from 31.13.95.0/24
Allow from 69.171.253.0/24
Allow from 69.63.186.0/24
Allow from 31.13.81.0/24
Allow from 179.60.192.0/22
Allow from 179.60.192.0/24
Allow from 179.60.193.0/24
Allow from 179.60.194.0/24
Allow from 179.60.195.0/24
Allow from 185.60.216.0/22
Allow from 45.64.40.0/22
Allow from 185.60.216.0/24
Allow from 185.60.217.0/24
Allow from 185.60.218.0/24
Allow from 185.60.219.0/24
Allow from 129.134.0.0/16
Allow from 157.240.0.0/16
Allow from 157.240.8.0/24
Allow from 157.240.0.0/24
Allow from 157.240.1.0/24
Allow from 157.240.2.0/24
Allow from 157.240.3.0/24
Allow from 157.240.4.0/24
Allow from 157.240.5.0/24
Allow from 157.240.6.0/24
Allow from 157.240.7.0/24
Allow from 157.240.9.0/24
Allow from 157.240.10.0/24
Allow from 204.15.20.0/22
Allow from 69.63.176.0/20
Allow from 69.63.176.0/21
Allow from 69.63.184.0/21
Allow from 66.220.144.0/20
Allow from 69.63.176.0/20
Allow from 2620:0:1c00::/40
Allow from 2a03:2880::/32
Allow from 2a03:2880:fffe::/48
Allow from 2a03:2880:ffff::/48
Allow from 2620:0:1cff::/48
Allow from 2a03:2880:f000::/48
Allow from 2a03:2880:f001::/48
Allow from 2a03:2880:f002::/48
Allow from 2a03:2880:f003::/48
Allow from 2a03:2880:f004::/48
Allow from 2a03:2880:f005::/48
Allow from 2a03:2880:f006::/48
Allow from 2a03:2880:f007::/48
Allow from 2a03:2880:f008::/48
Allow from 2a03:2880:f009::/48
Allow from 2a03:2880:f00a::/48
Allow from 2a03:2880:f00b::/48
Allow from 2a03:2880:f00c::/48
Allow from 2a03:2880:f00d::/48
Allow from 2a03:2880:f00e::/48
Allow from 2a03:2880:f00f::/48
Allow from 2a03:2880:f010::/48
Allow from 2a03:2880:f011::/48
Allow from 2a03:2880:f012::/48
Allow from 2a03:2880:f013::/48
Allow from 2a03:2880:f014::/48
Allow from 2a03:2880:f015::/48
Allow from 2a03:2880:f016::/48
Allow from 2a03:2880:f017::/48
Allow from 2a03:2880:f018::/48
Allow from 2a03:2880:f019::/48
Allow from 2a03:2880:f01a::/48
Allow from 2a03:2880:f01b::/48
Allow from 2a03:2880:f01c::/48
Allow from 2a03:2880:f01d::/48
Allow from 2a03:2880:f01e::/48
Allow from 2a03:2880:f01f::/48
Allow from 2a03:2880:1000::/36
Allow from 2a03:2880:2000::/36
Allow from 2a03:2880:3000::/36
Allow from 2a03:2880:4000::/36
Allow from 2a03:2880:5000::/36
Allow from 2a03:2880:6000::/36
Allow from 2a03:2880:7000::/36
Allow from 2a03:2880:f020::/48
Allow from 2a03:2880:f021::/48
Allow from 2a03:2880:f022::/48
Allow from 2a03:2880:f023::/48
Allow from 2a03:2880:f024::/48
Allow from 2a03:2880:f025::/48
Allow from 2a03:2880:f026::/48
Allow from 2a03:2880:f027::/48
Allow from 2a03:2880:f028::/48
Allow from 2a03:2880:f029::/48
Allow from 2a03:2880:f02b::/48
Allow from 2a03:2880:f02c::/48
Allow from 2a03:2880:f02d::/48
Allow from 2a03:2880:f02e::/48
Allow from 2a03:2880:f02f::/48
Allow from 2a03:2880:f030::/48
Allow from 2a03:2880:f031::/48
Allow from 2a03:2880:f032::/48
Allow from 2a03:2880:f033::/48
Allow from 2a03:2880:f034::/48
Allow from 2a03:2880:f035::/48
Allow from 2a03:2880:f036::/48
Allow from 2a03:2880:f037::/48
Allow from 2a03:2880:f038::/48
Allow from 2a03:2880:f039::/48
Allow from 2a03:2880:f03a::/48
Allow from 2a03:2880:f03b::/48
Allow from 2a03:2880:f03c::/48
Allow from 2a03:2880:f03d::/48
Allow from 2a03:2880:f03e::/48
Allow from 2a03:2880:f03f::/48
Allow from 2401:db00::/32
Allow from 2a03:2880::/36
Allow from 2803:6080::/32
Allow from 2a03:2880:f100::/48
Allow from 2a03:2880:f200::/48
Allow from 2a03:2880:f101::/48
Allow from 2a03:2880:f201::/48
Allow from 2a03:2880:f102::/48
Allow from 2a03:2880:f202::/48
Allow from 2a03:2880:f103::/48
Allow from 2a03:2880:f203::/48
Allow from 2a03:2880:f104::/48
Allow from 2a03:2880:f204::/48
Allow from 2a03:2880:f107::/48
Allow from 2a03:2880:f207::/48
Allow from 2a03:2880:f108::/48
Allow from 2a03:2880:f208::/48
Allow from 2a03:2880:f109::/48
Allow from 2a03:2880:f209::/48
Allow from 2a03:2880:f10a::/48
Allow from 2a03:2880:f20a::/48
Allow from 2a03:2880:f10b::/48
Allow from 2a03:2880:f20b::/48
Allow from 2a03:2880:f10d::/48
Allow from 2a03:2880:f20d::/48
Allow from 2a03:2880:f10e::/48
Allow from 2a03:2880:f20e::/48
Allow from 2a03:2880:f10f::/48
Allow from 2a03:2880:f20f::/48
Allow from 2a03:2880:f110::/48
Allow from 2a03:2880:f210::/48
Allow from 2a03:2880:f111::/48
Allow from 2a03:2880:f211::/48
Allow from 2a03:2880:f112::/48
Allow from 2a03:2880:f212::/48
Allow from 2a03:2880:f114::/48
Allow from 2a03:2880:f214::/48
Allow from 2a03:2880:f115::/48
Allow from 2a03:2880:f215::/48
Allow from 2a03:2880:f116::/48
Allow from 2a03:2880:f216::/48
Allow from 2a03:2880:f117::/48
Allow from 2a03:2880:f217::/48
Allow from 2a03:2880:f118::/48
Allow from 2a03:2880:f218::/48
Allow from 2a03:2880:f119::/48
Allow from 2a03:2880:f219::/48
Allow from 2a03:2880:f11a::/48
Allow from 2a03:2880:f21a::/48
Allow from 2a03:2880:f11f::/48
Allow from 2a03:2880:f21f::/48
Allow from 2a03:2880:f121::/48
Allow from 2a03:2880:f221::/48
Allow from 2a03:2880:f122::/48
Allow from 2a03:2880:f222::/48
Allow from 2a03:2880:f123::/48
Allow from 2a03:2880:f223::/48
Allow from 2a03:2880:f10c::/48
Allow from 2a03:2880:f20c::/48
Allow from 2a03:2880:f126::/48
Allow from 2a03:2880:f226::/48
Allow from 2a03:2880:f105::/48
Allow from 2a03:2880:f205::/48
Allow from 2a03:2880:f125::/48
Allow from 2a03:2880:f225::/48
Allow from 2a03:2880:f106::/48
Allow from 2a03:2880:f206::/48
Allow from 2a03:2880:f11b::/48
Allow from 2a03:2880:f21b::/48
Allow from 2a03:2880:f113::/48
Allow from 2a03:2880:f213::/48
Allow from 2a03:2880:f11c::/48
Allow from 2a03:2880:f21c::/48
Allow from 2a03:2880:f128::/48
Allow from 2a03:2880:f228::/48
Allow from 2a03:2880:f02a::/48
Allow from 2a03:2880:f12a::/48
Allow from 2a03:2880:f22a::/48
#End Facebook
#Twitter routes from 06/07/2017
Allow from 199.96.57.0/24
Allow from 199.16.156.0/22
Allow from 199.59.148.0/22
Allow from 192.133.76.0/22
Allow from 192.133.76.0/23
Allow from 199.96.59.0/24
Allow from 199.96.58.0/24
Allow from 199.96.63.0/24
Allow from 199.96.56.0/21
Allow from 103.252.112.0/22
Allow from 103.252.114.0/23
Allow from 185.45.4.0/23
Allow from 69.12.56.0/21
Allow from 104.244.42.0/24
Allow from 185.45.5.0/24
Allow from 185.45.4.0/24
Allow from 199.96.56.0/24
Allow from 202.160.128.0/22
Allow from 202.160.128.0/24
Allow from 202.160.129.0/24
Allow from 202.160.130.0/24
Allow from 202.160.131.0/24
Allow from 188.64.224.0/24
Allow from 188.64.225.0/24
Allow from 188.64.226.0/24
Allow from 188.64.227.0/24
Allow from 188.64.228.0/24
Allow from 188.64.229.0/24
Allow from 188.64.230.0/24
Allow from 188.64.231.0/24
Allow from 188.64.224.0/21
Allow from 104.244.40.0/24
Allow from 104.244.41.0/24
Allow from 104.244.43.0/24
Allow from 185.45.6.0/23
Allow from 192.44.68.0/23
Allow from 192.48.236.0/23
Allow from 199.96.56.0/23
Allow from 199.69.58.0/23
Allow from 199.96.60.0/23
Allow from 199.96.62.0/23
Allow from 192.44.68.0/24
Allow from 192.44.69.0/24
Allow from 103.252.112.0/23
Allow from 188.64.226.0/23
Allow from 104.244.44.0/24
Allow from 104.244.45.0/24
Allow from 104.244.46.0/24
Allow from 104.244.47.0/24
Allow from 2400:6680::/32
Allow from 2606:1f80::/32
Allow from 2a04:9d40::/29
#End Twitter
#Keep this satisfy last
Satisfy any
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment