Skip to content

Instantly share code, notes, and snippets.

@tgardiner
Created January 13, 2019 03:01
Show Gist options
  • Save tgardiner/31672ab7fa0454e7422e982568c2d1de to your computer and use it in GitHub Desktop.
Save tgardiner/31672ab7fa0454e7422e982568c2d1de to your computer and use it in GitHub Desktop.
API Gateway Resource Policy to block Web Crawlers with wildcards
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:{REGION}:{ACCOUNT_ID}:{API_NAME}/*"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:{REGION}:{ACCOUNT_ID}:{API_NAME}/*",
"Condition": {
"StringLike": {
"aws:UserAgent": [
"*Googlebot*",
"*Google*",
"*Bingbot*",
"*Slurp*",
"*DuckDuckBot*",
"*Baiduspider*",
"*YandexBot*",
"*Sogou*",
"*Exabot*",
"*facebot*",
"*ia_archiver*"
]
}
}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment