Skip to content

Instantly share code, notes, and snippets.

@supairish
Created June 18, 2012 23:58
  • Star 62 You must be signed in to star a gist
  • Fork 24 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save supairish/2951524 to your computer and use it in GitHub Desktop.
Nginx - how to limit requests by User Agent
http {
map $http_user_agent $limit_bots {
default '';
~*(google|bing|yandex|msnbot) $binary_remote_addr;
}
limit_req_zone $limit_bots zone=bots:10m rate=1r/m;
server {
location / {
limit_req zone=bots burst=5 nodelay;
}
}
}
@lorenanicole
Copy link

This sounds promising for what I'm looking to do - it appears as if you will map the value of $limit_bots based on the value of the $http_user_agent. That is if the $http_user_agent is google, bing, yandex, or msnbot set the value of $limit_bots to be the binary of the IP address. What if, though, you want different rate limiting that takes into account both the IP address and the $http_user_agent?

@paktek123
Copy link

paktek123 commented Apr 29, 2016

One way of doing IP address and http_user_agent could be to put $binary_remote_addr and $http_user_agent into 1 variable for example "10.1.1.10 google" and use that as the key

hope this helps

@mike503
Copy link

mike503 commented Sep 7, 2016

this is what I did, and it works. had to define a $geo_whitelist variable FIRST, and then reuse that variable in the map {} for user agent (as the default value)

# ref: http://serverfault.com/questions/177461/how-to-rate-limit-in-nginx-but-including-excluding-certain-ip-addresses
# whitelisted IP ranges - will not have limits applied
geo $geo_whitelist {
  default 0;
  1.2.3.4 1;
  2.3.4.5/24 1;
}

# whitelisted user agents - will not have limits applied
map $http_user_agent $whitelist {
  default $geo_whitelist;
  ~*(google) 1;
}

# if whitelist is 0, put the binary IP address in $limit so the rate limiting has something to use
map $whitelist $limit {
  0 $binary_remote_addr;
  1 "";
}

limit_req_zone $limit zone=perip:30m rate=1r/s;

@Creepxz
Copy link

Creepxz commented Jul 17, 2019

thanks mike503 this help a lot :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment