kingluo/doc.md

## doc.md

      
    Raw
  

              doc.md
            
          
    rate limit policy


[,rate]: pass
(rate, burst]: sleep
(burst,): reject

Test

Configure a fake upstream as APISIX itself, and calculate the actual success rate via serverless plugin.
Compare them with the benchmark metrics from client tools, e.g. wrk.
curl http://127.0.0.1:9180/apisix/admin/routes/test_upstream \
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "uri": "/test_upstream",
    "plugins": {
        "serverless-pre-function": {
            "phase": "rewrite",
            "functions" : ["
            local info = {ts=ngx.time(),cnt=0};
            return function()
                local cur = ngx.time();
                if cur - info.ts >= 1 then
                    ngx.log(ngx.WARN, \"rate=\", info.cnt/(cur-info.ts))
                    info.ts = cur
                    info.cnt = 0
                else
                    info.cnt = info.cnt + 1
                end
                ngx.say(\"ok\");
                return ngx.exit(200);
            end"]
        }
    }
}'

curl http://127.0.0.1:9180/apisix/admin/routes/test_limit_req \
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "uri": "/test_limit_req",
    "plugins": {
        "proxy-rewrite": {
            "uri": "/test_upstream"
        },
        "limit-req": {
            "rate": 3000,
            "burst": 5000,
            "rejected_code": 503,
            "key_type": "var",
            "key": "remote_addr"
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "127.0.0.1:9080": 1
        }
    }
}'

ab -n 30000 -c 10 http://localhost:9080/test_limit_req
...
Server Software:        APISIX/3.1.0
Server Hostname:        localhost
Server Port:            9080

Document Path:          /test_limit_req
Document Length:        3 bytes

Concurrency Level:      30
Time taken for tests:   10.141 seconds
Complete requests:      30000
Failed requests:        0
Total transferred:      4230000 bytes
HTML transferred:       90000 bytes
Requests per second:    2958.34 [#/sec] (mean)
Time per request:       10.141 [ms] (mean)
Time per request:       0.338 [ms] (mean, across all concurrent requests)
Transfer rate:          407.35 [Kbytes/sec] received


./wrk -t10 -c100 -d30s http://localhost:9080/test_limit_req
Running 30s test @ http://localhost:9080/test_limit_req
  10 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    33.30ms    2.16ms  61.44ms   94.88%
    Req/Sec   301.50     10.05   460.00     84.97%
  90089 requests in 30.03s, 15.81MB read
Requests/sec:   2999.99
Transfer/sec:    539.06KB
error.log

2023/02/09 18:48:35 [warn] 2308626#2308626: *148088 [lua] [string "..."]:6: func(): rate=3002, client: ::1, server: _, request: "GET /test_upstream HTTP/1.1", host: "localhost:9080"
2023/02/09 18:48:36 [warn] 2308626#2308626: *148500 [lua] [string "..."]:6: func(): rate=3002, client: ::1, server: _, request: "GET /test_upstream HTTP/1.1", host: "localhost:9080"
2023/02/09 18:48:37 [warn] 2308626#2308626: *148489 [lua] [string "..."]:6: func(): rate=2996, client: ::1, server: _, request: "GET /test_upstream HTTP/1.1", host: "localhost:9080"
2023/02/09 18:48:38 [warn] 2308626#2308626: *148120 [lua] [string "..."]:6: func(): rate=2999, client: ::1, server: _, request: "GET /test_upstream HTTP/1.1", host: "localhost:9080"
2023/02/09 18:48:39 [warn] 2308626#2308626: *148119 [lua] [string "..."]:6: func(): rate=2998, client: ::1, server: _, request: "GET /test_upstream HTTP/1.1", host: "localhost:9080"
2023/02/09 18:48:40 [warn] 2308626#2308626: *148113 [lua] [string "..."]:6: func(): rate=3000, client: ::1, server: _, request: "GET /test_upstream HTTP/1.1", host: "localhost:9080"

k6

It's recommended to use k6.

An extensible load testing tool built for developer happiness

It's a javascript benchmark framework, which is easy to define custom metrics to reflect the success rate only.
Note that I suggest to install earlyoom to avoid freezeing the system due to high memory allocated by k6 running with big number of virtual users.
/opt/k6_httpbin_simple.js
import http from 'k6/http';
import { sleep } from 'k6';
import { Counter } from 'k6/metrics';

const successRate = new Counter('success_rate');

export default function () {
  //http.get('https://server.apisix.dev:9443/get');
  const resp = http.get('http://server.apisix.dev:9080/get');
  if (resp.status == 200) {
      successRate.add(1);
  }
  //sleep(0.03);
}
c put /routes/1 -d '{
    "uri":"/*",
    "plugins": {
        "serverless-pre-function": {
            "phase": "before_proxy",
            "functions": [
                "return function(conf,ctx)
                    ngx.say(\"bypass\")
                    return ngx.exit(ngx.HTTP_OK)
                end"
            ]
        },
        "limit-req": {
            "rate": 100,
            "burst": 300,
            "rejected_code": 503,
            "key_type": "var",
            "key": "remote_addr"
        }
    }
}'

k6 run -u 3000 -d 20s /opt/k6_httpbin_simple.js
From the output, success_rate is 99.348238/s, while the actual rate is 10451.173784/s, including the failed requests. The output matches the rate limit.