BERT large QA benchmarks

bench.zsh

(venv_the_pattern) ➜  qasearch git:(benchmark) ✗ python3 transformers_plain_bert_qa.py
[101, 2040, 10438, 13434, 6726, 2426, 6001, 1029, 102, 2019, 17727, 2098, 14428, 3372, 2000, 2925, 2913, 1997, 10519, 6726, 1997, 16464, 15245, 2003, 1996, 3768, 1997, 2511, 4725, 2005, 1996, 10788, 1997, 10519, 16464, 12702, 21759, 7088, 19230, 6413, 2005, 4958, 5178, 4328, 10091, 2913, 102]
airborne transmission of respiratory infections is the lack of established methods for the detection of airborne respiratory microorganisms
10.494894326999999 seconds
(venv_the_pattern) ➜  qasearch git:(benchmark) ✗ redis-benchmark -p 30003 -h 127.0.0.1 -n 10 get "bertqa{8YG}_PMC302072.xml:{8YG}:10_Who performs viral transmission among adults"

get bertqa{8YG}_PMC302072.xml:{8YG}:10_Who performs viral transmission among aduget bertqa{8YG}_PMC302072.xml:{8YG}:10_Who performs viral transmission among aduget bertqa{8YG}_PMC302072.xml:{8YG}:10_Who performs viral transmission among aduget bertqa{8YG}_PMC302072.xml:{8YG}:10_Who performs viral transmission among aduget bertqa{8YG}_PMC302072.xml:{8YG}:10_Who performs viral transmission among adu                                                                                ====== get bertqa{8YG}_PMC302072.xml:{8YG}:10_Who performs viral transmission among adults ======
  10 requests completed in 1.10 seconds
  50 parallel clients
  99 bytes payload
  keep alive: 1
  host configuration "save": 3600 1 300 100 60 10000
  host configuration "appendonly": yes
  multi-thread: no

Latency by percentile distribution:
0.000% <= 176.383 milliseconds (cumulative count 1)
50.000% <= 578.047 milliseconds (cumulative count 5)
75.000% <= 882.175 milliseconds (cumulative count 8)
87.500% <= 985.599 milliseconds (cumulative count 9)
93.750% <= 1088.511 milliseconds (cumulative count 10)
100.000% <= 1088.511 milliseconds (cumulative count 10)

Cumulative distribution of latencies:
0.000% <= 0.103 milliseconds (cumulative count 0)
10.000% <= 177.151 milliseconds (cumulative count 1)
20.000% <= 275.199 milliseconds (cumulative count 2)
30.000% <= 376.319 milliseconds (cumulative count 3)
40.000% <= 477.183 milliseconds (cumulative count 4)
50.000% <= 578.559 milliseconds (cumulative count 5)
60.000% <= 679.423 milliseconds (cumulative count 6)
70.000% <= 781.311 milliseconds (cumulative count 7)
80.000% <= 882.175 milliseconds (cumulative count 8)
90.000% <= 985.599 milliseconds (cumulative count 9)
100.000% <= 1088.511 milliseconds (cumulative count 10)

Summary:
  throughput summary: 9.06 requests per second
  latency summary (msec):
          avg       min       p50       p95       p99       max
      629.536   176.256   578.047  1088.511  1088.511  1088.511