Skip to content

Instantly share code, notes, and snippets.

@grantjenks
Last active October 28, 2021 18:00
Show Gist options
  • Save grantjenks/dacc0a1e7fa9a08264439b9c6a05ec5b to your computer and use it in GitHub Desktop.
Save grantjenks/dacc0a1e7fa9a08264439b9c6a05ec5b to your computer and use it in GitHub Desktop.
Server-side I/O Performance in Python
"""Server-side I/O Performance in Python
Based on the article and discussion at:
https://www.toptal.com/back-end/server-side-io-performance-node-php-java-go
The code was posted at:
https://peabody.io/post/server-env-benchmarks/
The winner was golang which is not too surprising given how the benchmark was
run. So I wondered, how would Python compare?
Here's the golang source code:
```go
package main
import (
"io/ioutil"
"net/http"
"fmt"
"crypto/sha256"
"log"
)
func main() {
http.HandleFunc("/test", func(w http.ResponseWriter, r *http.Request) {
b, err := ioutil.ReadFile("/tmp/data")
if err != nil { panic(err) }
s := ""
nstr := r.FormValue("n")
var n int
fmt.Sscanf(nstr, "%d", &n)
for i := 0; i < n; i++ {
h := sha256.New()
h.Write(b)
s = fmt.Sprintf("%x", h.Sum(nil))
}
w.Write([]byte(s))
})
log.Fatal(http.ListenAndServe("127.0.0.1:4000", nil))
}
```
To me, that source looks gross but it's pretty easy to translate to Python. I
chose the Bottle web framework because its easy and lightweight. Solution
below.
I benchmarked each of them as described in the original article:
ab -n 2000 -c 300 "http://127.0.0.1:8000/test?n=1"
Here's the golang results:
```sh
$ ab -n 2000 -c 300 "http://127.0.0.1:4000/test?n=1"
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 4000
Document Path: /test?n=1
Document Length: 64 bytes
Concurrency Level: 300
Time taken for tests: 0.292 seconds
Complete requests: 2000
Failed requests: 0
Total transferred: 362000 bytes
HTML transferred: 128000 bytes
Requests per second: 6856.69 [#/sec] (mean)
Time per request: 43.753 [ms] (mean)
Time per request: 0.146 [ms] (mean, across all concurrent requests)
Transfer rate: 1211.97 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 16 24.6 10 212
Processing: 2 16 9.3 11 50
Waiting: 2 14 7.4 11 49
Total: 6 31 25.0 23 223
Percentage of the requests served within a certain time (ms)
50% 23
66% 29
75% 33
80% 34
90% 47
95% 119
98% 122
99% 122
100% 223 (longest request)
```
And the Python results:
```sh
$ ab -n 2000 -c 300 "http://127.0.0.1:8000/test?n=1"
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests
Server Software: gunicorn/19.7.1
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /test?n=1
Document Length: 64 bytes
Concurrency Level: 300
Time taken for tests: 0.374 seconds
Complete requests: 2000
Failed requests: 0
Total transferred: 448000 bytes
HTML transferred: 128000 bytes
Requests per second: 5347.55 [#/sec] (mean)
Time per request: 56.100 [ms] (mean)
Time per request: 0.187 [ms] (mean, across all concurrent requests)
Transfer rate: 1169.78 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 15 36.8 5 306
Processing: 1 19 4.8 19 42
Waiting: 1 16 4.5 18 41
Total: 8 34 37.7 23 325
Percentage of the requests served within a certain time (ms)
50% 23
66% 25
75% 28
80% 32
90% 39
95% 128
98% 134
99% 230
100% 325 (longest request)
```
The Python results are really good: 5,347 requests per second for Python vs
6,856 for Golang. I expected more from Golang. My guess is the benchmark is
CPU-bound so all the time is spent reading the file and hashing it. For Python,
that all happens in optimized C code.
"""
import hashlib
from bottle import get, request, run
@get('/test')
def test():
n = int(request.query.n or 1)
with open('/tmp/data', 'rb') as reader:
content = reader.read()
for count in range(n):
h = hashlib.sha256(content)
return h.hexdigest()
if __name__ == '__main__':
run(host='127.0.0.1', port=8000, server='gunicorn', workers=8)
@grantjenks
Copy link
Author

I also ran the benchmark using japronto which is on the bleeding-edge of Python webserver performance. Summary of results:

Configuration Req/sec % slower
Python japronto 7,753.77 baseline
Python bottle+gunicorn 5,347.55 31% slower
golang 6,856.69 12% slower

Here's the code:

import hashlib
import multiprocessing as mp
from japronto import Application

def test(request):
    n = int(request.query.get('n', 1))

    with open('/tmp/data', 'rb') as reader:
        content = reader.read()

    for count in range(n):
        h = hashlib.sha256(content)

    return request.Response(text=h.hexdigest())

app = Application()
app.router.add_route('/test', test)

if __name__ == '__main__':
    app.run('127.0.0.1', 8080, worker_num=mp.cpu_count())

And here's the results:

$ ab -n 2000 -c 300 "http://127.0.0.1:8080/test?n=1"
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests


Server Software:        
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /test?n=1
Document Length:        64 bytes

Concurrency Level:      300
Time taken for tests:   0.258 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      288000 bytes
HTML transferred:       128000 bytes
Requests per second:    7753.77 [#/sec] (mean)
Time per request:       38.691 [ms] (mean)
Time per request:       0.129 [ms] (mean, across all concurrent requests)
Transfer rate:          1090.37 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1   10  28.3      1     204
Processing:     1    5   7.9      2      38
Waiting:        1    4   7.3      2      33
Total:          2   14  29.2      3     207

Percentage of the requests served within a certain time (ms)
  50%      3
  66%      3
  75%      9
  80%     16
  90%     39
  95%    103
  98%    104
  99%    105
 100%    207 (longest request)

@Jonarod
Copy link

Jonarod commented Jul 17, 2019

Nice benchmark, thank you.
Could you share hardware/architecture used to get these results ?

@grantjenks
Copy link
Author

I don’t recall exactly. I think it was a 2015 MacBook Pro all running on localhost.

@rof20004
Copy link

@grantjenks

ioutil.ReadFile has much more cost performance to read a file because it allocate dynamic buffer.

Try change your test with better implementation. Read this: https://www.openmymind.net/Your-Buffer-Is-Leaking/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment