Skip to content

Instantly share code, notes, and snippets.

@mamico

mamico/wsgi_filestream.md

Last active June 15, 2021 11:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mamico/c4e09e64793a73981da0f7bdb5bbec42 to your computer and use it in GitHub Desktop.
Save mamico/c4e09e64793a73981da0f7bdb5bbec42 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
wsgi_filestream.md

Filestream testing view

zopefoundation/Zope#719

$ dd if=/dev/urandom of=/tmp/largefile.dat bs=1024 count=1024000

from Products.Five.browser import BrowserView
from ZPublisher.Iterators import filestream_iterator
import os

class FilestreamTestView(BrowserView):

    def __call__(self):
        set_header = self.request.response.setHeader
        set_header('Expires', 'Sat, 1 Jan 2000 00:00:00 GMT')
        set_header('Cache-Control', 'max-age=0, must-revalidate, private')
        set_header('Content-Length', os.path.getsize('/tmp/largefile.dat'))
        return filestream_iterator('/tmp/largefile.dat', 'rb')

Summary

$ ab -c 64 -n 64 http://127.0.0.1:8080/@@filestream-test

  • Python 2.7 ZServer (1 thread) Time per request: 482 ms (mean, across all concurrent requests)
  • Python 3.7 WSGI Waitress (1 thread) Time per request: 665 ms
  • Python 3.7 WSGI Bjoern (1 thread) Time per request: 306 ms
  • Python 3.7 WSGI Waitress + wsgi.file_wrapper patch (1 thread) Time per request: 297 ms
  • Python 3.7 WSGI Bjoern + wsgi.file_wrapper patch (1 thread) Time per request: 188 ms

Details

Python 2.7 ZServer (1 thread)

$ ab -c 64 -n 64  http://127.0.0.1:8080/@@filestream-test
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        Zope/(4.1.1,
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /@@filestream-test
Document Length:        1048576000 bytes

Concurrency Level:      64
Time taken for tests:   30.863 seconds
Complete requests:      64
Failed requests:        0
Total transferred:      67108881920 bytes
HTML transferred:       67108864000 bytes
Requests per second:    2.07 [#/sec] (mean)
Time per request:       30862.937 [ms] (mean)
Time per request:       482.233 [ms] (mean, across all concurrent requests)
Transfer rate:          2123453.69 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.4      1       2
Processing: 30787 30836  14.7  30838   30855
Waiting:        5  130  91.0    144     298
Total:      30787 30837  14.4  30839   30856

Percentage of the requests served within a certain time (ms)
  50%  30839
  66%  30848
  75%  30851
  80%  30853
  90%  30855
  95%  30855
  98%  30856
  99%  30856
 100%  30856 (longest request)

Python 3.7 WSGI Waitress (1 thread)

$ ab -c 64 -n 64  http://127.0.0.1:8080/@@filestream-test
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        waitress
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /@@filestream-test
Document Length:        1048576000 bytes

Concurrency Level:      64
Time taken for tests:   42.560 seconds
Complete requests:      64
Failed requests:        0
Total transferred:      67108884224 bytes
HTML transferred:       67108864000 bytes
Requests per second:    1.50 [#/sec] (mean)
Time per request:       42559.568 [ms] (mean)
Time per request:       664.993 [ms] (mean, across all concurrent requests)
Transfer rate:          1539865.72 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   0.5      2       3
Processing:  2519 21938 12029.2  22279   42551
Waiting:        5 19345 11971.2  19713   39727
Total:       2522 21940 12028.8  22281   42552

Percentage of the requests served within a certain time (ms)
  50%  22281
  66%  28898
  75%  32683
  80%  34140
  90%  38442
  95%  40867
  98%  42389
  99%  42552
 100%  42552 (longest request)

Python 3.7 WSGI Bjoern (1 thread)

$ ab -c 64 -n 64  http://127.0.0.1:8080/@@filestream-test
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        bjoern
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /@@filestream-test
Document Length:        1048576000 bytes

Concurrency Level:      64
Time taken for tests:   19.555 seconds
Complete requests:      64
Failed requests:        0
Total transferred:      67108882112 bytes
HTML transferred:       67108864000 bytes
Requests per second:    3.27 [#/sec] (mean)
Time per request:       19555.316 [ms] (mean)
Time per request:       305.552 [ms] (mean, across all concurrent requests)
Transfer rate:          3351314.69 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   0.4      2       2
Processing: 19521 19550   3.7  19550   19551
Waiting:        2   60   7.4     60      62
Total:      19521 19551   4.0  19552   19553

Percentage of the requests served within a certain time (ms)
  50%  19552
  66%  19552
  75%  19552
  80%  19553
  90%  19553
  95%  19553
  98%  19553
  99%  19553

Python 3.7 WSGI Waitress w/ wsgi.file_wrapper patch (1 thread)

$ ab -c 64 -n 64  http://127.0.0.1:8080/@@filestream-test
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        waitress
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /@@filestream-test
Document Length:        1048576000 bytes

Concurrency Level:      64
Time taken for tests:   18.990 seconds
Complete requests:      64
Failed requests:        0
Total transferred:      67108884224 bytes
HTML transferred:       67108864000 bytes
Requests per second:    3.37 [#/sec] (mean)
Time per request:       18990.100 [ms] (mean)
Time per request:       296.720 [ms] (mean, across all concurrent requests)
Transfer rate:          3451062.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.3      1       2
Processing:   304 9521 5477.6   9745   18982
Waiting:        6 9225 5470.9   9467   18606
Total:        304 9522 5477.4   9746   18983

Percentage of the requests served within a certain time (ms)
  50%   9746
  66%  12609
  75%  14351
  80%  15180
  90%  16827
  95%  17979
  98%  18607
  99%  18983
 100%  18983 (longest request)

Python 3.7 WSGI Bjoern w/ wsgi.file_wrapper patch (1 thread)

$ ab -c 64 -n 64  http://127.0.0.1:8080/@@filestream-test
mauro@mauro-laptop:~/Work/sendfile/py37$ ab -c 64 -n 64  http://127.0.0.1:8080/@@filestream-test
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done

Server Software:        
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /@@filestream-test
Document Length:        1048576000 bytes

Concurrency Level:      64
Time taken for tests:   12.060 seconds
Complete requests:      64
Failed requests:        0
Total transferred:      67108882112 bytes
HTML transferred:       67108864000 bytes
Requests per second:    5.31 [#/sec] (mean)
Time per request:       12059.946 [ms] (mean)
Time per request:       188.437 [ms] (mean, across all concurrent requests)
Transfer rate:          5434188.32 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.4      1       2
Processing: 12039 12045   0.8  12045   12045
Waiting:       12   52   5.0     52      53
Total:      12039 12046   1.1  12046   12047

Percentage of the requests served within a certain time (ms)
  50%  12046
  66%  12047
  75%  12047
  80%  12047
  90%  12047
  95%  12047
  98%  12047
  99%  12047
 100%  12047 (longest request)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment