Benching utilery

README.md

Install

You need imposm3, postgresql, postgis, python 3.5, siege.

Data

wget http://download.geofabrik.de/europe/france/ile-de-france-latest.osm.pbf

Import:

env PGHOST=/var/run/postgresql/ imposm3 import -mapping ~/Code/maps/pagesjaunes/mapping.yml -read ~/Data/geo/pbf/ile-de-france-latest.osm.pbf -write -connection="postgis:///pagesjaunes" -appendcache -deployproduction

Run

siege -f ~/Code/py/utilery/urls.txt -c 10 -r 10 -b

results.md

Master with simple_server

Command:

python utilery/server.py

Stats:

Transactions:                 40 hits
Availability:             100.00 %
Elapsed time:              84.55 secs
Data transferred:           1.49 MB
Response time:             18.10 secs
Transaction rate:           0.47 trans/sec
Throughput:             0.02 MB/sec
Concurrency:                8.56
Successful transactions:          40
Failed transactions:               0
Longest transaction:           23.95
Shortest transaction:           0.95

Master with 4 gunicorn worker

Command:

gunicorn utilery.views:app -b 0.0.0.0:3579 -w 4

Stats:

Transactions:                100 hits
Availability:             100.00 %
Elapsed time:              72.56 secs
Data transferred:           3.83 MB
Response time:              6.61 secs
Transaction rate:           1.38 trans/sec
Throughput:             0.05 MB/sec
Concurrency:                9.11
Successful transactions:         100
Failed transactions:               0
Longest transaction:           10.29
Shortest transaction:           0.95

Branch async

Command:

python utilery/server.py

Stats:

Transactions:                100 hits
Availability:             100.00 %
Elapsed time:              88.94 secs
Data transferred:           3.83 MB
Response time:              8.01 secs
Transaction rate:           1.12 trans/sec
Throughput:             0.04 MB/sec
Concurrency:                9.00
Successful transactions:         100
Failed transactions:               0
Longest transaction:           16.66
Shortest transaction:           1.18

Branch async with 5 Gunicorn worker (custom worker)

Command:

gunicorn utilery.views:application -b 0.0.0.0:3579 -k utilery.worker.Worker -w 5

Stats:

Transactions:                100 hits
Availability:             100.00 %
Elapsed time:              52.85 secs
Data transferred:           3.83 MB
Response time:              4.39 secs
Transaction rate:           1.89 trans/sec
Throughput:             0.07 MB/sec
Concurrency:                8.30
Successful transactions:         100
Failed transactions:               0
Longest transaction:           12.18
Shortest transaction:           0.89

Branch async with 8 Gunicorn worker (custom worker)

Command:

gunicorn utilery.views:application -b 0.0.0.0:3579 -k utilery.worker.Worker -w 8

Stats:

Transactions:                100 hits
Availability:             100.00 %
Elapsed time:              49.69 secs
Data transferred:           3.83 MB
Response time:              4.31 secs
Transaction rate:           2.01 trans/sec
Throughput:             0.08 MB/sec
Concurrency:                8.67
Successful transactions:         100
Failed transactions:               0
Longest transaction:           13.20
Shortest transaction:           0.92

aiovectortiler

Command:

python aiovectortiler/serve.py --server_configs config.yml --layer_recipes /home/ybon/Code/other/utilery-osm-recipe/ --host 0.0.0.0 --port 3579

Stats:

Transactions:                100 hits
Availability:             100.00 %
Elapsed time:             182.85 secs
Data transferred:           4.18 MB
Response time:             16.83 secs
Transaction rate:           0.55 trans/sec
Throughput:             0.02 MB/sec
Concurrency:                9.20
Successful transactions:         100
Failed transactions:               0
Longest transaction:           32.02
Shortest transaction:           1.65

urls.txt

http://localhost:3579/all/15/16589/11270.mvt
http://localhost:3579/all/15/16588/11270.mvt
http://localhost:3579/all/15/16589/11269.mvt
http://localhost:3579/all/15/16588/11269.mvt
http://localhost:3579/all/15/16588/11268.mvt
http://localhost:3579/all/15/16589/11268.mvt
http://localhost:3579/all/15/16590/11269.mvt
http://localhost:3579/all/15/16587/11269.mvt
http://localhost:3579/all/15/16588/11271.mvt
http://localhost:3579/all/15/16590/11270.mvt
http://localhost:3579/all/15/16589/11271.mvt
http://localhost:3579/all/15/16587/11270.mvt
http://localhost:3579/all/15/16587/11271.mvt
http://localhost:3579/all/15/16590/11271.mvt
http://localhost:3579/all/15/16587/11268.mvt
http://localhost:3579/all/15/16590/11268.mvt
http://localhost:3579/all/15/16591/11270.mvt
http://localhost:3579/all/15/16586/11269.mvt
http://localhost:3579/all/15/16591/11269.mvt
http://localhost:3579/all/15/16586/11270.mvt
http://localhost:3579/all/15/16586/11268.mvt
http://localhost:3579/all/15/16591/11268.mvt
http://localhost:3579/all/15/16586/11271.mvt
http://localhost:3579/all/15/16591/11271.mvt
http://localhost:3579/all/15/16585/11270.mvt
http://localhost:3579/all/15/16592/11269.mvt
http://localhost:3579/all/15/16592/11270.mvt
http://localhost:3579/all/15/16585/11269.mvt
http://localhost:3579/all/15/16585/11268.mvt
http://localhost:3579/all/15/16592/11268.mvt
http://localhost:3579/all/15/16585/11271.mvt
http://localhost:3579/all/15/16592/11271.mvt
http://localhost:3579/all/15/16590/11267.mvt
http://localhost:3579/all/15/16591/11267.mvt
http://localhost:3579/all/15/16589/11267.mvt
http://localhost:3579/all/15/16588/11267.mvt
http://localhost:3579/all/15/16593/11268.mvt
http://localhost:3579/all/15/16587/11267.mvt
http://localhost:3579/all/15/16592/11267.mvt
http://localhost:3579/all/15/16593/11269.mvt
http://localhost:3579/all/15/16593/11267.mvt
http://localhost:3579/all/15/16594/11268.mvt
http://localhost:3579/all/15/16593/11270.mvt
http://localhost:3579/all/15/16594/11269.mvt
http://localhost:3579/all/15/16586/11267.mvt
http://localhost:3579/all/15/16594/11267.mvt
http://localhost:3579/all/15/16594/11270.mvt
http://localhost:3579/all/15/16590/11266.mvt
http://localhost:3579/all/15/16591/11266.mvt
http://localhost:3579/all/15/16589/11266.mvt
http://localhost:3579/all/15/16588/11266.mvt
http://localhost:3579/all/15/16587/11266.mvt
http://localhost:3579/all/15/16592/11266.mvt
http://localhost:3579/all/15/16593/11266.mvt
http://localhost:3579/all/15/16586/11266.mvt
http://localhost:3579/all/15/16594/11266.mvt
http://localhost:3579/all/15/16590/11265.mvt
http://localhost:3579/all/15/16592/11265.mvt
http://localhost:3579/all/15/16589/11265.mvt
http://localhost:3579/all/15/16591/11265.mvt
http://localhost:3579/all/15/16594/11265.mvt
http://localhost:3579/all/15/16593/11265.mvt
http://localhost:3579/all/15/16588/11265.mvt
http://localhost:3579/all/15/16587/11265.mvt
http://localhost:3579/all/15/16595/11265.mvt
http://localhost:3579/all/15/16595/11266.mvt
http://localhost:3579/all/15/16595/11267.mvt
http://localhost:3579/all/15/16595/11268.mvt
http://localhost:3579/all/15/16596/11267.mvt
http://localhost:3579/all/15/16595/11269.mvt
http://localhost:3579/all/15/16597/11267.mvt
http://localhost:3579/all/15/16596/11266.mvt
http://localhost:3579/all/15/16596/11268.mvt
http://localhost:3579/all/15/16596/11269.mvt
http://localhost:3579/all/15/16597/11266.mvt
http://localhost:3579/all/15/16597/11269.mvt
http://localhost:3579/all/15/16597/11268.mvt
http://localhost:3579/all/15/16598/11266.mvt
http://localhost:3579/all/15/16598/11267.mvt
http://localhost:3579/all/15/16598/11269.mvt
http://localhost:3579/all/15/16598/11268.mvt
http://localhost:3579/all/15/16595/11270.mvt
http://localhost:3579/all/15/16596/11270.mvt
http://localhost:3579/all/15/16598/11270.mvt
http://localhost:3579/all/15/16597/11270.mvt
http://localhost:3579/all/15/16595/11271.mvt
http://localhost:3579/all/15/16593/11271.mvt
http://localhost:3579/all/15/16596/11271.mvt
http://localhost:3579/all/15/16594/11271.mvt
http://localhost:3579/all/15/16597/11271.mvt
http://localhost:3579/all/15/16598/11271.mvt

@yohanboniface interesting comparisons. FWIW I believe the version of aiovectortiler at the time was using strict ST_Intersects as opposed to the && operator for selecting the geometry so may be part of the reason for slower performance. Though I suspect non aiohttp version would still be faster at the end of the day.