Test code for http://blog.ionelmc.ro/2015/11/22/memory-use-and-speed-of-json-parsers/
To run:
tox
To run with different dataset (example: canada.json):
tox -- canada.json
For a VERY big data set download (you need tons of RAM for that): https://github.com/zemirco/sf-city-lots-json/raw/master/citylots.json
Other datasets you can try: https://github.com/miloyip/nativejson-benchmark/tree/master/data
If you only want to compare few implementations then run:
BM_IMPL='json simplejson msgpack' tox
On Windows (note that memory measurements are disabled there):
set BM_IMPL=json simplejson msgpack tox
To bind to a different CPU (default is 0
):
BM_CPU='1' tox
CPU binding helps a bit during speed tests, as code is single-threaded.
Measure memory differently: you can use ru_maxrss
instead of valgrind (similar to what GNU /bin/time
does with
%M
):
BM_MEM=maxrss tox
Implementations that are tested by default:
- cbor
- cjson
- cPickle
- ijson
- json
- jsonlib
- jsonlib2
- jsonstreamer
- msgpack
- pickle
- rapidjson
- simplejson
- ujson
- yajl
Extra implementations not included in the defafult BM_IMPL
:
- marshal
- pyjsmn
- ruamel.yaml
- yaml
- lxml
To test everything plus lxml
:
BM_IMPL_ADD='lxml' tox
To test just cbor
and msgpack
:
BM_IMPL='cbor msgpack' tox
pip install tox
You might need to run:
apt-get install build-essential git-core python-dev python3.5-dev valgrind libyajl2 libyajl-dev
Python 3.5 is available from the Deadsnakes PPA:
apt-add-repository ppa:fkrull/deadsnakes