Python Binary Serialization

Simple string
-------------

>>> s='foo bar baz'


>>> %timeit str(s)
1000000 loops, best of 3: 214 ns per loop


>>> %timeit sj=simplejson.dumps(s)
100000 loops, best of 3: 2.33 us per loop

>>> %timeit simplejson.loads(sj)
100000 loops, best of 3: 5.88 us per loop


>>> %timeit sp=cPickle.dumps(s)
1000000 loops, best of 3: 1.55 us per loop

>>> %timeit cPickle.loads(sc)
1000000 loops, best of 3: 905 ns per loop


>>> %timeit sb=wbin.serialize(s)
1000000 loops, best of 3: 488 ns per loop

>>> %timeit wbin.deserialize(sb)                                                                                                         
1000000 loops, best of 3: 351 ns per loop


# BSON only does dicts, so this is unfair, but still competitive.

>>> d={'':s}
>>> %timeit sm=str(bson.BSON.from_dict(d))
100000 loops, best of 3: 4.68 us per loop

>>> %timeit bson.BSON(sm).to_dict()['']
100000 loops, best of 3: 2.89 us per loop




Dictionary
----------

>>> d=dict([(str(i), random.random()) for i in xrange(200)])


>>> %timeit sj=simplejson.dumps(d)                                                                                                      
1000 loops, best of 3: 329 us per loop

>>> %timeit simplejson.loads(sj)
10000 loops, best of 3: 154 us per loop

>>> len(sj)
5496


>>> %timeit sp=cPickle.dumps(d)
1000 loops, best of 3: 326 us per loop

>>> %timeit cPickle.loads(sp)
10000 loops, best of 3: 175 us per loop

>>> len(sp)
6546


>>> %timeit sm=str(pymongo.bson.BSON.from_dict(d))
10000 loops, best of 3: 32.5 us per loop

>>> %timeit pymongo.bson.BSON(sm).to_dict()
10000 loops, best of 3: 36 us per loop

>>> len(sm)
2495


>>> %timeit sb=wbin.serialize(d)
10000 loops, best of 3: 29.2 us per loop

>>> %timeit wbin.deserialize(sb)
10000 loops, best of 3: 55.6 us per loop

>>> len(sb)
3696



List of various types
---------------------
>>> randgen = {
  'i': lambda: int(random.randrange(sys.maxint)),
  'f': lambda: random.uniform(-1e9, 1e9),
  's': lambda: ''.join([random.choice(string.ascii_letters) for _ in xrange(random.randrange(1, 255))]),
  'l': lambda: [_rand(types='ifs') for _ in xrange(random.randrange(1, 16))],
  'd': lambda: dict([(_rand('s'), _rand(types='ifsl')) for _ in xrange(random.randrange(1, 16))]),
}

>>> def _rand(typ=None, types='ifsld'):
    return randgen[typ or random.choice(types)]()

>>> d=[_rand() for _ in xrange(1000)]


>>> %timeit sj=simplejson.dumps(d)
100 loops, best of 3: 8.56 ms per loop

>>> %timeit simplejson.loads(sj)
100 loops, best of 3: 6.8 ms per loop

>>> len(sj)
538673


>>> %timeit sp=cPickle.dumps(d)
100 loops, best of 3: 12.6 ms per loop

>>> %timeit cPickle.loads(sp)
100 loops, best of 3: 8.97 ms per loop

>>> len(sp)
569564


>>> %timeit sb=wbin.serialize(d)
1000 loops, best of 3: 1.06 ms per loop

>>> %timeit wbin.deserialize(sb)
100 loops, best of 3: 3.28 ms per loop

>>> len(sb)
506781


# BSON only does dicts, so this is unfair, but still competitive.

>>> dm=dict([(str(i), e) for i,e in enumerate(d)])

>>> %timeit sm=str(pymongo.bson.BSON.from_dict(dm))
100 loops, best of 3: 6.55 ms per loop

>>> %timeit pymongo.bson.BSON(sm).to_dict().values()
100 loops, best of 3: 8.93 ms per loop

>>> len(sm)
508495