SerializationSizes.markdown

Object Graph Serialization File Size Comparison

• Update: Added tbuf, broke out tables, you can use https://github.com/schwa/transmogrifier to convert between msgpack, yaml, bson, json and xml plists

• Update: Added bencode, renamed tbuf to yabon

• Update: Added disclamer, added pickle

Disclaimer

This table compares file size to the baseline JSON:

Please note that this list is just using one particular batch of data. Different data will provide different results. For example binary plists do a great job of removing redundant data - this data set doesn't emphasis that.

Also some data formats MIGHT be losing data (e.g. bencode doesn't support floats, so any floats in the data - and there are some - get downgraded to ints).

Please don't take this table as the last word on which format is best/worst.

Formats

Tested

• bencode: Text based format, invented for bittorrent, has some limitations. http://en.wikipedia.org/wiki/Bencode
• bplist: Apple's binary plist format.
• bson: http://en.wikipedia.org/wiki/BSON
• json: Text based format: http://json.org
• msgpack: http://msgpack.org/
• pickle: Python's native binary object format http://docs.python.org/library/pickle.html
• plist: Apple's XML property list format
• yaml: Text based markup http://www.yaml.org/
• yabon: "Yet another binary object notation" format. https://github.com/schwa/Yet-Another-Binary-Object-Notation

Not tested:

I'm not testing any format that requires a schema to be specified before encoding or decoding the format.

• next-plist: Apple's old style (NeXT) format. This is pretty much a dead format.
• thrift: http://incubator.apache.org/thrift/
• protocol buffers: http://code.google.com/p/protobuf/
• extprot: http://eigenclass.org/R2/writings/extprot-extensible-protocols-intro
• avro: http://avro.apache.org/

File Size

Combined
.msgpack.gz     28.54%
.yabon.gz       28.77% * NEW
.yaml.gz        30.25%
.bson.gz        34.59%
.json.gz        37.21%
.msgpack        37.79%
.pickle.gz      37.91% * NEW
.bencode.gz     38.04% * NEW
.bplist.gz      43.61%
.yabon          44.18% * NEW
.bplist         45.89%
.yaml           48.40%
.bencode        53.29% * NEW
.pickle         53.29% * NEW
.bson           57.19%
.plist.gz       72.12% * NEW
.json          100.00%
.plist         190.79% * NEW

No gz
.msgpack        37.79%
.yabon          44.18% * NEW
.bplist         45.89%
.yaml           48.40%
.bencode        53.29% * NEW
.pickle         53.29% * NEW
.bson           57.19%
.json          100.00%
.plist         190.79% * NEW

gz only
.msgpack.gz     28.54%
.yabon.gz       28.77% * NEW
.yaml.gz        30.25%
.bson.gz        34.59%
.json.gz        37.21%
.pickle.gz      37.91% * NEW
.bencode.gz     38.04% * NEW
.bplist.gz      43.61%
.plist.gz       72.12% * NEW