cs-flatbuffers-perf

cs-flatbuffers-perf.md

CPU: Intel i7-4770K @ 3.50Ghz

Windows 10, 64-bit

Vanilla

Safe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               1.082  0.349 us/op
Decode                               0.018  0.039 us/op
Traverse                             0.438  0.229 us/op

Unsafe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               0.916  0.326 us/op
Decode                               0.010  0.030 us/op
Traverse                             0.448  0.714 us/op

Change to CreateString

Safe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               1.000  0.347 us/op
Decode                               0.014  0.035 us/op
Traverse                             0.424  0.206 us/op

Unsafe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               0.872  0.366 us/op
Decode                               0.018  0.039 us/op
Traverse                             0.310  0.210 us/op

After Pad Optimization - changed the Pad method to loop inside the ByteBuffer, removing extra method calls. Only call Pad from Prep when there's a size.

Safe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               0.854  0.317 us/op
Decode                               0.012  0.033 us/op
Traverse                             0.390  0.198 us/op

Unsafe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               0.782  0.334 us/op
Decode                               0.008  0.027 us/op
Traverse                             0.306  0.239 us/op

A further experiment was to #define away the bounds checks in the ByteBuffer (AssertOffsetAndLength). This leads to a slight gain, mostly in the unsafe configuration. Obviously carried more risk in Unsafe.

Safe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               0.844  0.338 us/op
Decode                               0.018  0.039 us/op
Traverse                             0.392  0.246 us/op

Unsafe .NET 3.5

Name                                  Mean   StdD  Unit
Encode                               0.668  0.311 us/op
Decode                               0.018  0.039 us/op
Traverse                             0.242  0.214 us/op

Ran the Java Benchmark on the same machine (https://github.com/ennerf/flatbuffers-java-benchmark)

Benchmark                                       Mode  Samples  Score  Score error  Units
o.e.f.b.BenchmarkComparison.flatDecodeDirect    avgt       20  0.005        0.000  us/op
o.e.f.b.BenchmarkComparison.flatDecodeHeap      avgt       20  0.007        0.000  us/op
o.e.f.b.BenchmarkComparison.flatEncodeDirect    avgt       20  0.706        0.020  us/op
o.e.f.b.BenchmarkComparison.flatEncodeHeap      avgt       20  0.907        0.022  us/op
o.e.f.b.BenchmarkComparison.flatUseDirect       avgt       20  0.375        0.009  us/op
o.e.f.b.BenchmarkComparison.flatUseHeap         avgt       20  0.471        0.006  us/op

It looks like with the optimizations above, the C# version is roughly equivalent to the performance of Java version. Java Direct is slightly quicker than the Unsafe version of C# without my removal of the bounds checking.

Upgraded the projects to .NET 4.5 and re-ran the CreateString + Pad test in unsafe. This retains the bounds check.

Name                                  Mean   StdD  Unit
Encode                               0.636  0.295 us/op
Decode                               0.012  0.033 us/op
Traverse                             0.448  1.048 us/op

Looks like the improvements in .NET between 3.5 and 4.5 will benefit. We now outperform Java in encoding and traversal ;)

Very cool! Hopefully we can centralize this benchmarking some time soon.