For sure. So using some tests I had while testing mem <-> fst segment serialization, as a poor man's benchmark:
# pilosa
❯ time go test ./index/segment/tests -count=10
ok github.com/m3db/m3ninx/index/segment/tests 2.192s
/Users/prungta/code/go1.10.2/bin/go test ./index/segment/tests -count=10 3.81s user 0.47s system 131% cpu 3.266 total
# roaring binary
❯ time go test ./index/segment/tests -count=10
ok github.com/m3db/m3ninx/index/segment/tests 1.966s
/Users/prungta/code/go1.10.2/bin/go test ./index/segment/tests -count=10 3.76s user 0.46s system 137% cpu 3.073 total
# roaring msgpack
❯ time go test ./index/segment/tests -count=10
ok github.com/m3db/m3ninx/index/segment/test 8.219s
/Users/prungta/code/go1.10.2/bin/go test ./index/segment/tests -count=10 17.38s user 0.78s system 197% cpu 9.211 total
conclusions:
- upstream roaring msgpack is a pos
- binary perf of upstream roaring v pilosa is comparable.
This is despite the fact that in the case of the pilosa tests, we're only using pilosa for serialization; and always convert from []byte -> pilosa -> upstream roaring (and vice versa during writes) to satisfy our existing interfaces. So once we switch our postings implementation to pilosa, it'd be able to avoid a bunch of things it's having to do now.
Also checked mem usage, binary roaring v pilosa were about 15% different - 103MB v 119MB alloc space (again, despite the extra work our pilosa impl had to do for conversions)
