Skip to content

Instantly share code, notes, and snippets.

@mhgrove
Last active December 14, 2015 02:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mhgrove/5013411 to your computer and use it in GitHub Desktop.
Save mhgrove/5013411 to your computer and use it in GitHub Desktop.
A series of snippets of streaming protobuf serialization code, works with protobuf 2.4.1 but not 2.5.0rc1.
// some initialization which takes place in the constructor of the streamed
// protobug message writer
mBytes = new ByteArrayOutputStream((int)(64 * (1 << 10)));
coded = CodedOutputStream.newInstance(mBytes);
// then as protobuf messages come in, i append to the stream
// if the message puts us over the threshold of 64k, we write out what's in
// the buffer and start fresh
private void write(final MessageLite aObj) throws IOException {
final int aObjSize = aObj.getSerializedSize();
final int aCurrSize = mBytes.size();
final int aNewSize = aObjSize + aCurrSize;
if (aNewSize > THRESHOLD) {
write();
mBytes.reset();
}
// mTag is initialized in the constructor with the int representing the repeated field
// in the example from my email, this would be something like MessageType3.TYPE2_FIELD_NUMBER
coded.writeMessage(mTag, aObj);
coded.flush();
}
// here's the shortened write method
private void write() throws IOException {
// mPrototypeAsBytes is created in the constructor from a call like
// MessageType3.getDefaultInstance().toByteArray() -- almost always this is
// a zero length byte array, but i allow for the case that we've set defalut values
// on the object we'd like to send
coded.flush();
final byte[] aBytes = mBytes.toByteArray();
int sz = CodedOutputStream.computeRawVarint32Size(aBytes.length + mPrototypeAsBytes.length);
final byte[] size = new byte[sz];
CodedOutputStream c = CodedOutputStream.newInstance(size);
c.writeRawVarint32(aBytes.length + mPrototypeAsBytes.length);
byte[] aPayload = Bytes.concat(size, mPrototypeAsBytes, aBytes);
// the rest is ommitted, we take the payload and send it across the wire at this point
// where on the other end we use the standard mergeFrom to parse the message back into
// a protobuf message object
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment