Skip to content

Instantly share code, notes, and snippets.

@23inhouse
Last active April 8, 2016 22:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 23inhouse/798fdee8826b96355733 to your computer and use it in GitHub Desktop.
Save 23inhouse/798fdee8826b96355733 to your computer and use it in GitHub Desktop.
Service Communication

Services Communication Design

TL;DR The plan is to use protobuf for the format. Message calls should be made with gRPC when the client wants a response. High level messages should also be published to the bus. Message calls (events) that don't require a response can just be published to the bus.

There are 3 parts to consider:
  • Messaging format
  • Client calls that require a response - RPC
  • Publishing events - BUS

Messaging Format

The message format must be statically typed, binary, good adoption and have RPC integration.

Protobuf V3 is recommended because it is a mature message format. It is fast to decode and contains all the information required to be decoded. Protobuf is statically typed, has validations and can be used internally and externally. Protobuf can be used wit gRPC and with a message bus.

Alternatives and why they are not suitable

JSON

  • Does not contain the information required to decode the message
  • Does not work well with statically typed languages
  • Not binary
  • No RPC integaretion (only error prone - JSON over HTTP/1.x)

Apache thrift

  • Documentation isn't as good as protobuf
  • Doesn't use HTTP/2

Cap'n Proto

  • Eh?

Flatbuffer

  • Low adoption
  • Design for games

Message Pack

  • Only a bit better than JSON
  • Not statically typed
  • No RPC

Apache Avro

  • Low adoption
  • Bad documentation
  • Not statically typed

BSON

  • Not worth mentioning
  • Sorry.

Client calls that require a response -RPC

  • why is this required
  • requirements
  • recommendations

Publishing events - BUS

  • why is this required
  • requirements
  • recommendations

Protobuf V3 is one of the most mature message formats. It works well with statically typed languages and is fast to decode. Other formats like JSON do not contain the information required to decode the message. Protobuf is statically typed, has validations and can be used internally and externally. Protobuf can be used wit gRPC and with a message bus.

Protobuf is extended by Google's gRPC, once the data structure has been defined, gRPC generates code that can read and write the data on a variety of streams in a variety of languages. gRPC allows for streaming because it uses HTTP/2

These languages are supported: C, C++, Java, Go, Node.js, Python, Ruby, Objective-C, PHP and C#.

For more info: https://developers.google.com/protocol-buffers/

Example:

message Person {
  required int32 id = 1;
  required string name = 2;
  optional string email = 3;
}

Service calls

Service calls are different from communication over a bus. A client makes a call to a specific service and knows what to expect in response. This method is very simple to implement and is a good way to start implementing small services. Google's gRPC implementation is faster than JSON over HTTP and saves battery life on mobile clients. gRPC has tooling to generate code to communicate between client and server.

For more info: http://www.grpc.io/

Example

message HelloRequest {
  string greeting = 1;
}

message HelloResponse {
  string reply = 1;
}

service HelloService {
  rpc SayHello(HelloRequest) returns (HelloResponse);
}

Full working example:

# in terminal 1
# make sure docker is ready to go
git clone git@github.com:brainly/srv-question.git
cd srv-question
./script/build
./script/go go run main.go
# in terminal 2
# make sure docker is ready to go
git clone git@github.com:brainly/dummy-question-client.git
cd dummy-question-client
./script/build
./script/go go run main.go any-uuid-param

Failed contenders

Events on a Bus

...

@vanthome
Copy link

vanthome commented Apr 8, 2016

Hi, very interesting write up! I actually designing a very similar approach (RPC next to bus based messaging).
For the bus part, I would like to know how you ended up pertaining technologies, products, protocols etc.?
Unfortunately there seems to be no event bus that is based on gRPC or protobufs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment