This is intended to be a realistic benchmark for a typical production microservice. Each module builds to a Docker container which is intended to be deployed in an autoscaling cluster topology (usually Kubernetes) with mocked upstreams. All measurements are intended to be end-to-end using a scaled test harness like Gatling. In a meaningful sense, this is intended to be a more representative alternative to measurement frameworks such as TechEmpower.
Each module contains a fully independent, idiomatic, and independently-tuned implementation of the same service within a different ecosystem. Pull requests welcome! I'm not personally equally familiar with all of the different frameworks and languages represented within this repository, so if you see something that could be more optimal and/or idiomatic, please feel free to make the change! With that said, the goal is for all implementations to be roughly on the same level in terms of their extensibility and maintainability, meaning that we shouldn't have a situation where one implementation is considerably more (or less) abstracted than the others (since this comes with a cost). Due to differences in frameworks and languages, this isn't a precisely objective metric, but let's do our best.
All performance tests are artificial and biased, and this one is no different. Please read below for a full description of the service and tests. If this matches the archetype of your problem space and service deployment technology, then the results are likely quite predictive of what you will see in your production environment. If this setup does not match your production environment, then the applicability of the results is unclear. Be careful not to take these results out of context.
These endpoint semantics have been chosen to correspond to common microservice scenarios. The request and response bodies have been carefully chosen to defeat over-fitting in JIT optimizations, particularly on the JVM.
Basic healthcheck endpoint. Simply returns 200 at all times. Exempt from soft connection limits (503 responses) but still subject to TCP socket limits.
None
OK
Decodes the JSON body and uses the contents to compute a cache key. This key is then checked in an in-memory hashtable. The results are then wrapped in a JSON response if present; if absent, 404 is produced.
{
"_id": "b1a35a9a-b946-4895-b58b-f024977ce324", // random guid
"lorem": 12345,
"dolor": ["sit", "amet", "consectetur"],
"vel": [0, 1, 2, 3, 4, 5],
"sollicitudin": {
"ullamcorper": "lobortis",
"leo": "metus",
"non": false,
"fringilla": "porttitor", // randomly generated string
"rutrum": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed tempus velit sit amet nunc viverra, sit amet consectetur purus varius. Ut accumsan dolor sed orci consequat laoreet. Morbi vitae mauris quis felis interdum bibendum. Nullam eu congue velit, quis vulputate nisl. Maecenas elementum erat nunc, vel gravida nunc viverra eu. Praesent ut condimentum sem. In ut arcu id ante tincidunt imperdiet quis id nunc. Praesent convallis libero quis felis eleifend cursus."
}
}{
"_id": "b1a35a9a-b946-4895-b58b-f024977ce324", // copy _id from request
"adipiscing": { // retrieve from cache based on .sollicitudin.fringilla key
"elit": {},
"fusce": 3.1415,
"faucibus": {
"hendrerit": [true, false, null],
"orci": null
}
},
"vel": [0, 1, 2, 3, 4, 5]
}Decodes the JSON body and uses the contents to assemble a JSON request body. This request body is then posted to a mocked remote upstream. The response from this upstream is decoded as JSON and its data extracted, subsetted, and reassembled into the JSON response from the endpoint. If the timeout is hit, resources should be cleaned up and an appropriate 503 response returned.
{
"_id": "b1a35a9a-b946-4895-b58b-f024977ce324", // random guid
"rhoncus": ["augue", "nulla", "mauris", "malesuada", "erat"],
"adipiscing": {
"dignissim": 3.1415,
"erat": {
"purus": [true, false, null, true, true, false, true],
}
},
"ante": "lectus",
"dictum": "sapien"
}Decodes the JSON body and uses the contents to assemble three different JSON request bodies. These request bodies are then posted in parallel to three independent upstreams. The responses from these upstreams are decoded as JSON and their data extracted, subsetted, and reassembled into the JSON response from the endpoint.
If timeouts or errors are hit for upstreams fanout1 or fanout2, then the other upstreams should be canceled and the appropriate 5XX response returned to the downstream. If a timeout or error is hit for upstream fanout0, fallback data should be used instead in the response body.
It is intended that each of these upstreams is hosted on a separate server to defeat connection pooling optimizations.
- Timeout: 1s
- P50: 25ms
- P99: 750ms
- P9999: 2.5s
- Error Rate: 0.1%
{
"field1": "contents1",
"field2": {
"field21": 42,
"field22": true
},
"field3": ["contents2", null, [3.14,"content3",{}]]
"field4": {"field41":{"field411":[]}}
}- Timeout: 1s
- P50: 25ms
- P99: 750ms
- P9999: 2.5s
- Error Rate: 5%
- Timeout: 2.5s
- P50: 100ms
- P99: 2s
- P9999: 5s
- Error Rate: 0.5%
- Timeout: 500ms
- P50: 10ms
- P99: 300ms
- P9999: 1s
- Error Rate: 1%
todo