I was not part of the initial API design so I don't know the reasons for using gRPC. For the config mgmt portion, we have been trying to keep to a REST pattern for the exposed HTTP requests. For the config mgmt service, we have leaned more towards passing parameters in the URL than in the body of the request. Where compliance service passes the request parameters more in the body of the request. There are trade-offs on both sides, but from a users' perspective, it would be a cleaner API to stick with one pattern.
One problem we have had with gRPC is ingestion. It is slow to convert large unstructured data like Ohai data into gRPC. We have had to make custom parsers to increase the speed. Also, what Stephan mentioned that the 4m limit has been a problem since the beginning.
One of the plans for using gRPC was to expose those endpoints externally. Then the plan was to update Chef Server and Chef Client to use the gRPC endpoints directly. Also to not use the data-colletor endpoint but to route directly to compliance-service and ingest-service with gRPC. The thought around this was if the Chef client on the node initially converted the message to gRPC then Automate would not have to use its resources on the conversion.
For what Stephan mentioned on the versioning of the API. The plan for Config mgmt team was to use the API versioning only if an existing endpoint needed to be changed in a way that was a breaking change for users. This case never happened therefore we have never increased the version.
One problem with the OpenAPI with Swagger we have run into is that we have to write custom Swagger docs for the streaming endpoints. Other than that problem, I found that documenting the protobuf file to be intuitive. I also like the idea of Gloo's Envoy discovering the endpoints with Swagger.