For the last three months, I've worked on an implementation of the Broker and Trigger API of Knative Eventing on top of Apache Kafka, alongside a lot of other dependent projects.
- https://github.com/knative-sandbox/eventing-kafka-broker repository hosts the final project.
- Documentation on how to use the project: https://knative.dev/docs/eventing/broker/kafka-broker/
- Project demo: https://youtu.be/XQeWOkpQcgw
- 61 merged PRs in knative-sandbox/eventing-kafka-broker (main project)
- 19 merged PRs in knative/eventing
- 23 merged PRs in knative/eventing-contrib
- 8 merged PRs in knative/docs
- 2 merged PRs in knative/test-infra
- 1 merged PR in cloudevents/sdk-java
- 34 PRs reviewed in knative/eventing
- 17 PRs reviewed in knative/eventing-contrib
- 5 PRs reviewed in knative-sandbox/eventing-kafka-broker
- 2 PRs reviewed in knative/docs
- 1 PR reviewed in knative/test-infra
- 10 PRs reviewed in cloudevents/sdk-java
- Helped a lot of people, both core developers, and users, on various Knative Slack channels
- Participated in all Eventing and Event Delivery Working Group meetings
- Proposed a new Kubernetes CRD (WIP): knative-extensions/eventing-kafka-broker#134
- Implemented the data plane components:
- The receiver is the component responsible for accepting incoming events and sending them to Kafka.
- The dispatcher is the component responsible for reading messages from Kafka and sending them to downstream services.
- Implemented the control plane components:
- The Broker reconciler is the component responsible for scheduling Kafka producers on receiver replicas.
- The Trigger reconciler is the component responsible for scheduling Kafka consumers on dispatcher replicas.
Every new feature or component implementation was tested with unit tests, integration tests, and end-to-end tests, that were running as presumbit jobs and continuously.
A big thank you to my mentors, Francesco and Ali, and to all other people that weren't official GSoC mentors but that had been helpful during the whole period.