GSoC 2020 with JBoss by Red Hat
For the last three months, I've worked on an implementation of the Broker and Trigger API of Knative Eventing on top of Apache Kafka, alongside a lot of other dependent projects.
- https://github.com/knative-sandbox/eventing-kafka-broker repository hosts the final project.
- Documentation on how to use the project: https://knative.dev/docs/eventing/broker/kafka-broker/
- Project demo: https://youtu.be/XQeWOkpQcgw
Overall Stats (Pull requests, reviews, and community participation)
- 61 merged PRs in knative-sandbox/eventing-kafka-broker (main project)
- 19 merged PRs in knative/eventing
- 23 merged PRs in knative/eventing-contrib
- 8 merged PRs in knative/docs
- 2 merged PRs in knative/test-infra
- 1 merged PR in cloudevents/sdk-java
- 34 PRs reviewed in knative/eventing
- 17 PRs reviewed in knative/eventing-contrib
- 5 PRs reviewed in knative-sandbox/eventing-kafka-broker
- 2 PRs reviewed in knative/docs
- 1 PR reviewed in knative/test-infra
- 10 PRs reviewed in cloudevents/sdk-java
- Helped a lot of people, both core developers, and users, on various Knative Slack channels
- Participated in all Eventing and Event Delivery Working Group meetings
- Proposed a new Kubernetes CRD (WIP): https://github.com/knative-sandbox/eventing-kafka-broker/pull/134
What was done
- Implemented the data plane components:
- The receiver is the component responsible for accepting incoming events and sending them to Kafka.
- The dispatcher is the component responsible for reading messages from Kafka and sending them to downstream services.
- Implemented the control plane components:
- The Broker reconciler is the component responsible for scheduling Kafka producers on receiver replicas.
- The Trigger reconciler is the component responsible for scheduling Kafka consumers on dispatcher replicas.
Every new feature or component implementation was tested with unit tests, integration tests, and end-to-end tests, that were running as presumbit jobs and continuously.