Note: Pseudo code...
Below is some yaml (and no implementation) for Kafka Source and Channel...
We need a ClusterProvisioner
for our Source
e.g. like the container provisioner from Nicolas (see #513):
apiVersion: eventing.knative.dev/v1alpha1
kind: ClusterProvisioner
metadata:
name: container
spec:
reconciles:
group: eventing.knative.dev
kind: Source
and another one for the Channel (e.g. the Kafka one from Sabari in #486)
apiVersion: eventing.knative.dev/v1alpha1
kind: ClusterProvisioner
metadata:
name: kafka
spec:
reconciles:
group: eventing.knative.dev/v1alpha1
kind: Channel
Now, that we have the two Provisioners together....
Next we would need a channel:
apiVersion: eventing.knative.dev/v1alpha1
kind: Channel
metadata:
name: mychannel
spec:
provisioner:
ref:
apiVersion: eventing.knative.dev/v1alpha1
kind: ClusterProvisioner
name: kafka
And we want to connect to Apache Kafka on our source, so we might have:
apiVersion: eventing.knative.dev/v1alpha1
kind: Source
metadata:
name: kafkaevents
namespace: default
spec:
provisioner:
ref:
name: container
arguments:
image: docker.io/mrbean/kafkasource
args:
bootstrapservers: "something.somewhere.com:9092"
subscribe_to_topic: "the_truth"
Now, we need to hook our "service" to the channel:
apiVersion: eventing.knative.dev/v1alpha1
kind: Subscription
metadata:
name: listentochannel
namespace: default
spec:
from:
kind: Channel
apiVersion: eventing.knative.dev/v1alpha1
name: kafka
call:
target:
kind: Service
apiVersion: serving.knative.dev/v1alpha1
name: my-kafka-logger
Question: I am not sure how to actually connect the Channel to the Source, so that the Source can write to the channel, and my ksvc
on the Subscription
can get messages from that populated channel?
This is a point of contention in the spec proposal.
The current proposal says that the Source should reference the Channel it wants to emit events to, if not specified, then the provisioner must create a channel and emit to it. The Subscription would then be
from
the kafkaevents source.My proposal in n3wscott/eventing#4 says that a Source should never be responsible for creating a Channel and should only emit events to an exiting Channel (
mychannel
in your example). Thelistentochannel
subscription should befrom
mychannel
.In the example, you have both a Source and Channel backed by Kafka. The example may be more clear if you picked different technologies to back the Source or Channel.