Build Event-Driven Applications With Knative and Crossplane

By Paolo Dettori, Aleksander Slominski, Ansu Varghese, Lionel Villard

In recent years, Kubernetes has emerged as the de facto standard for building cloud native applications. Although most use cases for Kubernetes have been around container management and orchestration, more recently, projects such as Crossplane have been extending the Kubernetes control plane for managing infrastructure services and providing developer-oriented abstractions to consume those services.

Getting Started

To get started with Crossplane, follow the installation instructions for Self-Hosted Crossplane on a kind cluster. Make sure the kind cluster is a Kubernetes version supported by Knative; e.g. Knative v0.21.0 requires a Kubernetes cluster v1.17 or newer. Instead of provisioning the Kubernetes version suggested in the above instructions we recommend using a more recent Kubernetes distribution:

kind create cluster --image kindest/node:v1.20.2 --name kncross --wait 5m
kubectl create ns knapp

Creating the Event Streams Service on IBM Cloud

The Crossplane provider for IBM Cloud project supports the provisioning of managed services on IBM public cloud. The provider templates for this blog can be accessed by cloning the following project in a directory of your choice:

git clone https://github.com/pdettori/crossplane-knative-examples.gitcd crossplane-knative-examples
kubectl apply -f eventing1/resource-instance-eventstreams.yaml
kubectl describe resourceinstance myes
kubectl get resourceinstance myes
NAME STATUS STATE CLASS AGE
myes active 45s
kubectl apply -f eventing1/resource-key-eventstream.yaml
kubectl describe resourcekey myes-creds
kubectl get resourcekey myes-creds
kubectl get secrets -n knapp myes
NAME TYPE DATA AGE
myes connection.crossplane.io/v1alpha1 17 1m
BROKER=$(kubectl get secret myes -n knapp -o=jsonpath="{.data['kafka_brokers_sasl\.0']}" | base64 -D)
USER=$(kubectl get secrets myes -n knapp -o=jsonpath="{.data.user}" | base64 -D)
PASSWORD=$(kubectl get secrets myes -n knapp -o=jsonpath="{.data.password}" | base64 -D)

Create Kafka Topic

To create a Kafka topic, we will first create a ConfigMap with the security configuration info required to connect to the Event Streams service:

kubectl get pods -n knapp
NAME READY STATUS RESTARTS AGE
kafka-tools 1/1 Running 0 64s
kubectl exec -it kafka-tools -n knapp -- bin/kafka-topics.sh --bootstrap-server=$BROKER --command-config /etc/config/producer.config --create --topic fruits

Create a KafkaSource

Run the following command to create a KafkaSource:

kubectl get kafkasources.sources.knative.dev -n knapp
kubectl get kafkasources.sources.knative.dev -n knapp
NAME TOPICS BOOTSTRAPSERVERS READY REASON AGE
fruits ["fruits"] ["broker-0-0r9l4j279lqrv08z.kafka.svc08.us-south.eventstreams.cloud.ibm.com:9093"] True 23s

Run Sample Application

It is time now to deploy our Knative Eventing sample application. The purpose of this application is to display data stored in the Kafka topic named fruits:

kubectl apply -f eventing1/event-display.yaml
kubectl get deployments.apps event-display -n knapp
NAME READY UP-TO-DATE AVAILABLE AGE
event-display 1/1 1 1 47s
kubectl exec -it kafka-tools -n knapp -- bin/kafka-console-producer.sh --broker-list $BROKER --producer.config /etc/config/producer.config --topic fruits
kubectl logs -n knapp -l app=event-display --tail=-1
☁️  cloudevents.Event
Validation: valid
Context Attributes,
specversion: 1.0
type: dev.knative.kafka.event
source: /apis/v1/namespaces/knapp/kafkasources/fruits#fruits
subject: partition:0#0
id: partition:0/offset:0
time: 2021-04-13T01:00:00.283Z
Extensions,
traceparent: 00-fae5d19baa348ff6a2462da683677672-dc70bd0cf67c67c7-00
Data,
Hello

Cleaning Up

Delete your Event Streams key and instance on IBM Cloud by running the following commands:

kubectl delete resourcekey myes-creds
kubectl delete resourceinstance myes
kind delete cluster --name kncross

Conclusion

In this post, we have shown how to combine the power of Knative Eventing and Crossplane to build a simple Knative application backed by a Kafka instance running on IBM Cloud. Crossplane and Knative provide powerful abstraction that will help you to turbocharge your event driven application.

Acknowledgements

We would like to thank Paul Castro for his valuable help with this post and testing the eventing app.

Links

Sr. Technical Staff Member at IBM. Passionate about Cloud, Containers and cooking Italian food. Views are my own.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store