APISIX ingress controller integration with Knative Serving
The Apache Software Foundation
Fang Hu
Shuyang Wu, Ming Wen
Main PR: #606 (note that the PR here only includes my partial work due to code review purpose and dependency conflict)
Other PRs related to AIC (most recent last): #406, #438, #546, #556, #557, #558
Conformance and E2E test for APISIX Ingress Controller
Hello-world app from Hello World - Go in Knative Serving. The demo prints Hello Go Sample v1!, which indicates that traffic is correctly routed by APISIX.
Developing branch with immature features
Briefly speaking, I made APISIX Ingress Controller (AIC) an implementation of Knative Ingress gateway and provided test, documentation and a demo, along with other community contributions.
Specifically, my work is to integrate AIC, an ingress controller for Kubernetes, with Knative Serving, a middleware component that supports serverless application deployment. Though heavily based on Kubernetes Ingress, Knative Ingress has some highlighted differences which need to be addressed in a novel approach. The ultimate goal is to make AIC on the official list of Knative supported ingress gateway like Istio.
The major deliverable is a new module that detects, translates, and sync Knative Ingress. First, when a new Knative Ingress is added, the module is able to detect the add event. Second, a translation is triggered, which parses Knative Ingress and converts it to APISIX resources like APISIX Route, Upstream, SSL, etc. Then it keeps watching the newly added Knative Ingress and updates APISIX resources when it is modified or deleted.
I also developed end-to-end test, adapted from knative/networking. The test includes knative conformance test and well as e2e test, and reports test outcome (pass/fail) and runtime. Now it passes around 70% of the conformance test (see Appendices for details). Note that as stated here, 75% pass of a custom ingress gateway implementation can be listed on the Knative Serving supported ingress gateway as an early phase.
There are also two docs and a demo. One doc describes how to deploy knative-enabled AIC. The other doc shows how to run the test from scratch, delivered with a out-of-the-box setup shell script. The demo is a Hello World provided in Knative code sample, which is used in many other alternative ingress gateway as a get-started guide.
To make it clear, non-trivial work I did throughout the program includes but not limited to:
- Integrate AIC with Knative Serving
- Implement Knative Ingress detection, translation, and synchronization
- Support AppendHeaders feature of Knative Ingress
- Add knative conformance and conflict test, as well as a unit test
- Add documentation for deploying and testing AIC with support for Knative Serving
- Add a Hello-world demo
- Other community contributions
- Add e2e test cases for server-info plugin
- Patch route with the correct apiVersion
- Update and clarify installation by Kustomize
-
Make knative conformance test bug-free again
Knative conformance test was able to run until recently. When AIC upgraded to its first GA version (v1.x), its dependency change on the k8s packages created conflicts with knative/pkg. The newer version of k8s packages which AIC depends on causes compile errors when building chaosduck image, an image used in knative conformance test. I have contact knative/pkg community in an issue and offered to create a PR to upgrade the dependency.
-
Wait for feature support in AIC
There are some features that AIC is missing or under development such as native support for SNI based TLS, traffic-split, ingress class and visibility. Despite the immaturity, I have drafted a version in the developing branch that supports more Knative Ingress features like TLS, traffic split, and ingress class. Once the unsupported features are done, corresponding support for Knative Ingress can be added. Note that there are two exceptions, i.e., gRPC and HTTP/2. These two features are not implemented even though AIC does seem to support them.
-
Run conflict test
Now the conflict test cannot be run within expected time.
I thank Shuyang Wu (@Yiyiyimu) for his mentorship. He has been highly responsive and supportive. I also thank Ming Wen for being one of my mentors. I am fortunate to have worked with such great people during the summer. Besides, I express my gratitude to APISIX Ingress Controller community and the Apache Software Foundation for offering such a great opportunity. Last but not least, I would like to thank the Google Summer of Code team who created such an unforgettable experience.
- Output of knative conformance and conflict test
--- FAIL: TestIngressConformance (0.00s)
--- FAIL: TestIngressConformance/dispatch/percentage (46.86s)
--- PASS: TestIngressConformance/websocket (8.20s)
--- FAIL: TestIngressConformance/tls (4.15s)
--- PASS: TestIngressConformance/retry (4.15s)
--- PASS: TestIngressConformance/hosts/multiple (5.13s)
--- PASS: TestIngressConformance/websocket/split (10.26s)
--- PASS: TestIngressConformance/update (116.28s)
--- FAIL: TestIngressConformance/grpc/split (10.27s)
--- FAIL: TestIngressConformance/basics/http2 (4.21s)
--- FAIL: TestIngressConformance/dispatch/path_and_percentage (17.46s)
--- FAIL: TestIngressConformance/grpc (6.08s)
--- PASS: TestIngressConformance/basics (4.12s)
--- PASS: TestIngressConformance/dispatch/rule (7.23s)
--- PASS: TestIngressConformance/dispatch/path (16.42s)
--- PASS: TestIngressConformance/dispatch/path//foo (0.01s)
--- PASS: TestIngressConformance/dispatch/path//asdf (0.00s)
--- PASS: TestIngressConformance/dispatch/path/#00 (0.00s)
--- PASS: TestIngressConformance/dispatch/path//baz (0.00s)
--- PASS: TestIngressConformance/dispatch/path//bar (0.00s)
--- PASS: TestIngressConformance/headers/pre-split (4.18s)
--- PASS: TestIngressConformance/headers/pre-split/Check_without_passing_header (0.05s)
--- PASS: TestIngressConformance/headers/pre-split/Check_with_passing_header (0.04s)
--- FAIL: TestIngressConformance/headers/post-split (10.44s)
--- FAIL: TestIngressConformance/headers/post-split/Check_without_passing_header (1.27s)
--- FAIL: TestIngressConformance/headers/post-split/Check_with_passing_header (0.48s)
--- FAIL: TestIngressConformance/visibility/split (52.57s)
--- PASS: TestIngressConformance/timeout (4.13s)
--- PASS: TestIngressConformance/timeout/no_delays_is_OK (10.03s)
--- PASS: TestIngressConformance/timeout/large_delay_after_headers_is_ok (10.00s)
--- PASS: TestIngressConformance/timeout/large_delay_before_headers_is_ok (10.00s)
--- PASS: TestIngressConformance/headers/probe (4.22s)
--- PASS: TestIngressConformance/headers/probe/kingress_generates_hash (0.02s)
--- PASS: TestIngressConformance/headers/probe/request_overrides_hash (0.06s)
--- FAIL: TestIngressConformance/visibility/path (21.50s)
--- PASS: TestIngressConformance/visibility/path/#00 (0.04s)
--- PASS: TestIngressConformance/visibility/path//baz (0.00s)
--- PASS: TestIngressConformance/visibility/path//bar (0.01s)
--- PASS: TestIngressConformance/visibility/path//foo (0.01s)
--- PASS: TestIngressConformance/visibility/path//asdf (0.00s)
--- FAIL: TestIngressConformance/ingressclass (3.12s)
--- FAIL: TestIngressConformance/ingressclass/nil (2.04s)
--- FAIL: TestIngressConformance/ingressclass/incorrect (2.05s)
--- FAIL: TestIngressConformance/ingressclass/empty (2.03s)
--- FAIL: TestIngressConformance/ingressclass/omitted (2.03s)
--- FAIL: TestIngressConformance/visibility (4.14s)
--- PASS: TestIngressConformance/visibility/shortest (7.14s)
--- PASS: TestIngressConformance/visibility/short (9.17s)
--- PASS: TestIngressConformance/visibility/fqdn (8.16s)
FAIL
FAIL github.com/apache/apisix-ingress-controller/test/conformance 216.963s
=== RUN TestConflictingDomains
conflict_test.go:20: [conflicting-domains-tjunhlml] Using port 81
conflict_test.go:20: [conflicting-domains-tjunhlml] Using containerPort 8034
conflict_test.go:41: ingressendpoint: "172.18.0.2"
panic: test timed out after 5m0s