My Google Summer of Code project with CNCF was to improve the usability of cert-manager on multiple cloud providers. This gist contains a description of what work was done, what code got merged, what code didn't get merged, and what's left to do.
As part of this project, we aimed to test cert-manager on multiple cloud providers so that we could capture any current/future problems which arise when using cert-manager with these cloud providers. The project was divided into three sections which have been explained below.
The first phase was creating the actual infrastructure where we would run our tests. We decided to go with AWS first and created the infrastructure using Terraform. We chose the infrastructure as code approach since it is more maintainable and easily used in CI jobs (more on that later). The infrastructure was created in a separate repository: https://github.com/cert-manager/test-infra
List of PRs to create the infrastructure:
- cert-manager/infrastructure#1
- cert-manager/infrastructure#2
- cert-manager/infrastructure#3
- cert-manager/infrastructure#4
- cert-manager/infrastructure#5
- cert-manager/infrastructure#6
The existing e2e tests which are used to test cert-manager weren't directly usable with the infrastructure we created. The two major problems to be solved here were:
- Adding flags that parametrize the hardcoded values which were being used earlier in tests so that we're able to run the tests on any cloud provider.
- Creating a new feature set for the specific tests we wanted to run. Public ACME servers don't support all the e2e tests so we created a new feature set for the ones we wanted to run.
List of PRs which made these changes:
- cert-manager/cert-manager#4056
- cert-manager/cert-manager#4110
- cert-manager/cert-manager#4126
- cert-manager/cert-manager#4164
- cert-manager/cert-manager#4244
PRs in this phase were made to the main cert-manager repository: https://github.com/jetstack/cert-manager
The final step was adding a prow job that would run these tests. This involved building a custom docker image with all the tools we required and then using that image to run the actual prow job. PRs in this phase were made to the testing repository: https://github.com/jetstack/testing
List of PRs which made these changes:
- cert-manager/testing#539
- cert-manager/testing#541
- cert-manager/testing#542
- cert-manager/testing#512
- cert-manager/testing#538
- cert-manager/testing#546
- cert-manager/testing#547
- cert-manager/testing#548
- cert-manager/testing#550
- cert-manager/testing#551
- cert-manager/testing#554
- cert-manager/testing#555
- cert-manager/testing#557
With this, the project was finally completed. We added a prow job that would run once every two days, create our AWS infrastructure, and then run the e2e tests on it. The project can be extended by adding more cloud providers on which we can test cert-manager. Currently, as part of the unmerged code, I've made two PRs which add the infrastructure for GCP and Azure.
- PR for GCP infra - cert-manager/infrastructure#4
- PR for Azure infra - cert-manager/infrastructure#6
If you want to use the project work, you can look at the prow job results on the cert-manager repository. You can also use the AWS infrastructure created for this project which is present here - https://github.com/cert-manager/test-infra/tree/main/aws
I also presented my work at the bi-weekly cert-manager developer meeting on 11th August 2021.
Slides used in the meeting can be found here - https://docs.google.com/presentation/d/1Bo4D_zK4SuNJ5GPDNuMElbf3riEvXgWnN87EN_auaas/edit?usp=sharing