Skip to content

Instantly share code, notes, and snippets.

@renormalize

renormalize/ci-e2e-kind-azure-output-1.txt Secret

Last active February 9, 2024 08:35
Show Gist options
  • Save renormalize/cedc79071fc8199c0c3378b1034c2b6d to your computer and use it in GitHub Desktop.
Save renormalize/cedc79071fc8199c0c3378b1034c2b6d to your computer and use it in GitHub Desktop.
Download ZIP
Output of running end-to-end tests for `etcd-druid` using Azurite as the object store.
Raw
ci-e2e-kind-azure-output-1.txt
➜ etcd-druid git:(azurite) ✗ make ci-e2e-kind-azure
BUCKET_NAME="e2e-test" ./hack/ci-e2e-kind-azure.sh
📌 NOTE: To target the newly created KinD cluster, please run the following command:
export KUBECONFIG=/Users/i585850/go/src/github.com/gardener/etcd-druid/hack/e2e-test/infrastructure/kind/kubeconfig
./hack/kind-up.sh
Creating cluster "etcd-druid-e2e" ...
✓ Ensuring node image (kindest/node:v1.27.1) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-etcd-druid-e2e"
You can now use your cluster with:
kubectl cluster-info --context kind-etcd-druid-e2e
Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/
node/etcd-druid-e2e-control-plane condition met
./hack/deploy-azurite.sh
deployment.apps/azurite created
service/azurite-service created
Waiting for deployment "azurite" rollout to finish: 0 of 1 updated replicas are available...
deployment "azurite" successfully rolled out
pod/azurite-95f4c6d9c-t6l8f condition met
namespace/e2e-test created
-------------------
Setting up environment
-------------------
namespace/e2e-test unchanged
Generating tags...
Checking cache...
Starting test...
Tags used in deployment:
Starting deploy...
Starting pre-deploy hooks...
Deleting previous job
No resources found
Completed pre-deploy hooks
Loading images into kind cluster nodes...
Images loaded in 166ns
- configmap/az-infra-common created
- configmap/az-run-script created
- secret/az-infra-secret created
- job.batch/az-infra-job created
Starting post-deploy hooks...
Wait until job is ready
job.batch/az-infra-job condition met
Completed post-deploy hooks
You can also run [skaffold run --tail] to get the logs
-------------------
Deploying Druid
-------------------
Generating tags...
- europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid -> europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid:v0.22.0-5-g573fcbfd-dirty
Checking cache...
ERRO[0003] gcloud binary not found
- europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid: Found Locally
Starting test...
Tags used in deployment:
- europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid -> europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid:987f5decabc12f2b5a3400d9932482549c5cb09eb44fd5ef912d6fa2774a1e9e
Starting deploy...
Loading images into kind cluster nodes...
- europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid:987f5decabc12f2b5a3400d9932482549c5cb09eb44fd5ef912d6fa2774a1e9e -> Loaded
Images loaded in 1.248 second
Helm release etcd-druid not installed. Installing...
NAME: etcd-druid
LAST DEPLOYED: Thu Feb 8 11:35:25 2024
NAMESPACE: e2e-test
STATUS: deployed
REVISION: 1
TEST SUITE: None
Waiting for deployments to stabilize...
- e2e-test:deployment/etcd-druid is ready.
Deployments stabilized in 1.087 second
You can also run [skaffold run --tail] to get the logs
-------------------
Running e2e tests
-------------------
=== RUN TestIntegration
Running Suite: e2e Suite - /Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e
========================================================================================
Random Seed: 1707372334
Will run 7 of 7 specs
------------------------------
[BeforeSuite]
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:72
> Enter [BeforeSuite] TOP-LEVEL @ 02/08/24 11:35:34.469
{"level":"info","ts":"2024-02-08T11:35:34+05:30","msg":"creating namespace","namespace":"shoot"}
{"level":"info","ts":"2024-02-08T11:35:34+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"ca-etcd-az"}}
{"level":"info","ts":"2024-02-08T11:35:34+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-server-cert-az"}}
{"level":"info","ts":"2024-02-08T11:35:34+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-client-tls-az"}}
< Exit [BeforeSuite] TOP-LEVEL @ 02/08/24 11:35:34.506 (38ms)
[BeforeSuite] PASSED [0.038 seconds]
------------------------------
Etcd Backup when single-node etcd is configured with provider az Should create, test backup and delete etcd with backup
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_backup_test.go:78
> Enter [BeforeEach] Etcd Backup @ 02/08/24 11:35:34.506
< Exit [BeforeEach] Etcd Backup @ 02/08/24 11:35:34.506 (0s)
> Enter [BeforeEach] with provider az @ 02/08/24 11:35:34.506
{"level":"info","ts":"2024-02-08T11:35:34+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-backup-az"}}
< Exit [BeforeEach] with provider az @ 02/08/24 11:35:34.519 (13ms)
> Enter [It] Should create, test backup and delete etcd with backup @ 02/08/24 11:35:34.519
STEP: Create etcd @ 02/08/24 11:35:34.519
{"level":"info","ts":"2024-02-08T11:35:34+05:30","msg":"Waiting for etcd to become ready","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:36:22+05:30","msg":"etcd is ready","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:36:22+05:30","msg":"Checking statefulset","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:36:22+05:30","msg":"Checking configmap","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:36:22+05:30","msg":"Checking client service","etcd":"shoot/etcd-az"}
STEP: Create debug pod @ 02/08/24 11:36:22.784
STEP: Check initial snapshot is available @ 02/08/24 11:36:34.842
STEP: Put keys into etcd @ 02/08/24 11:36:34.917
{"level":"info","ts":"2024-02-08T11:36:34+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"}
{"level":"info","ts":"2024-02-08T11:36:39+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"}
{"level":"info","ts":"2024-02-08T11:36:39+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"}
{"level":"info","ts":"2024-02-08T11:36:40+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"}
{"level":"info","ts":"2024-02-08T11:36:40+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"}
{"level":"info","ts":"2024-02-08T11:36:40+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"}
{"level":"info","ts":"2024-02-08T11:36:40+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"}
{"level":"info","ts":"2024-02-08T11:36:40+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"}
{"level":"info","ts":"2024-02-08T11:36:41+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"}
{"level":"info","ts":"2024-02-08T11:36:41+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"}
{"level":"info","ts":"2024-02-08T11:36:41+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"}
STEP: Check snapshot after putting data into etcd @ 02/08/24 11:36:41.456
STEP: Trigger on-demand full snapshot @ 02/08/24 11:36:41.547
STEP: Put additional data into etcd @ 02/08/24 11:36:41.608
{"level":"info","ts":"2024-02-08T11:36:41+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"}
{"level":"info","ts":"2024-02-08T11:36:45+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"}
{"level":"info","ts":"2024-02-08T11:36:45+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"}
{"level":"info","ts":"2024-02-08T11:36:45+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"}
{"level":"info","ts":"2024-02-08T11:36:45+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"}
{"level":"info","ts":"2024-02-08T11:36:46+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"}
STEP: Trigger on-demand delta snapshot @ 02/08/24 11:36:46.108
STEP: Test cluster restoration by deleting data directory @ 02/08/24 11:36:46.166
{"level":"info","ts":"2024-02-08T11:36:46+05:30","msg":"waiting for sts to become unready","statefulSetName":"etcd-az"}
{"level":"info","ts":"2024-02-08T11:37:12+05:30","msg":"sts is unready","statefulSetName":"etcd-az"}
{"level":"info","ts":"2024-02-08T11:37:12+05:30","msg":"waiting for sts to become ready again","statefulSetName":"etcd-az"}
{"level":"info","ts":"2024-02-08T11:37:30+05:30","msg":"sts is ready","statefulSetName":"etcd-az"}
{"level":"info","ts":"2024-02-08T11:37:30+05:30","msg":"fetching etcd key-value pairs"}
{"level":"info","ts":"2024-02-08T11:37:30+05:30","msg":"fetched key-value pair from etcd","key":"foo-1","value":"bar-1"}
{"level":"info","ts":"2024-02-08T11:37:30+05:30","msg":"fetched key-value pair from etcd","key":"foo-2","value":"bar-2"}
{"level":"info","ts":"2024-02-08T11:37:31+05:30","msg":"fetched key-value pair from etcd","key":"foo-3","value":"bar-3"}
{"level":"info","ts":"2024-02-08T11:37:31+05:30","msg":"fetched key-value pair from etcd","key":"foo-4","value":"bar-4"}
{"level":"info","ts":"2024-02-08T11:37:31+05:30","msg":"fetched key-value pair from etcd","key":"foo-5","value":"bar-5"}
{"level":"info","ts":"2024-02-08T11:37:31+05:30","msg":"fetched key-value pair from etcd","key":"foo-6","value":"bar-6"}
{"level":"info","ts":"2024-02-08T11:37:31+05:30","msg":"fetched key-value pair from etcd","key":"foo-7","value":"bar-7"}
{"level":"info","ts":"2024-02-08T11:37:31+05:30","msg":"fetched key-value pair from etcd","key":"foo-8","value":"bar-8"}
{"level":"info","ts":"2024-02-08T11:37:32+05:30","msg":"fetched key-value pair from etcd","key":"foo-9","value":"bar-9"}
{"level":"info","ts":"2024-02-08T11:37:32+05:30","msg":"fetched key-value pair from etcd","key":"foo-10","value":"bar-10"}
{"level":"info","ts":"2024-02-08T11:37:32+05:30","msg":"fetched key-value pair from etcd","key":"foo-11","value":"bar-11"}
{"level":"info","ts":"2024-02-08T11:37:32+05:30","msg":"fetched key-value pair from etcd","key":"foo-12","value":"bar-12"}
{"level":"info","ts":"2024-02-08T11:37:32+05:30","msg":"fetched key-value pair from etcd","key":"foo-13","value":"bar-13"}
{"level":"info","ts":"2024-02-08T11:37:33+05:30","msg":"fetched key-value pair from etcd","key":"foo-14","value":"bar-14"}
{"level":"info","ts":"2024-02-08T11:37:33+05:30","msg":"fetched key-value pair from etcd","key":"foo-15","value":"bar-15"}
STEP: Delete debug pod @ 02/08/24 11:37:33.258
STEP: Delete etcd @ 02/08/24 11:37:33.261
{"level":"info","ts":"2024-02-08T11:37:33+05:30","msg":"Checking if etcd is gone","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:37:35+05:30","msg":"Checking if statefulset is gone","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:37:35+05:30","msg":"Checking if configmap is gone","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:37:35+05:30","msg":"Checking client service is gone","etcd":"shoot/etcd-az"}
< Exit [It] Should create, test backup and delete etcd with backup @ 02/08/24 11:37:35.294 (2m0.774s)
• [120.787 seconds]
------------------------------
Etcd when multi-node is configured should perform etcd operations
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:87
> Enter [BeforeEach] Etcd @ 02/08/24 11:37:35.294
{"level":"info","ts":"2024-02-08T11:37:35+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-backup-az"}}
< Exit [BeforeEach] Etcd @ 02/08/24 11:37:35.318 (24ms)
> Enter [It] should perform etcd operations @ 02/08/24 11:37:35.318
STEP: Create etcd @ 02/08/24 11:37:35.318
{"level":"info","ts":"2024-02-08T11:37:35+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:39:44+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:39:44+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:39:44+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:39:44+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Hibernate etcd (Scale down from 3 replicas to 0) @ 02/08/24 11:39:44.03
{"level":"info","ts":"2024-02-08T11:39:44+05:30","msg":"Waiting to hibernate","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:39:44+05:30","msg":"Checking etcd","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:39:46+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:39:46+05:30","msg":"etcd is hibernated","etcd-multi-node":"shoot/etcd-az"}
STEP: Wakeup etcd (Scale up from 0->3 replicas) @ 02/08/24 11:39:46.058
{"level":"info","ts":"2024-02-08T11:41:50+05:30","msg":"Waiting for etcd to become ready"}
{"level":"info","ts":"2024-02-08T11:42:06+05:30","msg":"etcd is ready"}
{"level":"info","ts":"2024-02-08T11:42:06+05:30","msg":"Checking statefulset"}
{"level":"info","ts":"2024-02-08T11:42:06+05:30","msg":"Checking configmap"}
{"level":"info","ts":"2024-02-08T11:42:06+05:30","msg":"Checking client service"}
STEP: Zero downtime rolling updates @ 02/08/24 11:42:06.84
{"level":"info","ts":"2024-02-08T11:42:06+05:30","msg":"Creating job to ensure etcd zero downtime","job":"etcd-zero-down-time-validator-rolling-update"}
{"level":"info","ts":"2024-02-08T11:42:24+05:30","msg":"Job is ready","job":"etcd-zero-down-time-validator-rolling-update"}
{"level":"info","ts":"2024-02-08T11:43:31+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:43:31+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:43:31+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:43:31+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:43:31+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:43:31+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd-multi-node":"shoot/etcd-az"}
STEP: Zero downtime maintenance operation: defragmentation @ 02/08/24 11:43:31.263
{"level":"info","ts":"2024-02-08T11:43:31+05:30","msg":"Configure defragmentation schedule for every 1 minute","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:44:15+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:44:35+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:44:35+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:44:35+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:44:35+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:44:35+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:44:35+05:30","msg":"Waiting for defragmentation to finish","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:00+05:30","msg":"Defragmentation is finished","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:00+05:30","msg":"Checking any Etcd downtime","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:00+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd-multi-node":"shoot/etcd-az"}
STEP: Member restart with data-dir/pvc intact @ 02/08/24 11:46:00.521
{"level":"info","ts":"2024-02-08T11:46:00+05:30","msg":"Delete one member pod","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:00+05:30","msg":"waiting for sts to become unready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:02+05:30","msg":"sts is unready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:02+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:16+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:16+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:16+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:16+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Single member restoration @ 02/08/24 11:46:16.664
{"level":"info","ts":"2024-02-08T11:46:16+05:30","msg":"Create debug pod","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:20+05:30","msg":"Delete member dir of one member pod","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:20+05:30","msg":"waiting for sts to become unready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:50+05:30","msg":"sts is unready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:04+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:04+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:04+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:04+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Delete debug pod @ 02/08/24 11:47:04.944
STEP: Delete etcd @ 02/08/24 11:47:04.95
{"level":"info","ts":"2024-02-08T11:47:04+05:30","msg":"Checking if etcd is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:06+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:06+05:30","msg":"Checking if configmap is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:06+05:30","msg":"Checking client service is gone","etcd-multi-node":"shoot/etcd-az"}
< Exit [It] should perform etcd operations @ 02/08/24 11:47:07.01 (9m31.706s)
> Enter [AfterEach] Etcd @ 02/08/24 11:47:07.01
< Exit [AfterEach] Etcd @ 02/08/24 11:47:07.013 (3ms)
• [571.733 seconds]
------------------------------
Etcd when a single-node is configured should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS not enabled for peerUrl)
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:152
> Enter [BeforeEach] Etcd @ 02/08/24 11:47:07.013
{"level":"info","ts":"2024-02-08T11:47:07+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-backup-az"}}
< Exit [BeforeEach] Etcd @ 02/08/24 11:47:07.035 (21ms)
> Enter [It] should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS not enabled for peerUrl) @ 02/08/24 11:47:07.035
STEP: Creating a single-node etcd @ 02/08/24 11:47:07.035
{"level":"info","ts":"2024-02-08T11:47:07+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:35+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:35+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:35+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:47:35+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling up a healthy cluster (from 1 to 3 replicas) @ 02/08/24 11:47:35.198
{"level":"info","ts":"2024-02-08T11:48:45+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:48:45+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:48:45+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:48:45+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:48:45+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Deleting etcd @ 02/08/24 11:48:45.593
{"level":"info","ts":"2024-02-08T11:48:45+05:30","msg":"Checking if etcd is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:48:51+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:48:51+05:30","msg":"Checking if configmap is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:48:51+05:30","msg":"Checking client service is gone","etcd-multi-node":"shoot/etcd-az"}
< Exit [It] should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS not enabled for peerUrl) @ 02/08/24 11:48:51.679 (1m44.644s)
> Enter [AfterEach] Etcd @ 02/08/24 11:48:51.679
< Exit [AfterEach] Etcd @ 02/08/24 11:48:51.683 (5ms)
• [104.671 seconds]
------------------------------
Etcd when a single-node is configured should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS enabled for peerUrl)
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:171
> Enter [BeforeEach] Etcd @ 02/08/24 11:48:51.683
{"level":"info","ts":"2024-02-08T11:48:51+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-backup-az"}}
< Exit [BeforeEach] Etcd @ 02/08/24 11:48:51.7 (17ms)
> Enter [It] should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS enabled for peerUrl) @ 02/08/24 11:48:51.7
STEP: Creating a single-node etcd @ 02/08/24 11:48:51.7
{"level":"info","ts":"2024-02-08T11:48:51+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:49:19+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:49:19+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:49:19+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:49:19+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling up a healthy cluster (from 1 to 3 replicas) with TLS enabled for peerUrl @ 02/08/24 11:49:19.879
{"level":"info","ts":"2024-02-08T11:51:58+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:04+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:04+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:04+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:04+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Deleting the single-node etcd @ 02/08/24 11:52:04.689
{"level":"info","ts":"2024-02-08T11:52:04+05:30","msg":"Checking if etcd is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:06+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:06+05:30","msg":"Checking if configmap is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:06+05:30","msg":"Checking client service is gone","etcd-multi-node":"shoot/etcd-az"}
< Exit [It] should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS enabled for peerUrl) @ 02/08/24 11:52:06.738 (3m15.039s)
> Enter [AfterEach] Etcd @ 02/08/24 11:52:06.738
< Exit [AfterEach] Etcd @ 02/08/24 11:52:06.741 (3ms)
• [195.059 seconds]
------------------------------
Etcd when a single-node is configured should scale down a single-node etcd to 0, then scale up from 0->1 replicas and then from 1->3 replicas
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:191
> Enter [BeforeEach] Etcd @ 02/08/24 11:52:06.742
{"level":"info","ts":"2024-02-08T11:52:06+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-backup-az"}}
< Exit [BeforeEach] Etcd @ 02/08/24 11:52:06.755 (13ms)
> Enter [It] should scale down a single-node etcd to 0, then scale up from 0->1 replicas and then from 1->3 replicas @ 02/08/24 11:52:06.755
STEP: Creating a single-node etcd @ 02/08/24 11:52:06.755
{"level":"info","ts":"2024-02-08T11:52:06+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:34+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:34+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:34+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:34+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 02/08/24 11:52:34.926
{"level":"info","ts":"2024-02-08T11:52:40+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:41+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:41+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:41+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:52:41+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling up cluster (from 0 to 1 replica) @ 02/08/24 11:52:41.014
{"level":"info","ts":"2024-02-08T11:53:07+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:53:07+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:53:07+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:53:07+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:53:07+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling up a healthy cluster (from 1 to 3 replica) @ 02/08/24 11:53:07.191
{"level":"info","ts":"2024-02-08T11:54:23+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:23+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:23+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:23+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:23+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Deleting the single-node etcd @ 02/08/24 11:54:23.637
{"level":"info","ts":"2024-02-08T11:54:23+05:30","msg":"Checking if etcd is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:29+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:29+05:30","msg":"Checking if configmap is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:29+05:30","msg":"Checking client service is gone","etcd-multi-node":"shoot/etcd-az"}
< Exit [It] should scale down a single-node etcd to 0, then scale up from 0->1 replicas and then from 1->3 replicas @ 02/08/24 11:54:29.71 (2m22.956s)
> Enter [AfterEach] Etcd @ 02/08/24 11:54:29.71
< Exit [AfterEach] Etcd @ 02/08/24 11:54:29.713 (3ms)
• [142.972 seconds]
------------------------------
Etcd when a single-node is configured should scale down a single-node etcd to 0 replica, then scale up from 0->1 replica and then from 1->3 replicas with TLS enabled for cluster peerUrl
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:220
> Enter [BeforeEach] Etcd @ 02/08/24 11:54:29.713
{"level":"info","ts":"2024-02-08T11:54:29+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-backup-az"}}
< Exit [BeforeEach] Etcd @ 02/08/24 11:54:29.727 (13ms)
> Enter [It] should scale down a single-node etcd to 0 replica, then scale up from 0->1 replica and then from 1->3 replicas with TLS enabled for cluster peerUrl @ 02/08/24 11:54:29.727
STEP: Creating a single-node etcd @ 02/08/24 11:54:29.727
{"level":"info","ts":"2024-02-08T11:54:29+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:57+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:57+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:57+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:57+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 02/08/24 11:54:57.922
{"level":"info","ts":"2024-02-08T11:54:59+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:59+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:59+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:59+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:54:59+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling up cluster (from 0 to 1 replica) @ 02/08/24 11:54:59.986
{"level":"info","ts":"2024-02-08T11:55:26+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:55:26+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:55:26+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:55:26+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T11:55:26+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Scaling up a healthy cluster (from 1 to 3 replica) with TLS enabled for peerUrl @ 02/08/24 11:55:26.159
{"level":"info","ts":"2024-02-08T12:00:21+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:21+05:30","msg":"etcd is ready","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:21+05:30","msg":"Checking statefulset","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:21+05:30","msg":"Checking configmap","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:21+05:30","msg":"Checking client service","etcd-multi-node":"shoot/etcd-az"}
STEP: Deleting the single-node etcd @ 02/08/24 12:00:21.911
{"level":"info","ts":"2024-02-08T12:00:21+05:30","msg":"Checking if etcd is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:27+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:27+05:30","msg":"Checking if configmap is gone","etcd-multi-node":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:27+05:30","msg":"Checking client service is gone","etcd-multi-node":"shoot/etcd-az"}
< Exit [It] should scale down a single-node etcd to 0 replica, then scale up from 0->1 replica and then from 1->3 replicas with TLS enabled for cluster peerUrl @ 02/08/24 12:00:27.977 (5m58.257s)
> Enter [AfterEach] Etcd @ 02/08/24 12:00:27.977
< Exit [AfterEach] Etcd @ 02/08/24 12:00:27.978 (2ms)
• [358.272 seconds]
------------------------------
Etcd Compaction when compaction is enabled for single-node etcd with provider az should test compaction on backup
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_compaction_test.go:71
> Enter [BeforeEach] Etcd Compaction @ 02/08/24 12:00:27.978
< Exit [BeforeEach] Etcd Compaction @ 02/08/24 12:00:27.978 (0s)
> Enter [BeforeEach] with provider az @ 02/08/24 12:00:27.978
{"level":"info","ts":"2024-02-08T12:00:27+05:30","msg":"creating secret","secret":{"namespace":"shoot","name":"etcd-backup-az"}}
< Exit [BeforeEach] with provider az @ 02/08/24 12:00:28 (22ms)
> Enter [It] should test compaction on backup @ 02/08/24 12:00:28
STEP: Create etcd @ 02/08/24 12:00:28
{"level":"info","ts":"2024-02-08T12:00:28+05:30","msg":"Waiting for etcd to become ready","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:56+05:30","msg":"etcd is ready","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:56+05:30","msg":"Checking statefulset","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:56+05:30","msg":"Checking configmap","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:00:56+05:30","msg":"Checking client service","etcd":"shoot/etcd-az"}
STEP: Create debug pod @ 02/08/24 12:00:56.159
STEP: Check initial snapshot is available @ 02/08/24 12:01:00.187
STEP: Put keys into etcd @ 02/08/24 12:01:00.286
{"level":"info","ts":"2024-02-08T12:01:00+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"}
{"level":"info","ts":"2024-02-08T12:01:04+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"}
{"level":"info","ts":"2024-02-08T12:01:05+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"}
{"level":"info","ts":"2024-02-08T12:01:05+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"}
{"level":"info","ts":"2024-02-08T12:01:05+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"}
{"level":"info","ts":"2024-02-08T12:01:05+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"}
{"level":"info","ts":"2024-02-08T12:01:05+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"}
{"level":"info","ts":"2024-02-08T12:01:06+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"}
{"level":"info","ts":"2024-02-08T12:01:06+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"}
{"level":"info","ts":"2024-02-08T12:01:06+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"}
{"level":"info","ts":"2024-02-08T12:01:06+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"}
STEP: Check snapshot after putting data into etcd @ 02/08/24 12:01:06.588
STEP: Put additional data into etcd @ 02/08/24 12:01:06.68
{"level":"info","ts":"2024-02-08T12:01:06+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"}
{"level":"info","ts":"2024-02-08T12:01:09+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"}
{"level":"info","ts":"2024-02-08T12:01:09+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"}
{"level":"info","ts":"2024-02-08T12:01:10+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"}
{"level":"info","ts":"2024-02-08T12:01:10+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"}
{"level":"info","ts":"2024-02-08T12:01:10+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"}
STEP: Trigger on-demand delta snapshot @ 02/08/24 12:01:10.508
{"level":"info","ts":"2024-02-08T12:01:10+05:30","msg":"waiting for compaction job to become successful"}
{"level":"info","ts":"2024-02-08T12:01:44+05:30","msg":"compaction job is successful"}
STEP: Verify that all the delta snapshots are compacted to full snapshots by compaction triggerred at first 15th revision @ 02/08/24 12:01:44.692
STEP: Put additional data into etcd @ 02/08/24 12:01:44.763
{"level":"info","ts":"2024-02-08T12:01:44+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-16","fromValue":"bar-16","toKey":"foo-20","toValue":"bar-20"}
{"level":"info","ts":"2024-02-08T12:01:47+05:30","msg":"put key-value successful","key":"foo-16","value":"bar-16"}
{"level":"info","ts":"2024-02-08T12:01:48+05:30","msg":"put key-value successful","key":"foo-17","value":"bar-17"}
{"level":"info","ts":"2024-02-08T12:01:48+05:30","msg":"put key-value successful","key":"foo-18","value":"bar-18"}
{"level":"info","ts":"2024-02-08T12:01:48+05:30","msg":"put key-value successful","key":"foo-19","value":"bar-19"}
{"level":"info","ts":"2024-02-08T12:01:48+05:30","msg":"put key-value successful","key":"foo-20","value":"bar-20"}
STEP: Trigger on-demand delta snapshot @ 02/08/24 12:01:48.665
STEP: Verify that there are new delta snapshots as compaction is not triggered yet because delta events have not reached next 15 revision @ 02/08/24 12:01:48.719
STEP: Delete debug pod @ 02/08/24 12:01:48.758
STEP: Delete etcd @ 02/08/24 12:01:48.761
{"level":"info","ts":"2024-02-08T12:01:48+05:30","msg":"Checking if etcd is gone","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:01:54+05:30","msg":"Checking if statefulset is gone","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:01:54+05:30","msg":"Checking if configmap is gone","etcd":"shoot/etcd-az"}
{"level":"info","ts":"2024-02-08T12:01:54+05:30","msg":"Checking client service is gone","etcd":"shoot/etcd-az"}
< Exit [It] should test compaction on backup @ 02/08/24 12:01:54.829 (1m26.83s)
• [86.851 seconds]
------------------------------
[AfterSuite]
/Users/i585850/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:108
> Enter [AfterSuite] TOP-LEVEL @ 02/08/24 12:01:54.829
{"level":"info","ts":"2024-02-08T12:01:54+05:30","msg":"deleting namespace","namespace":"shoot","namespace":"shoot"}
< Exit [AfterSuite] TOP-LEVEL @ 02/08/24 12:02:00.872 (6.043s)
[AfterSuite] PASSED [6.044 seconds]
------------------------------
Ran 7 of 7 Specs in 1586.428 seconds
SUCCESS! -- 7 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestIntegration (1586.43s)
PASS
ok github.com/gardener/etcd-druid/test/e2e 1586.477s
hack/tools/bin/kind delete cluster --name etcd-druid-e2e
Deleting cluster "etcd-druid-e2e" ...
Deleted nodes: ["etcd-druid-e2e-control-plane"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment