Skip to content

Instantly share code, notes, and snippets.

@rajguptavm
Last active April 20, 2024 09:23
Show Gist options
  • Save rajguptavm/df3997573f7729e7df6de4560a1095e3 to your computer and use it in GitHub Desktop.
Save rajguptavm/df3997573f7729e7df6de4560a1095e3 to your computer and use it in GitHub Desktop.
multi-svc sts test fix
Starting: /Users/rajgupta/go/bin/dlv dap --listen=127.0.0.1:52927 --log-dest=3 from /Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e
DAP server listening at: 127.0.0.1:52927
W0420 13:01:00.986091 45102 test_context.go:517] Unable to find in-cluster config, using default host : https://127.0.0.1:6443
Apr 20 13:01:00.986: INFO: The --provider flag is not set. Continuing as if --provider=skeleton had been used.
Running Suite: CNS CSI Driver End-to-End Tests - /Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e
==========================================================================================================
Random Seed: 1713598260
Will run 1 of 849 specs
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[csi-multi-svc] Multi-SVC [csi-multi-svc] Workload creation on each of the clusters [p0, wcp, multiSvc, newTest]
/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/multi_svc_test.go:239
Apr 20 13:01:01.012: INFO: Creating new VC session
Apr 20 13:01:02.786: INFO: To get number of compute cluster and it's path - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc namespace.cluster.ls
Apr 20 13:01:10.011: INFO: storage policy id: fd4ab249-959f-4f07-a1a9-560f3779c4e5 for storage policy name is: shared-ds-policy
Apr 20 13:01:10.875: INFO: storage policy id: 01692d8a-ad49-4fa4-872a-de3361c4ebbb for storage policy name is: shared-ds-policy1
STEP Getting User and Supervisor-Id for both the supervisors @ 04/20/24 13:01:13.423
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
STEP Create StatefulSet with 3 replicas with parallel pod management @ 04/20/24 13:01:13.935
STEP Create service @ 04/20/24 13:01:13.935
Apr 20 13:01:13.935: INFO: Parsing service from tests/e2e/testing-manifests/statefulset/nginx/service.yaml
Apr 20 13:01:14.554: INFO: Create StatefulSet
Apr 20 13:01:14.554: INFO: Preparing StatefulSet Spec
Apr 20 13:01:14.554: INFO: Parsing statefulset from tests/e2e/testing-manifests/statefulset/nginx/statefulset.yaml
Apr 20 13:01:14.563: INFO: Creating statefulset
Apr 20 13:01:14.563: INFO: Creating statefulset e2e-test-namespace/web with 3 replicas and selector &LabelSelector{MatchLabels:map[string]string{app: nginx,},MatchExpressions:[]LabelSelectorRequirement{},}
Apr 20 13:01:15.392: INFO: Found 0 stateful pods, waiting for 3
Apr 20 13:01:25.875: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:01:35.866: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:01:45.897: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:01:55.890: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:02:05.955: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:02:15.861: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:02:15.861: INFO: Waiting for pod web-1 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:02:15.861: INFO: Waiting for pod web-2 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:02:25.856: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:02:25.856: INFO: Waiting for pod web-1 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:02:25.856: INFO: Waiting for pod web-2 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:02:25.856: INFO: Wait for StatefulSet pods to be in up and running state
Apr 20 13:02:25.856: INFO: Waiting for statefulset status.replicas updated to 3
Apr 20 13:02:26.336: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-0 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 13:02:28.904: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 13:02:28.905: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 07:32 /usr/share/nginx/html\n"
Apr 20 13:02:28.905: INFO: stdout of ls -idlh /usr/share/nginx/html on web-0: 2 drwxr-xr-x 3 root root 4.0K Apr 20 07:32 /usr/share/nginx/html
Apr 20 13:02:28.905: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-1 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 13:02:31.459: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 13:02:31.459: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 07:32 /usr/share/nginx/html\n"
Apr 20 13:02:31.459: INFO: stdout of ls -idlh /usr/share/nginx/html on web-1: 2 drwxr-xr-x 3 root root 4.0K Apr 20 07:32 /usr/share/nginx/html
Apr 20 13:02:31.459: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-2 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 13:02:34.055: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 13:02:34.055: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 07:32 /usr/share/nginx/html\n"
Apr 20 13:02:34.055: INFO: stdout of ls -idlh /usr/share/nginx/html on web-2: 2 drwxr-xr-x 3 root root 4.0K Apr 20 07:32 /usr/share/nginx/html
Apr 20 13:02:34.526: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-0 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 13:02:37.146: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 13:02:37.146: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 13:02:37.146: INFO: stdout of find /usr/share/nginx/html on web-0: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 13:02:37.147: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-1 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 13:02:39.641: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 13:02:39.642: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 13:02:39.642: INFO: stdout of find /usr/share/nginx/html on web-1: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 13:02:39.642: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-2 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 13:02:42.354: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 13:02:42.354: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 13:02:42.354: INFO: stdout of find /usr/share/nginx/html on web-2: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 13:02:42.816: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-0 -- /bin/sh -x -c touch /usr/share/nginx/html/1713598346086311000'
Apr 20 13:02:45.210: INFO: stderr: "+ touch /usr/share/nginx/html/1713598346086311000\n"
Apr 20 13:02:45.210: INFO: stdout: ""
Apr 20 13:02:45.210: INFO: stdout of touch /usr/share/nginx/html/1713598346086311000 on web-0:
Apr 20 13:02:45.210: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-1 -- /bin/sh -x -c touch /usr/share/nginx/html/1713598346086311000'
Apr 20 13:02:47.674: INFO: stderr: "+ touch /usr/share/nginx/html/1713598346086311000\n"
Apr 20 13:02:47.674: INFO: stdout: ""
Apr 20 13:02:47.674: INFO: stdout of touch /usr/share/nginx/html/1713598346086311000 on web-1:
Apr 20 13:02:47.675: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-2 -- /bin/sh -x -c touch /usr/share/nginx/html/1713598346086311000'
Apr 20 13:02:50.365: INFO: stderr: "+ touch /usr/share/nginx/html/1713598346086311000\n"
Apr 20 13:02:50.365: INFO: stdout: ""
Apr 20 13:02:50.365: INFO: stdout of touch /usr/share/nginx/html/1713598346086311000 on web-2:
Apr 20 13:02:50.863: INFO: Scale up sts replica count to 5
STEP Scaling up statefulsets to number of Replica: 5 @ 04/20/24 13:02:50.863
Apr 20 13:02:50.863: INFO: Scaling statefulset web to 5
Apr 20 13:02:52.133: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 13:02:52.434: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 13:02:52.745: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 13:03:03.005: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 13:03:13.012: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 13:03:23.079: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 4
STEP Verify all volumes are attached to Nodes after Statefulsets is scaled up @ 04/20/24 13:03:33.495
STEP Verify volume: cbc2f035-369a-489b-80f7-c6ef1f3674d6 is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 13:03:34.517
Apr 20 13:03:37.994: INFO: vmRef: VirtualMachine:vm-177 for the VM uuid: 503b43e8-8465-00cb-b5a1-9597bcfdee46
Apr 20 13:03:38.546: INFO: Found FCDID "cbc2f035-369a-489b-80f7-c6ef1f3674d6" attached to VM "web-0"
Apr 20 13:03:38.546: INFO: Found the disk "cbc2f035-369a-489b-80f7-c6ef1f3674d6" is attached to the VM with UUID: "503b43e8-8465-00cb-b5a1-9597bcfdee46"
STEP successfully verified metadata of the volume "cbc2f035-369a-489b-80f7-c6ef1f3674d6" @ 04/20/24 13:03:39.641
STEP Verify volume: 25951f22-4802-4312-a7ff-687073cb9fec is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 13:03:40.662
Apr 20 13:03:43.217: INFO: vmRef: VirtualMachine:vm-178 for the VM uuid: 503b80fb-0456-5ab4-720a-40b75658ffe5
Apr 20 13:03:43.837: INFO: Found FCDID "25951f22-4802-4312-a7ff-687073cb9fec" attached to VM "web-1"
Apr 20 13:03:43.837: INFO: Found the disk "25951f22-4802-4312-a7ff-687073cb9fec" is attached to the VM with UUID: "503b80fb-0456-5ab4-720a-40b75658ffe5"
STEP successfully verified metadata of the volume "25951f22-4802-4312-a7ff-687073cb9fec" @ 04/20/24 13:03:44.866
STEP Verify volume: 5642c5a2-9eb1-411e-8558-d32135ef165b is attached to the node: sc2-10-187-97-114.nimbus.eng.vmware.com @ 04/20/24 13:03:45.929
Apr 20 13:03:48.251: INFO: vmRef: VirtualMachine:vm-179 for the VM uuid: 503baf82-cda5-ac6e-795f-536bc92c16a7
Apr 20 13:03:48.754: INFO: Found FCDID "5642c5a2-9eb1-411e-8558-d32135ef165b" attached to VM "web-2"
Apr 20 13:03:48.754: INFO: Found the disk "5642c5a2-9eb1-411e-8558-d32135ef165b" is attached to the VM with UUID: "503baf82-cda5-ac6e-795f-536bc92c16a7"
STEP successfully verified metadata of the volume "5642c5a2-9eb1-411e-8558-d32135ef165b" @ 04/20/24 13:03:49.774
STEP Verify volume: 9bb1b3f5-8865-476d-b6f9-f0c4af0af036 is attached to the node: sc2-10-187-102-243.nimbus.eng.vmware.com @ 04/20/24 13:03:50.903
Apr 20 13:03:53.765: INFO: vmRef: VirtualMachine:vm-180 for the VM uuid: 503bb636-37cc-0bed-93e7-cc4e971e5033
Apr 20 13:03:54.384: INFO: Found FCDID "9bb1b3f5-8865-476d-b6f9-f0c4af0af036" attached to VM "web-3"
Apr 20 13:03:54.384: INFO: Found the disk "9bb1b3f5-8865-476d-b6f9-f0c4af0af036" is attached to the VM with UUID: "503bb636-37cc-0bed-93e7-cc4e971e5033"
STEP successfully verified metadata of the volume "9bb1b3f5-8865-476d-b6f9-f0c4af0af036" @ 04/20/24 13:03:55.917
STEP Verify volume: e63c1206-3705-4944-92f3-e2cb7b869522 is attached to the node: sc2-10-187-104-167.nimbus.eng.vmware.com @ 04/20/24 13:03:56.901
Apr 20 13:03:59.705: INFO: vmRef: VirtualMachine:vm-181 for the VM uuid: 503bc6a0-a9a2-4165-1a0d-76c748cef420
Apr 20 13:04:00.224: INFO: Found FCDID "e63c1206-3705-4944-92f3-e2cb7b869522" attached to VM "web-4"
Apr 20 13:04:00.224: INFO: Found the disk "e63c1206-3705-4944-92f3-e2cb7b869522" is attached to the VM with UUID: "503bc6a0-a9a2-4165-1a0d-76c748cef420"
STEP successfully verified metadata of the volume "e63c1206-3705-4944-92f3-e2cb7b869522" @ 04/20/24 13:04:01.437
Apr 20 13:04:01.437: INFO: Scale down sts replica count to 1
STEP Scaling down statefulsets to number of Replica: 1 @ 04/20/24 13:04:01.437
Apr 20 13:04:01.437: INFO: Scaling statefulset web to 1
Apr 20 13:04:23.667: INFO: Waiting for statefulset status.replicas updated to 1
STEP Verify Volumes are detached from Nodes after Statefulsets is scaled down @ 04/20/24 13:04:24.291
STEP successfully verified metadata of the volume "cbc2f035-369a-489b-80f7-c6ef1f3674d6" @ 04/20/24 13:04:26.534
STEP Create StatefulSet with 3 replicas with parallel pod management @ 04/20/24 13:04:26.539
STEP Create service @ 04/20/24 13:04:26.539
Apr 20 13:04:26.539: INFO: Parsing service from tests/e2e/testing-manifests/statefulset/nginx/service.yaml
Apr 20 13:04:27.277: INFO: Create StatefulSet
Apr 20 13:04:27.277: INFO: Preparing StatefulSet Spec
Apr 20 13:04:27.277: INFO: Parsing statefulset from tests/e2e/testing-manifests/statefulset/nginx/statefulset.yaml
Apr 20 13:04:27.280: INFO: Creating statefulset
Apr 20 13:04:27.280: INFO: Creating statefulset e2e-test-namespace1/web with 3 replicas and selector &LabelSelector{MatchLabels:map[string]string{app: nginx,},MatchExpressions:[]LabelSelectorRequirement{},}
Apr 20 13:04:27.863: INFO: Found 0 stateful pods, waiting for 3
Apr 20 13:04:38.322: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:04:48.340: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 13:04:58.487: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:04:58.488: INFO: Waiting for pod web-1 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:04:58.488: INFO: Waiting for pod web-2 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 13:04:58.488: INFO: Wait for StatefulSet pods to be in up and running state
Apr 20 13:04:58.488: INFO: Waiting for statefulset status.replicas updated to 3
Apr 20 13:04:59.030: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-0 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 13:05:01.912: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 13:05:01.913: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 07:34 /usr/share/nginx/html\n"
Apr 20 13:05:01.913: INFO: stdout of ls -idlh /usr/share/nginx/html on web-0: 2 drwxr-xr-x 3 root root 4.0K Apr 20 07:34 /usr/share/nginx/html
Apr 20 13:05:01.914: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-1 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 13:05:04.415: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 13:05:04.415: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 07:34 /usr/share/nginx/html\n"
Apr 20 13:05:04.415: INFO: stdout of ls -idlh /usr/share/nginx/html on web-1: 2 drwxr-xr-x 3 root root 4.0K Apr 20 07:34 /usr/share/nginx/html
Apr 20 13:05:04.415: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-2 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 13:05:06.945: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 13:05:06.945: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 07:34 /usr/share/nginx/html\n"
Apr 20 13:05:06.945: INFO: stdout of ls -idlh /usr/share/nginx/html on web-2: 2 drwxr-xr-x 3 root root 4.0K Apr 20 07:34 /usr/share/nginx/html
Apr 20 13:05:07.497: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-0 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 13:05:10.252: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 13:05:10.252: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 13:05:10.252: INFO: stdout of find /usr/share/nginx/html on web-0: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 13:05:10.252: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-1 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 13:05:12.682: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 13:05:12.682: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 13:05:12.682: INFO: stdout of find /usr/share/nginx/html on web-1: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 13:05:12.682: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-2 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 13:05:15.474: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 13:05:15.475: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 13:05:15.475: INFO: stdout of find /usr/share/nginx/html on web-2: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 13:05:15.939: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-0 -- /bin/sh -x -c touch /usr/share/nginx/html/1713598498791420000'
Apr 20 13:05:18.396: INFO: stderr: "+ touch /usr/share/nginx/html/1713598498791420000\n"
Apr 20 13:05:18.396: INFO: stdout: ""
Apr 20 13:05:18.397: INFO: stdout of touch /usr/share/nginx/html/1713598498791420000 on web-0:
Apr 20 13:05:18.397: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-1 -- /bin/sh -x -c touch /usr/share/nginx/html/1713598498791420000'
Apr 20 13:05:20.769: INFO: stderr: "+ touch /usr/share/nginx/html/1713598498791420000\n"
Apr 20 13:05:20.769: INFO: stdout: ""
Apr 20 13:05:20.769: INFO: stdout of touch /usr/share/nginx/html/1713598498791420000 on web-1:
Apr 20 13:05:20.769: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-2 -- /bin/sh -x -c touch /usr/share/nginx/html/1713598498791420000'
Apr 20 13:05:23.068: INFO: stderr: "+ touch /usr/share/nginx/html/1713598498791420000\n"
Apr 20 13:05:23.068: INFO: stdout: ""
Apr 20 13:05:23.068: INFO: stdout of touch /usr/share/nginx/html/1713598498791420000 on web-2:
Apr 20 13:05:23.562: INFO: Scale up sts replica count to 5
STEP Scaling up statefulsets to number of Replica: 5 @ 04/20/24 13:05:23.562
Apr 20 13:05:23.562: INFO: Scaling statefulset web to 5
Apr 20 13:05:24.393: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 13:05:24.631: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 13:05:24.862: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 13:05:35.100: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 13:05:45.095: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
STEP Verify all volumes are attached to Nodes after Statefulsets is scaled up @ 04/20/24 13:05:55.564
STEP Verify volume: d935c9b3-c3c4-4ff2-b5b9-3989cddf6336 is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 13:05:56.528
Apr 20 13:05:59.000: INFO: vmRef: VirtualMachine:vm-184 for the VM uuid: 503bd36a-abaa-c23a-3355-c3675a8c9098
Apr 20 13:05:59.469: INFO: Found FCDID "d935c9b3-c3c4-4ff2-b5b9-3989cddf6336" attached to VM "web-0"
Apr 20 13:05:59.469: INFO: Found the disk "d935c9b3-c3c4-4ff2-b5b9-3989cddf6336" is attached to the VM with UUID: "503bd36a-abaa-c23a-3355-c3675a8c9098"
STEP successfully verified metadata of the volume "d935c9b3-c3c4-4ff2-b5b9-3989cddf6336" @ 04/20/24 13:06:01.404
STEP Verify volume: 103e9e19-e508-4019-8928-6d7442c71c27 is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 13:06:02.567
Apr 20 13:06:04.868: INFO: vmRef: VirtualMachine:vm-183 for the VM uuid: 503b6f51-e198-c406-aa62-b84a645e13ad
Apr 20 13:06:05.334: INFO: Found FCDID "103e9e19-e508-4019-8928-6d7442c71c27" attached to VM "web-1"
Apr 20 13:06:05.334: INFO: Found the disk "103e9e19-e508-4019-8928-6d7442c71c27" is attached to the VM with UUID: "503b6f51-e198-c406-aa62-b84a645e13ad"
STEP successfully verified metadata of the volume "103e9e19-e508-4019-8928-6d7442c71c27" @ 04/20/24 13:06:07.114
STEP Verify volume: 4439884c-d664-4368-b5fd-de0034fe25b2 is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 13:06:08.047
Apr 20 13:06:10.711: INFO: vmRef: VirtualMachine:vm-182 for the VM uuid: 503ba9f2-806d-66e0-5ac4-95608204eabb
Apr 20 13:06:11.179: INFO: Found FCDID "4439884c-d664-4368-b5fd-de0034fe25b2" attached to VM "web-2"
Apr 20 13:06:11.179: INFO: Found the disk "4439884c-d664-4368-b5fd-de0034fe25b2" is attached to the VM with UUID: "503ba9f2-806d-66e0-5ac4-95608204eabb"
STEP successfully verified metadata of the volume "4439884c-d664-4368-b5fd-de0034fe25b2" @ 04/20/24 13:06:13.03
STEP Verify volume: 932dc121-a294-44c9-bf19-810133fa7805 is attached to the node: sc2-10-187-104-186.nimbus.eng.vmware.com @ 04/20/24 13:06:14.155
Apr 20 13:06:16.637: INFO: vmRef: VirtualMachine:vm-186 for the VM uuid: 503bae1b-660a-70ae-1abd-d2f239090da7
Apr 20 13:06:17.101: INFO: Found FCDID "932dc121-a294-44c9-bf19-810133fa7805" attached to VM "web-3"
Apr 20 13:06:17.101: INFO: Found the disk "932dc121-a294-44c9-bf19-810133fa7805" is attached to the VM with UUID: "503bae1b-660a-70ae-1abd-d2f239090da7"
STEP successfully verified metadata of the volume "932dc121-a294-44c9-bf19-810133fa7805" @ 04/20/24 13:06:18.943
STEP Verify volume: 62c8f35f-2eab-499c-8974-b60993233278 is attached to the node: sc2-10-187-100-27.nimbus.eng.vmware.com @ 04/20/24 13:06:19.991
Apr 20 13:06:22.368: INFO: vmRef: VirtualMachine:vm-185 for the VM uuid: 503bbc10-b30d-7636-d3b1-b98f78ce6d8d
Apr 20 13:06:22.836: INFO: Found FCDID "62c8f35f-2eab-499c-8974-b60993233278" attached to VM "web-4"
Apr 20 13:06:22.837: INFO: Found the disk "62c8f35f-2eab-499c-8974-b60993233278" is attached to the VM with UUID: "503bbc10-b30d-7636-d3b1-b98f78ce6d8d"
STEP successfully verified metadata of the volume "62c8f35f-2eab-499c-8974-b60993233278" @ 04/20/24 13:06:24.703
Apr 20 13:06:24.703: INFO: Scale down sts replica count to 1
STEP Scaling down statefulsets to number of Replica: 1 @ 04/20/24 13:06:24.703
Apr 20 13:06:24.703: INFO: Scaling statefulset web to 1
Apr 20 13:06:46.264: INFO: Waiting for statefulset status.replicas updated to 1
STEP Verify Volumes are detached from Nodes after Statefulsets is scaled down @ 04/20/24 13:06:46.725
STEP successfully verified metadata of the volume "d935c9b3-c3c4-4ff2-b5b9-3989cddf6336" @ 04/20/24 13:06:49.485
Apr 20 13:06:49.717: INFO: Scaling statefulset web to 0
Apr 20 13:07:00.848: INFO: Waiting for statefulset status.replicas updated to 0
Apr 20 13:07:01.157: INFO: Deleting statefulset web
Apr 20 13:07:01.772: INFO: Deleting pvc: www-web-0 with volume pvc-173ae001-df97-4f66-abf7-501a8202e89d
Apr 20 13:07:04.077: INFO: Deleting pvc: www-web-1 with volume pvc-2f90e7d9-5ffe-42c4-882e-c0431846740b
Apr 20 13:07:04.611: INFO: Deleting pvc: www-web-2 with volume pvc-2d83603c-3dc2-4423-a6e2-4641d7c394d2
Apr 20 13:07:04.972: INFO: Deleting pvc: www-web-3 with volume pvc-5c7dd833-d0c7-4720-9286-c8995d2e23c6
Apr 20 13:07:05.361: INFO: Deleting pvc: www-web-4 with volume pvc-1730ee77-a215-4e41-b91b-a287a725dc2f
Apr 20 13:07:06.140: INFO: Still waiting for pvs of statefulset to disappear:
pvc-1730ee77-a215-4e41-b91b-a287a725dc2f: {Phase:Released Message: Reason:}
pvc-173ae001-df97-4f66-abf7-501a8202e89d: {Phase:Released Message: Reason:}
pvc-2d83603c-3dc2-4423-a6e2-4641d7c394d2: {Phase:Released Message: Reason:}
pvc-2f90e7d9-5ffe-42c4-882e-c0431846740b: {Phase:Released Message: Reason:}
pvc-5c7dd833-d0c7-4720-9286-c8995d2e23c6: {Phase:Released Message: Reason:}
Apr 20 13:07:17.429: INFO: Scaling statefulset web to 0
Apr 20 13:07:38.415: INFO: Waiting for statefulset status.replicas updated to 0
Apr 20 13:07:38.653: INFO: Deleting statefulset web
Apr 20 13:07:39.150: INFO: Deleting pvc: www-web-0 with volume pvc-6d1f77af-80af-4156-aecc-94737c80b910
Apr 20 13:07:39.575: INFO: Deleting pvc: www-web-1 with volume pvc-5367b7be-a99e-4b56-93a3-3efe5c57ba8e
Apr 20 13:07:39.882: INFO: Deleting pvc: www-web-2 with volume pvc-d7f8c7f6-b15e-4098-bcb4-1346d2fd5f1f
Apr 20 13:07:40.172: INFO: Deleting pvc: www-web-3 with volume pvc-95b757a7-d2d6-4a88-b2a7-aa455a2e8171
Apr 20 13:07:40.738: INFO: Deleting pvc: www-web-4 with volume pvc-32525171-b4a2-44ed-acb0-d4a5f1567cc2
Apr 20 13:07:41.404: INFO: Still waiting for pvs of statefulset to disappear:
pvc-32525171-b4a2-44ed-acb0-d4a5f1567cc2: {Phase:Released Message: Reason:}
pvc-5367b7be-a99e-4b56-93a3-3efe5c57ba8e: {Phase:Released Message: Reason:}
pvc-6d1f77af-80af-4156-aecc-94737c80b910: {Phase:Released Message: Reason:}
pvc-95b757a7-d2d6-4a88-b2a7-aa455a2e8171: {Phase:Released Message: Reason:}
pvc-d7f8c7f6-b15e-4098-bcb4-1346d2fd5f1f: {Phase:Released Message: Reason:}
STEP Deleting all statefulsets in namespace: e2e-test-namespace @ 04/20/24 13:07:52.262
STEP Deleting service nginx in namespace: e2e-test-namespace @ 04/20/24 13:07:53.082
STEP Deleting all PVCs in namespace: e2e-test-namespace @ 04/20/24 13:07:53.381
STEP Deleting all statefulsets in namespace: e2e-test-namespace1 @ 04/20/24 13:07:54.407
STEP Deleting service nginx in namespace: e2e-test-namespace1 @ 04/20/24 13:07:55.153
STEP Deleting all PVCs in namespace: e2e-test-namespace1 @ 04/20/24 13:07:55.429
• [415.472 seconds]
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report
autogenerated by Ginkgo
[ReportAfterSuite] PASSED [0.056 seconds]
------------------------------
Ran 1 of 849 Specs in 415.492 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 848 Skipped
PASS
Process 45102 has exited with status 0
Starting: /Users/rajgupta/go/bin/dlv dap --listen=127.0.0.1:50471 --log-dest=3 from /Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e
DAP server listening at: 127.0.0.1:50471
Type 'dlv help' for list of commands.
W0420 11:25:14.140463 30428 test_context.go:517] Unable to find in-cluster config, using default host : https://127.0.0.1:6443
Apr 20 11:25:14.140: INFO: The --provider flag is not set. Continuing as if --provider=skeleton had been used.
Running Suite: CNS CSI Driver End-to-End Tests - /Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e
==========================================================================================================
Random Seed: 1713592514
Will run 1 of 849 specs
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[csi-multi-svc] Multi-SVC [csi-multi-svc] Verify volume lifecycle ops post password rotation [p0, wcp, multiSvc, newTest]
/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/multi_svc_test.go:313
Apr 20 11:25:14.171: INFO: Creating new VC session
Apr 20 11:25:16.030: INFO: To get number of compute cluster and it's path - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc namespace.cluster.ls
Apr 20 11:25:20.149: INFO: storage policy id: fd4ab249-959f-4f07-a1a9-560f3779c4e5 for storage policy name is: shared-ds-policy
Apr 20 11:25:21.692: INFO: storage policy id: 01692d8a-ad49-4fa4-872a-de3361c4ebbb for storage policy name is: shared-ds-policy1
STEP Getting User and Supervisor-Id for both the supervisors @ 04/20/24 11:25:23.862
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
STEP Create StatefulSet with 3 replicas with parallel pod management @ 04/20/24 11:25:24.333
STEP Create service @ 04/20/24 11:25:24.333
Apr 20 11:25:24.333: INFO: Parsing service from tests/e2e/testing-manifests/statefulset/nginx/service.yaml
Apr 20 11:25:24.665: INFO: Create StatefulSet
Apr 20 11:25:24.665: INFO: Preparing StatefulSet Spec
Apr 20 11:25:24.665: INFO: Parsing statefulset from tests/e2e/testing-manifests/statefulset/nginx/statefulset.yaml
Apr 20 11:25:24.676: INFO: Creating statefulset
Apr 20 11:25:24.676: INFO: Creating statefulset e2e-test-namespace/web with 3 replicas and selector &LabelSelector{MatchLabels:map[string]string{app: nginx,},MatchExpressions:[]LabelSelectorRequirement{},}
Apr 20 11:25:25.255: INFO: Found 0 stateful pods, waiting for 3
Apr 20 11:25:35.820: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 11:25:45.739: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 11:25:55.764: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 11:26:05.824: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 11:26:05.824: INFO: Waiting for pod web-1 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 11:26:05.824: INFO: Waiting for pod web-2 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 11:26:05.824: INFO: Wait for StatefulSet pods to be in up and running state
Apr 20 11:26:05.824: INFO: Waiting for statefulset status.replicas updated to 3
Apr 20 11:26:06.417: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-0 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 11:26:09.034: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 11:26:09.034: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 05:55 /usr/share/nginx/html\n"
Apr 20 11:26:09.034: INFO: stdout of ls -idlh /usr/share/nginx/html on web-0: 2 drwxr-xr-x 3 root root 4.0K Apr 20 05:55 /usr/share/nginx/html
Apr 20 11:26:09.035: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-1 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 11:26:11.456: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 11:26:11.456: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 05:55 /usr/share/nginx/html\n"
Apr 20 11:26:11.456: INFO: stdout of ls -idlh /usr/share/nginx/html on web-1: 2 drwxr-xr-x 3 root root 4.0K Apr 20 05:55 /usr/share/nginx/html
Apr 20 11:26:11.457: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-2 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 11:26:14.016: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 11:26:14.017: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 05:55 /usr/share/nginx/html\n"
Apr 20 11:26:14.017: INFO: stdout of ls -idlh /usr/share/nginx/html on web-2: 2 drwxr-xr-x 3 root root 4.0K Apr 20 05:55 /usr/share/nginx/html
Apr 20 11:26:14.545: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-0 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 11:26:16.932: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 11:26:16.933: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 11:26:16.933: INFO: stdout of find /usr/share/nginx/html on web-0: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 11:26:16.933: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-1 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 11:26:19.574: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 11:26:19.575: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 11:26:19.575: INFO: stdout of find /usr/share/nginx/html on web-1: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 11:26:19.575: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-2 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 11:26:22.153: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 11:26:22.153: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 11:26:22.153: INFO: stdout of find /usr/share/nginx/html on web-2: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 11:26:22.627: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-0 -- /bin/sh -x -c touch /usr/share/nginx/html/1713592566082355000'
Apr 20 11:26:25.116: INFO: stderr: "+ touch /usr/share/nginx/html/1713592566082355000\n"
Apr 20 11:26:25.116: INFO: stdout: ""
Apr 20 11:26:25.116: INFO: stdout of touch /usr/share/nginx/html/1713592566082355000 on web-0:
Apr 20 11:26:25.116: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-1 -- /bin/sh -x -c touch /usr/share/nginx/html/1713592566082355000'
Apr 20 11:26:27.462: INFO: stderr: "+ touch /usr/share/nginx/html/1713592566082355000\n"
Apr 20 11:26:27.462: INFO: stdout: ""
Apr 20 11:26:27.462: INFO: stdout of touch /usr/share/nginx/html/1713592566082355000 on web-1:
Apr 20 11:26:27.463: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube.config --namespace=e2e-test-namespace exec web-2 -- /bin/sh -x -c touch /usr/share/nginx/html/1713592566082355000'
Apr 20 11:26:30.148: INFO: stderr: "+ touch /usr/share/nginx/html/1713592566082355000\n"
Apr 20 11:26:30.148: INFO: stdout: ""
Apr 20 11:26:30.148: INFO: stdout of touch /usr/share/nginx/html/1713592566082355000 on web-2:
STEP Perform password rotation on the supervisor @ 04/20/24 11:26:30.628
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:26:30.890: INFO: Invoking command vmon-cli --stop wcp on vCenter host 10.187.99.63:22
Apr 20 11:26:40.572: INFO: Invoking command cd /etc/vmware/wcp; sudo -u wcp psql -U wcpuser -d VCDB -c "update cluster_db_configs set last_storage_pwd_rotation_timestamp=1713549400 where instance_id='9d9fe2bc-e733-49e7-9199-d885c59d541a'" on vCenter host 10.187.99.63:22
Apr 20 11:26:44.913: INFO: Invoking command vmon-cli --start wcp on vCenter host 10.187.99.63:22
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:26:58.839: INFO: Password from Config Secret of svc: E(QX'Pr0zgeD+?/2m>'\"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:26:59.145: INFO: Password from Config Secret of svc: E(QX'Pr0zgeD+?/2m>'\"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:27:59.460: INFO: Password from Config Secret of svc: E(QX'Pr0zgeD+?/2m>'\"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:28:59.465: INFO: Password from Config Secret of svc: E(QX'Pr0zgeD+?/2m>'\"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:29:59.480: INFO: Password from Config Secret of svc: E(QX'Pr0zgeD+?/2m>'\"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:30:59.480: INFO: Password from Config Secret of svc: E(QX'Pr0zgeD+?/2m>'\"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:31:59.496: INFO: Password from Config Secret of svc: 1VXxsLj~-pg`tR(\\)$}o
Apr 20 11:31:59.496: INFO: Scale up sts replica count to 5
STEP Scaling up statefulsets to number of Replica: 5 @ 04/20/24 11:31:59.496
Apr 20 11:31:59.497: INFO: Scaling statefulset web to 5
Apr 20 11:32:01.034: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 11:32:01.290: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 11:32:01.637: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 11:32:11.872: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 11:32:21.885: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
STEP Verify all volumes are attached to Nodes after Statefulsets is scaled up @ 04/20/24 11:32:32.567
STEP Verify volume: 41017161-28c4-4e30-887d-359347f7708c is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 11:32:33.538
Apr 20 11:32:37.090: INFO: vmRef: VirtualMachine:vm-153 for the VM uuid: 503b7b62-0cd2-cabc-e44f-e5351cb70fec
Apr 20 11:32:37.643: INFO: Found FCDID "41017161-28c4-4e30-887d-359347f7708c" attached to VM "web-0"
Apr 20 11:32:37.643: INFO: Found the disk "41017161-28c4-4e30-887d-359347f7708c" is attached to the VM with UUID: "503b7b62-0cd2-cabc-e44f-e5351cb70fec"
STEP successfully verified metadata of the volume "41017161-28c4-4e30-887d-359347f7708c" @ 04/20/24 11:32:38.714
STEP Verify volume: 89178c0e-26c9-4e9e-a2c1-1f05d543eca6 is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 11:32:39.698
Apr 20 11:32:42.504: INFO: vmRef: VirtualMachine:vm-155 for the VM uuid: 503b61c4-572a-ae95-f57a-8c5799b902c9
Apr 20 11:32:42.974: INFO: Found FCDID "89178c0e-26c9-4e9e-a2c1-1f05d543eca6" attached to VM "web-1"
Apr 20 11:32:42.975: INFO: Found the disk "89178c0e-26c9-4e9e-a2c1-1f05d543eca6" is attached to the VM with UUID: "503b61c4-572a-ae95-f57a-8c5799b902c9"
STEP successfully verified metadata of the volume "89178c0e-26c9-4e9e-a2c1-1f05d543eca6" @ 04/20/24 11:32:44.003
STEP Verify volume: bd8a30de-e1c8-4699-88df-4bb0085e1961 is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 11:32:45.064
Apr 20 11:32:47.829: INFO: vmRef: VirtualMachine:vm-154 for the VM uuid: 503b039e-62f1-de33-b54b-12f030a62fd8
Apr 20 11:32:48.450: INFO: Found FCDID "bd8a30de-e1c8-4699-88df-4bb0085e1961" attached to VM "web-2"
Apr 20 11:32:48.450: INFO: Found the disk "bd8a30de-e1c8-4699-88df-4bb0085e1961" is attached to the VM with UUID: "503b039e-62f1-de33-b54b-12f030a62fd8"
STEP successfully verified metadata of the volume "bd8a30de-e1c8-4699-88df-4bb0085e1961" @ 04/20/24 11:32:49.662
STEP Verify volume: 0ca61b7c-f7b1-4ffb-8911-4d77ea7b2e3d is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 11:32:50.696
Apr 20 11:32:53.563: INFO: vmRef: VirtualMachine:vm-156 for the VM uuid: 503bdaab-e77c-c00a-2e9a-0c5db74d0f82
Apr 20 11:32:54.040: INFO: Found FCDID "0ca61b7c-f7b1-4ffb-8911-4d77ea7b2e3d" attached to VM "web-3"
Apr 20 11:32:54.040: INFO: Found the disk "0ca61b7c-f7b1-4ffb-8911-4d77ea7b2e3d" is attached to the VM with UUID: "503bdaab-e77c-c00a-2e9a-0c5db74d0f82"
STEP successfully verified metadata of the volume "0ca61b7c-f7b1-4ffb-8911-4d77ea7b2e3d" @ 04/20/24 11:32:55.412
STEP Verify volume: 34b1fb5e-46da-4384-b24f-4faf4737ae68 is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 11:32:56.533
Apr 20 11:32:59.401: INFO: vmRef: VirtualMachine:vm-157 for the VM uuid: 503b34d3-8a83-d022-ea6e-8be1d3d609de
Apr 20 11:33:00.040: INFO: Found FCDID "34b1fb5e-46da-4384-b24f-4faf4737ae68" attached to VM "web-4"
Apr 20 11:33:00.040: INFO: Found the disk "34b1fb5e-46da-4384-b24f-4faf4737ae68" is attached to the VM with UUID: "503b34d3-8a83-d022-ea6e-8be1d3d609de"
STEP successfully verified metadata of the volume "34b1fb5e-46da-4384-b24f-4faf4737ae68" @ 04/20/24 11:33:01.347
Apr 20 11:33:01.348: INFO: Scale down sts replica count to 1
STEP Scaling down statefulsets to number of Replica: 1 @ 04/20/24 11:33:01.348
Apr 20 11:33:01.348: INFO: Scaling statefulset web to 1
Apr 20 11:33:13.022: INFO: Waiting for statefulset status.replicas updated to 1
STEP Verify Volumes are detached from Nodes after Statefulsets is scaled down @ 04/20/24 11:33:13.562
STEP successfully verified metadata of the volume "41017161-28c4-4e30-887d-359347f7708c" @ 04/20/24 11:33:15.787
STEP Creating StorageClass shared-ds-policy with scParameters: map[csi.storage.k8s.io/fstype:ext4 storagePolicyID:fd4ab249-959f-4f07-a1a9-560f3779c4e5] and allowedTopologies: [] and ReclaimPolicy: and allowVolumeExpansion: false @ 04/20/24 11:34:06.643
STEP Creating PVC using the Storage Class shared-ds-policy with disk size and labels: map[] accessMode: @ 04/20/24 11:34:18.575
Apr 20 11:34:18.974: INFO: PVC created: pvc-bjkkp in namespace: e2e-test-namespace
STEP Waiting for pvc to be in bound state @ 04/20/24 11:34:30.006
Apr 20 11:34:34.762: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-bjkkp] to have phase Bound
Apr 20 11:34:35.041: INFO: PersistentVolumeClaim pvc-bjkkp found and phase=Bound (279.078875ms)
STEP Creating pod to attach PV to the node @ 04/20/24 11:34:41.03
STEP Verify volume: 655df66c-b316-4b32-9254-d48b1b2db36b is attached to the node: sc2-10-187-105-188.nimbus.eng.vmware.com @ 04/20/24 11:35:58.504
Apr 20 11:35:58.504: INFO: VMUUID : 503b435a-6132-760a-00f6-e3b9d3bf471e
Apr 20 11:36:00.446: INFO: vmRef: VirtualMachine:vm-158 for the VM uuid: 503b435a-6132-760a-00f6-e3b9d3bf471e
Apr 20 11:36:00.923: INFO: Found FCDID "655df66c-b316-4b32-9254-d48b1b2db36b" attached to VM "pvc-tester-cxssk"
Apr 20 11:36:00.923: INFO: Found the disk "655df66c-b316-4b32-9254-d48b1b2db36b" is attached to the VM with UUID: "503b435a-6132-760a-00f6-e3b9d3bf471e"
STEP Create StatefulSet with 3 replicas with parallel pod management @ 04/20/24 11:36:00.927
STEP Create service @ 04/20/24 11:36:00.927
Apr 20 11:36:00.927: INFO: Parsing service from tests/e2e/testing-manifests/statefulset/nginx/service.yaml
Apr 20 11:36:01.682: INFO: Create StatefulSet
Apr 20 11:36:01.682: INFO: Preparing StatefulSet Spec
Apr 20 11:36:01.682: INFO: Parsing statefulset from tests/e2e/testing-manifests/statefulset/nginx/statefulset.yaml
Apr 20 11:36:01.686: INFO: Creating statefulset
Apr 20 11:36:01.686: INFO: Creating statefulset e2e-test-namespace1/web with 3 replicas and selector &LabelSelector{MatchLabels:map[string]string{app: nginx,},MatchExpressions:[]LabelSelectorRequirement{},}
Apr 20 11:36:02.351: INFO: Found 0 stateful pods, waiting for 3
Apr 20 11:36:12.845: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 11:36:22.831: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 11:36:33.029: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Pending - Ready=false
Apr 20 11:36:42.811: INFO: Waiting for pod web-0 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 11:36:42.812: INFO: Waiting for pod web-1 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 11:36:42.812: INFO: Waiting for pod web-2 to enter Running - Ready=true, currently Running - Ready=true
Apr 20 11:36:42.812: INFO: Wait for StatefulSet pods to be in up and running state
Apr 20 11:36:42.812: INFO: Waiting for statefulset status.replicas updated to 3
Apr 20 11:36:43.281: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-0 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 11:36:49.315: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 11:36:49.315: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 06:06 /usr/share/nginx/html\n"
Apr 20 11:36:49.315: INFO: stdout of ls -idlh /usr/share/nginx/html on web-0: 2 drwxr-xr-x 3 root root 4.0K Apr 20 06:06 /usr/share/nginx/html
Apr 20 11:36:49.315: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-1 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 11:36:51.760: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 11:36:51.760: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 06:06 /usr/share/nginx/html\n"
Apr 20 11:36:51.760: INFO: stdout of ls -idlh /usr/share/nginx/html on web-1: 2 drwxr-xr-x 3 root root 4.0K Apr 20 06:06 /usr/share/nginx/html
Apr 20 11:36:51.760: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-2 -- /bin/sh -x -c ls -idlh /usr/share/nginx/html'
Apr 20 11:36:54.269: INFO: stderr: "+ ls -idlh /usr/share/nginx/html\n"
Apr 20 11:36:54.269: INFO: stdout: "2 drwxr-xr-x 3 root root 4.0K Apr 20 06:06 /usr/share/nginx/html\n"
Apr 20 11:36:54.270: INFO: stdout of ls -idlh /usr/share/nginx/html on web-2: 2 drwxr-xr-x 3 root root 4.0K Apr 20 06:06 /usr/share/nginx/html
Apr 20 11:36:54.771: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-0 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 11:36:58.193: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 11:36:58.193: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 11:36:58.193: INFO: stdout of find /usr/share/nginx/html on web-0: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 11:36:58.194: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-1 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 11:37:00.755: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 11:37:00.755: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 11:37:00.755: INFO: stdout of find /usr/share/nginx/html on web-1: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 11:37:00.755: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-2 -- /bin/sh -x -c find /usr/share/nginx/html'
Apr 20 11:37:03.138: INFO: stderr: "+ find /usr/share/nginx/html\n"
Apr 20 11:37:03.138: INFO: stdout: "/usr/share/nginx/html\n/usr/share/nginx/html/lost+found\n"
Apr 20 11:37:03.138: INFO: stdout of find /usr/share/nginx/html on web-2: /usr/share/nginx/html
/usr/share/nginx/html/lost+found
Apr 20 11:37:03.599: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-0 -- /bin/sh -x -c touch /usr/share/nginx/html/1713593203047158000'
Apr 20 11:37:06.180: INFO: stderr: "+ touch /usr/share/nginx/html/1713593203047158000\n"
Apr 20 11:37:06.180: INFO: stdout: ""
Apr 20 11:37:06.180: INFO: stdout of touch /usr/share/nginx/html/1713593203047158000 on web-0:
Apr 20 11:37:06.181: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-1 -- /bin/sh -x -c touch /usr/share/nginx/html/1713593203047158000'
Apr 20 11:37:08.801: INFO: stderr: "+ touch /usr/share/nginx/html/1713593203047158000\n"
Apr 20 11:37:08.801: INFO: stdout: ""
Apr 20 11:37:08.801: INFO: stdout of touch /usr/share/nginx/html/1713593203047158000 on web-1:
Apr 20 11:37:08.801: INFO: Running '/opt/homebrew/bin/kubectl --kubeconfig=/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/kube1.config --namespace=e2e-test-namespace1 exec web-2 -- /bin/sh -x -c touch /usr/share/nginx/html/1713593203047158000'
Apr 20 11:37:11.378: INFO: stderr: "+ touch /usr/share/nginx/html/1713593203047158000\n"
Apr 20 11:37:11.378: INFO: stdout: ""
Apr 20 11:37:11.378: INFO: stdout of touch /usr/share/nginx/html/1713593203047158000 on web-2:
STEP Perform password rotation on the supervisor @ 04/20/24 11:37:11.841
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:37:12.227: INFO: Invoking command vmon-cli --stop wcp on vCenter host 10.187.99.63:22
Apr 20 11:37:18.992: INFO: Invoking command cd /etc/vmware/wcp; sudo -u wcp psql -U wcpuser -d VCDB -c "update cluster_db_configs set last_storage_pwd_rotation_timestamp=1713550038 where instance_id='1dc28816-5ba8-4114-9ebd-3219a819e17b'" on vCenter host 10.187.99.63:22
Apr 20 11:37:23.630: INFO: Invoking command vmon-cli --start wcp on vCenter host 10.187.99.63:22
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:37:42.662: INFO: Password from Config Secret of svc: l2%E5p}$zHG Yf2Dbf|0
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:37:42.898: INFO: Password from Config Secret of svc: l2%E5p}$zHG Yf2Dbf|0
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:38:43.231: INFO: Password from Config Secret of svc: l2%E5p}$zHG Yf2Dbf|0
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:39:43.127: INFO: Password from Config Secret of svc: l2%E5p}$zHG Yf2Dbf|0
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:40:43.194: INFO: Password from Config Secret of svc: l2%E5p}$zHG Yf2Dbf|0
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Apr 20 11:41:43.129: INFO: Password from Config Secret of svc: <N5q=inMc$#*zE@(NPAX
Apr 20 11:41:43.129: INFO: Scale up sts replica count to 5
STEP Scaling up statefulsets to number of Replica: 5 @ 04/20/24 11:41:43.129
Apr 20 11:41:43.129: INFO: Scaling statefulset web to 5
Apr 20 11:41:44.520: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 11:41:44.750: INFO: Waiting for statefulset status.replicas updated to 5
Apr 20 11:41:45.024: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 11:41:55.269: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
Apr 20 11:42:05.299: INFO: Waiting for stateful set status.readyReplicas to become 5, currently 3
STEP Verify all volumes are attached to Nodes after Statefulsets is scaled up @ 04/20/24 11:42:15.854
STEP Verify volume: 4e727d14-48c9-4b3a-9708-6950f9f8c05d is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 11:42:16.861
Apr 20 11:42:20.559: INFO: vmRef: VirtualMachine:vm-160 for the VM uuid: 503bc89c-aaac-1c6d-bff2-7a30f0f3fe49
Apr 20 11:42:21.150: INFO: Found FCDID "4e727d14-48c9-4b3a-9708-6950f9f8c05d" attached to VM "web-0"
Apr 20 11:42:21.150: INFO: Found the disk "4e727d14-48c9-4b3a-9708-6950f9f8c05d" is attached to the VM with UUID: "503bc89c-aaac-1c6d-bff2-7a30f0f3fe49"
STEP successfully verified metadata of the volume "4e727d14-48c9-4b3a-9708-6950f9f8c05d" @ 04/20/24 11:42:23.002
STEP Verify volume: fe1541b2-28b8-4849-add2-f8d53ad1865c is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 11:42:23.982
Apr 20 11:42:26.509: INFO: vmRef: VirtualMachine:vm-161 for the VM uuid: 503b5c8d-0aee-cca9-612e-d839e273fc45
Apr 20 11:42:27.213: INFO: Found FCDID "fe1541b2-28b8-4849-add2-f8d53ad1865c" attached to VM "web-1"
Apr 20 11:42:27.213: INFO: Found the disk "fe1541b2-28b8-4849-add2-f8d53ad1865c" is attached to the VM with UUID: "503b5c8d-0aee-cca9-612e-d839e273fc45"
STEP successfully verified metadata of the volume "fe1541b2-28b8-4849-add2-f8d53ad1865c" @ 04/20/24 11:42:29.057
STEP Verify volume: 1880b326-0355-41cd-a33a-95b248fa2005 is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 11:42:30.131
Apr 20 11:42:32.752: INFO: vmRef: VirtualMachine:vm-159 for the VM uuid: 503bb31f-1afc-de30-828c-24394ac4a442
Apr 20 11:42:33.367: INFO: Found FCDID "1880b326-0355-41cd-a33a-95b248fa2005" attached to VM "web-2"
Apr 20 11:42:33.367: INFO: Found the disk "1880b326-0355-41cd-a33a-95b248fa2005" is attached to the VM with UUID: "503bb31f-1afc-de30-828c-24394ac4a442"
STEP successfully verified metadata of the volume "1880b326-0355-41cd-a33a-95b248fa2005" @ 04/20/24 11:42:35.201
STEP Verify volume: 93936c64-0c50-4d16-8852-4d105a11b3f8 is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 11:42:36.148
Apr 20 11:42:38.784: INFO: vmRef: VirtualMachine:vm-163 for the VM uuid: 503b4cf2-5f30-21d3-fdc4-4a9ad1eea4e5
Apr 20 11:42:39.333: INFO: Found FCDID "93936c64-0c50-4d16-8852-4d105a11b3f8" attached to VM "web-3"
Apr 20 11:42:39.333: INFO: Found the disk "93936c64-0c50-4d16-8852-4d105a11b3f8" is attached to the VM with UUID: "503b4cf2-5f30-21d3-fdc4-4a9ad1eea4e5"
STEP successfully verified metadata of the volume "93936c64-0c50-4d16-8852-4d105a11b3f8" @ 04/20/24 11:42:41.188
STEP Verify volume: d2db0fb9-e4d8-4d5d-87ec-5284c7b50c31 is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 11:42:42.291
Apr 20 11:42:44.928: INFO: vmRef: VirtualMachine:vm-162 for the VM uuid: 503b3caa-1c81-cded-102d-711b500cd71f
Apr 20 11:42:45.448: INFO: Found FCDID "d2db0fb9-e4d8-4d5d-87ec-5284c7b50c31" attached to VM "web-4"
Apr 20 11:42:45.449: INFO: Found the disk "d2db0fb9-e4d8-4d5d-87ec-5284c7b50c31" is attached to the VM with UUID: "503b3caa-1c81-cded-102d-711b500cd71f"
STEP successfully verified metadata of the volume "d2db0fb9-e4d8-4d5d-87ec-5284c7b50c31" @ 04/20/24 11:42:47.259
Apr 20 11:42:47.260: INFO: Scale down sts replica count to 1
STEP Scaling down statefulsets to number of Replica: 1 @ 04/20/24 11:42:47.26
Apr 20 11:42:47.260: INFO: Scaling statefulset web to 1
Apr 20 11:42:59.169: INFO: Waiting for statefulset status.replicas updated to 1
STEP Verify Volumes are detached from Nodes after Statefulsets is scaled down @ 04/20/24 11:42:59.676
STEP successfully verified metadata of the volume "4e727d14-48c9-4b3a-9708-6950f9f8c05d" @ 04/20/24 11:43:02.753
STEP Creating StorageClass shared-ds-policy1 with scParameters: map[csi.storage.k8s.io/fstype:ext4 storagePolicyID:01692d8a-ad49-4fa4-872a-de3361c4ebbb] and allowedTopologies: [] and ReclaimPolicy: and allowVolumeExpansion: false @ 04/20/24 11:43:05.743
STEP Creating PVC using the Storage Class shared-ds-policy1 with disk size and labels: map[] accessMode: @ 04/20/24 11:43:05.979
Apr 20 11:43:06.366: INFO: PVC created: pvc-nr58g in namespace: e2e-test-namespace1
STEP Waiting for pvc to be in bound state @ 04/20/24 11:43:11.12
Apr 20 11:43:11.120: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-nr58g] to have phase Bound
Apr 20 11:43:11.346: INFO: PersistentVolumeClaim pvc-nr58g found and phase=Bound (226.679917ms)
STEP Creating pod to attach PV to the node @ 04/20/24 11:43:11.808
STEP Verify volume: 2c7dd7b1-b47a-4474-98a4-eb20dbac3d50 is attached to the node: sc2-10-187-103-104.nimbus.eng.vmware.com @ 04/20/24 11:43:33.113
Apr 20 11:43:33.114: INFO: VMUUID : 503b5d21-b279-2c26-0aec-96e2b04bf26b
Apr 20 11:43:34.591: INFO: vmRef: VirtualMachine:vm-164 for the VM uuid: 503b5d21-b279-2c26-0aec-96e2b04bf26b
Apr 20 11:43:35.137: INFO: Found FCDID "2c7dd7b1-b47a-4474-98a4-eb20dbac3d50" attached to VM "pvc-tester-gdkmk"
Apr 20 11:43:35.137: INFO: Found the disk "2c7dd7b1-b47a-4474-98a4-eb20dbac3d50" is attached to the VM with UUID: "503b5d21-b279-2c26-0aec-96e2b04bf26b"
STEP Deleting the pod @ 04/20/24 11:43:35.137
Apr 20 11:43:35.138: INFO: Deleting pod "pvc-tester-gdkmk" in namespace "e2e-test-namespace1"
Apr 20 11:43:35.419: INFO: Wait up to 5m0s for pod "pvc-tester-gdkmk" to be fully deleted
Apr 20 11:44:16.985: INFO: Deleting PersistentVolumeClaim "pvc-nr58g"
Apr 20 11:44:17.702: INFO: Scaling statefulset web to 0
Apr 20 11:44:28.968: INFO: Waiting for statefulset status.replicas updated to 0
Apr 20 11:44:29.274: INFO: Deleting statefulset web
Apr 20 11:44:29.817: INFO: Deleting pvc: www-web-0 with volume pvc-7aa844b1-4455-4223-82be-8e639709df54
Apr 20 11:44:30.138: INFO: Deleting pvc: www-web-1 with volume pvc-18bfacdc-09cd-4bef-b080-4462e1454b52
Apr 20 11:44:30.543: INFO: Deleting pvc: www-web-2 with volume pvc-0eb40639-0bd6-4c9d-a9f9-39d5b1797dc9
Apr 20 11:44:31.233: INFO: Deleting pvc: www-web-3 with volume pvc-f8ac7854-239f-4ed7-9449-106b695c5b79
Apr 20 11:44:31.557: INFO: Deleting pvc: www-web-4 with volume pvc-b0d6042e-52f0-41ae-a8e8-517bebad9695
Apr 20 11:44:32.105: INFO: Still waiting for pvs of statefulset to disappear:
pvc-0eb40639-0bd6-4c9d-a9f9-39d5b1797dc9: {Phase:Released Message: Reason:}
pvc-18bfacdc-09cd-4bef-b080-4462e1454b52: {Phase:Released Message: Reason:}
pvc-b0d6042e-52f0-41ae-a8e8-517bebad9695: {Phase:Released Message: Reason:}
pvc-f8ac7854-239f-4ed7-9449-106b695c5b79: {Phase:Released Message: Reason:}
STEP Deleting the pod @ 04/20/24 11:44:42.695
Apr 20 11:44:42.695: INFO: Deleting pod "pvc-tester-cxssk" in namespace "e2e-test-namespace"
Apr 20 11:44:43.511: INFO: Wait up to 5m0s for pod "pvc-tester-cxssk" to be fully deleted
Apr 20 11:45:29.206: INFO: Deleting PersistentVolumeClaim "pvc-nr58g"
Apr 20 11:45:29.795: INFO: Scaling statefulset web to 0
Apr 20 11:45:51.317: INFO: Waiting for statefulset status.replicas updated to 0
Apr 20 11:45:51.551: INFO: Deleting statefulset web
Apr 20 11:45:52.322: INFO: Deleting pvc: pvc-bjkkp with volume pvc-609fe8e3-dbfc-4ee0-8ca8-6716909ad1e6
Apr 20 11:45:52.780: INFO: Deleting pvc: www-web-0 with volume pvc-4e4de951-cee4-428c-ab6a-36f4d1a47f60
Apr 20 11:45:53.083: INFO: Deleting pvc: www-web-1 with volume pvc-fd44f081-acf3-4d5d-83e0-b2508e29fc76
Apr 20 11:45:53.505: INFO: Deleting pvc: www-web-2 with volume pvc-819c3b3f-ae43-4cd1-a7be-870a67615341
Apr 20 11:45:53.799: INFO: Deleting pvc: www-web-3 with volume pvc-d4a056df-fa48-4151-9360-922057092ca6
Apr 20 11:45:54.097: INFO: Deleting pvc: www-web-4 with volume pvc-663f21f5-e62c-4064-b030-85e7feb1c219
Apr 20 11:45:54.903: INFO: Still waiting for pvs of statefulset to disappear:
pvc-4e4de951-cee4-428c-ab6a-36f4d1a47f60: {Phase:Released Message: Reason:}
pvc-609fe8e3-dbfc-4ee0-8ca8-6716909ad1e6: {Phase:Released Message: Reason:}
pvc-663f21f5-e62c-4064-b030-85e7feb1c219: {Phase:Released Message: Reason:}
pvc-819c3b3f-ae43-4cd1-a7be-870a67615341: {Phase:Released Message: Reason:}
pvc-d4a056df-fa48-4151-9360-922057092ca6: {Phase:Released Message: Reason:}
pvc-fd44f081-acf3-4d5d-83e0-b2508e29fc76: {Phase:Released Message: Reason:}
STEP Deleting all statefulsets in namespace: e2e-test-namespace @ 04/20/24 11:46:05.53
STEP Deleting service nginx in namespace: e2e-test-namespace @ 04/20/24 11:46:06.253
STEP Deleting all PVCs in namespace: e2e-test-namespace @ 04/20/24 11:46:06.554
STEP Deleting all statefulsets in namespace: e2e-test-namespace1 @ 04/20/24 11:46:07.628
STEP Deleting service nginx in namespace: e2e-test-namespace1 @ 04/20/24 11:46:08.422
STEP Deleting all PVCs in namespace: e2e-test-namespace1 @ 04/20/24 11:46:08.657
• [1255.437 seconds]
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report
autogenerated by Ginkgo
[ReportAfterSuite] PASSED [0.059 seconds]
------------------------------
Ran 1 of 849 Specs in 1255.468 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 848 Skipped
PASS
Process 30428 has exited with status 0
Detaching
Starting: /Users/rajgupta/go/bin/dlv dap --listen=127.0.0.1:54646 --log-dest=3 from /Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e
DAP server listening at: 127.0.0.1:54646
W0420 13:52:23.160236 48357 test_context.go:517] Unable to find in-cluster config, using default host : https://127.0.0.1:6443
Apr 20 13:52:23.160: INFO: The --provider flag is not set. Continuing as if --provider=skeleton had been used.
Running Suite: CNS CSI Driver End-to-End Tests - /Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e
==========================================================================================================
Random Seed: 1713601343
Will run 1 of 849 specs
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[csi-multi-svc] Multi-SVC [csi-multi-svc] Verify permissions of the service account [p0, wcp, multiSvc, newTest]
/Users/rajgupta/GithubForked/vsphere-csi-driver/tests/e2e/multi_svc_test.go:396
Apr 20 13:52:23.188: INFO: Creating new VC session
Apr 20 13:52:24.980: INFO: To get number of compute cluster and it's path - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc namespace.cluster.ls
Apr 20 13:52:32.027: INFO: storage policy id: fd4ab249-959f-4f07-a1a9-560f3779c4e5 for storage policy name is: shared-ds-policy
Apr 20 13:52:32.806: INFO: storage policy id: 01692d8a-ad49-4fa4-872a-de3361c4ebbb for storage policy name is: shared-ds-policy1
STEP Getting User and Supervisor-Id for both the supervisors @ 04/20/24 13:52:35.292
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
Key: VirtualCenter, Value: "sc2-10-187-99-63.nimbus.eng.vmware.com"
STEP Verify permission on root folder for each of the wcp service account users @ 04/20/24 13:52:35.784
Apr 20 13:52:35.784: INFO: Verifying permission on root folder for user : wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9
Apr 20 13:52:35.784: INFO: Check permission of service account user on RootFolder - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls / | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:52:37.972: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:52:37.972: INFO: Verifying permission on root folder for user : wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9
Apr 20 13:52:37.972: INFO: Check permission of service account user on RootFolder - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls / | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:52:40.189: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
STEP Verify permission on clusters for each of the wcp service account users @ 04/20/24 13:52:40.19
Apr 20 13:52:40.190: INFO: Verifying permission on root folder for user: wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9
Apr 20 13:52:40.190: INFO: Check permission of service account user on Cluster - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls /WCPE2E_DC/host/wcpe2e-compute-cluster | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:52:43.565: INFO: Permission for wcp storgae user is : CNS-HOST-CONFIG-STORAGE-AND-CNS-VM
Apr 20 13:52:43.566: INFO: Verifying permission on root folder for user: wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9
Apr 20 13:52:43.566: INFO: Check permission of service account user on Cluster - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls /WCPE2E_DC/host/wcpe2e-compute-cluster | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:52:46.994: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:52:46.995: INFO: Verifying permission on root folder for user: wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9
Apr 20 13:52:46.995: INFO: Check permission of service account user on Cluster - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls /WCPE2E_DC/host/wcpe2e-compute-cluster1 | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:52:50.709: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:52:50.709: INFO: Verifying permission on root folder for user: wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9
Apr 20 13:52:50.709: INFO: Check permission of service account user on Cluster - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls /WCPE2E_DC/host/wcpe2e-compute-cluster1 | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:52:54.169: INFO: Permission for wcp storgae user is : CNS-HOST-CONFIG-STORAGE-AND-CNS-VM
STEP Verify service account permission on each of the datastore @ 04/20/24 13:52:54.17
Apr 20 13:52:54.891: INFO: Fetching datastore details
Apr 20 13:52:57.164: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/vsanDatastore (1)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:00.876: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:00.877: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/vsanDatastore (1)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:04.683: INFO: Permission for wcp storgae user is : CNS-Datastore
Apr 20 13:53:04.684: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/vsanDatastore' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:08.145: INFO: Permission for wcp storgae user is : CNS-Datastore
Apr 20 13:53:08.145: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/vsanDatastore' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:11.755: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:11.755: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (7)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:15.314: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:15.314: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (7)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:18.795: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:18.796: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (6)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:22.373: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:22.373: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (6)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:25.754: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:25.754: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:29.469: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:29.469: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:32.872: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:32.872: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (8)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:36.097: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:36.097: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (8)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:39.581: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:39.582: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (1)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:42.931: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:42.931: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (1)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:46.338: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:46.338: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (1)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:49.922: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:49.924: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (1)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:53.507: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:53.507: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (2)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:53:56.910: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:53:56.910: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (2)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:00.572: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:00.572: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (2)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:04.357: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:04.358: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (2)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:08.050: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:08.051: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/nfs0-1' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:12.044: INFO: Permission for wcp storgae user is : CNS-Datastore
Apr 20 13:54:12.045: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/nfs0-1' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:16.238: INFO: Permission for wcp storgae user is : CNS-Datastore
Apr 20 13:54:16.239: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (3)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:19.829: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:19.829: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (3)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:23.202: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:23.202: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (3)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:26.849: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:26.849: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (3)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:30.268: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:30.268: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (5)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:33.653: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:33.654: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (5)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:36.932: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:36.932: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (8)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:40.609: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:40.610: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (8)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:43.926: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:43.927: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (6)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:47.159: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:47.159: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (6)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:50.671: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:50.672: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (4)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:53.846: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:53.846: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (4)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:54:57.542: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:54:57.543: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (7)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:00.879: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:55:00.879: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (7)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:04.590: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:55:04.590: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (4)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:08.174: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:55:08.175: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1 (4)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:12.137: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:55:12.137: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:15.724: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:55:15.725: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-1' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:19.303: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:55:19.304: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (5)' | grep wcp-storage-user-9d9fe2bc-e733-49e7-9199-d885c59d541a-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:23.188: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
Apr 20 13:55:23.188: INFO: Check permission of service account user on Datastore - command : export GOVC_INSECURE=1;export GOVC_URL='https://Administrator@vsphere.local:COGNpPS.s4cmm+7e@10.187.99.63:443';govc permissions.ls '/WCPE2E_DC/datastore/local-0 (5)' | grep wcp-storage-user-1dc28816-5ba8-4114-9ebd-3219a819e17b-d7f8097a-8b1c-4de8-ba13-9251be2877c9 | awk '{print $1}'
Apr 20 13:55:27.397: INFO: Permission for wcp storgae user is : CNS-SEARCH-AND-SPBM
STEP Deleting all statefulsets in namespace: e2e-test-namespace @ 04/20/24 13:55:27.398
STEP Deleting service nginx in namespace: e2e-test-namespace @ 04/20/24 13:55:28.758
STEP Deleting all PVCs in namespace: e2e-test-namespace @ 04/20/24 13:55:28.995
STEP Deleting all statefulsets in namespace: e2e-test-namespace1 @ 04/20/24 13:55:30.081
STEP Deleting service nginx in namespace: e2e-test-namespace1 @ 04/20/24 13:55:31.375
STEP Deleting all PVCs in namespace: e2e-test-namespace1 @ 04/20/24 13:55:31.617
• [189.480 seconds]
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report
autogenerated by Ginkgo
[ReportAfterSuite] PASSED [0.059 seconds]
------------------------------
Ran 1 of 849 Specs in 189.511 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 848 Skipped
PASS
Process 48357 has exited with status 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment