Skip to content

Instantly share code, notes, and snippets.

@divyenpatel

divyenpatel/migration-inline-volume-testing.md

Created June 23, 2020 17:21
Show Gist options
  • Save divyenpatel/b06955dccf5944072f742f6bc9e2c59e to your computer and use it in GitHub Desktop.
Save divyenpatel/b06955dccf5944072f742f6bc9e2c59e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
migration-inline-volume-testing.md

Created a Pod using following YAML

# cat inline-deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-inline-deployment
  labels:
    app: sample
    tier: sample
spec:
  selector:
    matchLabels:
      app: sample
      tier: sample
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: sample
        tier: sample
    spec:
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
      containers:
      - image: busybox
        name: sample
        volumeMounts:
        - name: sample-volume
          mountPath: /data
        command: [ "sleep", "infinity" ]
      volumes:
      - name: sample-volume
        vsphereVolume:
          volumePath: "[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk"
          fsType: ext4

# kubectl create -f inline-deployment.yaml 
deployment.apps/sample-inline-deployment created


# kubectl get pods
NAME                                       READY   STATUS    RESTARTS   AGE
sample-inline-deployment-55b6f5fc7-rjk4v   1/1     Running   0          2m16s

Controller Publish Logs

2020-06-23T17:12:45.602264325Z {"level":"info","time":"2020-06-23T17:12:45.601725788Z","caller":"vanilla/controller.go:555","msg":"ControllerPublishVolume: called with args {VolumeId:[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk NodeId:k8s-node1 VolumeCapability:mount:<fs_type:\"ext4\" > access_mode:<mode:SINGLE_NODE_WRITER >  Readonly:false Secrets:map[] VolumeContext:map[] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:45.614096449Z {"level":"info","time":"2020-06-23T17:12:45.613616468Z","caller":"migration/migration.go:173","msg":"Could not retrieve VolumeID from cache for Volume Path: \"[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk\". volume may not be registered. Registering Volume with CNS","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:45.614242541Z {"level":"info","time":"2020-06-23T17:12:45.613848563Z","caller":"migration/migration.go:351","msg":"Registering volume: \"[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk\" using backingDiskURLPath :\"https://10.160.41.232/folder/0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk?dcPath=cns-datacenter&dsName=vsanDatastore\"","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:47.685749127Z {"level":"info","time":"2020-06-23T17:12:47.685288132Z","caller":"volume/manager.go:197","msg":"CreateVolume: VolumeName: \"c18a8b98-b574-11ea-ad39-eabf7796a608\", opId: \"4c812fd0\"","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:47.685818482Z {"level":"info","time":"2020-06-23T17:12:47.685381972Z","caller":"volume/manager.go:215","msg":"CreateVolume: Volume is already registered with CNS. VolumeName: \"c18a8b98-b574-11ea-ad39-eabf7796a608\", volumeID: \"31fb23f2-15f1-4fb9-94aa-214999668649\", opId: \"4c812fd0\"","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:47.685828897Z {"level":"info","time":"2020-06-23T17:12:47.68540011Z","caller":"migration/migration.go:375","msg":"successfully registered volume \"[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk\" as container volume with ID: \"31fb23f2-15f1-4fb9-94aa-214999668649\"","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:47.685836705Z {"level":"info","time":"2020-06-23T17:12:47.68541563Z","caller":"migration/migration.go:179","msg":"successfully registered volume: \"[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk\" with CNS. VolumeID: \"31fb23f2-15f1-4fb9-94aa-214999668649\"","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:47.685845429Z {"level":"info","time":"2020-06-23T17:12:47.68550872Z","caller":"migration/migration.go:258","msg":"creating CR for cnsVSphereVolumeMigration: &{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:31fb23f2-15f1-4fb9-94aa-214999668649 GenerateName: Namespace: SelfLink: UID: ResourceVersion: Generation:0 CreationTimestamp:0001-01-01 00:00:00 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[] Annotations:map[] OwnerReferences:[] Initializers:nil Finalizers:[] ClusterName: ManagedFields:[]} Spec:{VolumePath:[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk VolumeID:31fb23f2-15f1-4fb9-94aa-214999668649}}","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:47.694730539Z {"level":"info","time":"2020-06-23T17:12:47.694085836Z","caller":"migration/migration.go:268","msg":"successfully created CR for cnsVSphereVolumeMigration: &{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:31fb23f2-15f1-4fb9-94aa-214999668649 GenerateName: Namespace: SelfLink:/apis/cns.vmware.com/v1alpha1/cnsvspherevolumemigrations/31fb23f2-15f1-4fb9-94aa-214999668649 UID:8e6eac22-a6ae-404a-997e-b13cdacf6d80 ResourceVersion:6189353 Generation:1 CreationTimestamp:2020-06-23 17:12:47 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[] Annotations:map[] OwnerReferences:[] Initializers:nil Finalizers:[] ClusterName: ManagedFields:[{Manager:vsphere-csi Operation:Update APIVersion:cns.vmware.com/v1alpha1 Time:2020-06-23 17:12:47 +0000 UTC Fields:nil}]} Spec:{VolumePath:[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk VolumeID:31fb23f2-15f1-4fb9-94aa-214999668649}}","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:47.695072110Z {"level":"info","time":"2020-06-23T17:12:47.694922455Z","caller":"migration/migration.go:193","msg":"successfully saved cnsvSphereVolumeMigration CR: {{ } {31fb23f2-15f1-4fb9-94aa-214999668649   /apis/cns.vmware.com/v1alpha1/cnsvspherevolumemigrations/31fb23f2-15f1-4fb9-94aa-214999668649 8e6eac22-a6ae-404a-997e-b13cdacf6d80 6189353 1 2020-06-23 17:12:47 +0000 UTC <nil> <nil> map[] map[] [] nil []  [{vsphere-csi Update cns.vmware.com/v1alpha1 2020-06-23 17:12:47 +0000 UTC nil}]} {[vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk 31fb23f2-15f1-4fb9-94aa-214999668649}}","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:57.454657710Z {"level":"info","time":"2020-06-23T17:12:57.454025922Z","caller":"volume/manager.go:267","msg":"AttachVolume: volumeID: \"31fb23f2-15f1-4fb9-94aa-214999668649\", vm: \"VirtualMachine:vm-86 [VirtualCenterHost: 10.160.41.232, UUID: 42246c52-c2bd-49cf-1111-fbcbd29087da, Datacenter: Datacenter [Datacenter: Datacenter:datacenter-3, VirtualCenterHost: 10.160.41.232]]\", opId: \"4c812fd2\"","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}
2020-06-23T17:12:57.454713333Z {"level":"info","time":"2020-06-23T17:12:57.454388473Z","caller":"volume/manager.go:300","msg":"AttachVolume: Volume attached successfully. volumeID: \"31fb23f2-15f1-4fb9-94aa-214999668649\", opId: \"4c812fd2\", vm: \"VirtualMachine:vm-86 [VirtualCenterHost: 10.160.41.232, UUID: 42246c52-c2bd-49cf-1111-fbcbd29087da, Datacenter: Datacenter [Datacenter: Datacenter:datacenter-3, VirtualCenterHost: 10.160.41.232]]\", diskUUID: \"6000C29a-8dfb-decd-9788-e70763c1653f\"","TraceId":"ff10990e-4889-46b0-a91e-50131e81f030"}

CRDs

# kubectl get volumeattachment
NAME                                                                   ATTACHER                 PV    NODE        ATTACHED   AGE
csi-49012ea45cfb0100fa19d9d590c76814cb853375b3f9c1348f66a727f948678a   csi.vsphere.vmware.com         k8s-node1   true       2m31s



# kubectl describe volumeattachment
Name:         csi-49012ea45cfb0100fa19d9d590c76814cb853375b3f9c1348f66a727f948678a
Namespace:    
Labels:       <none>
Annotations:  csi.alpha.kubernetes.io/node-id: k8s-node1
API Version:  storage.k8s.io/v1
Kind:         VolumeAttachment
Metadata:
  Creation Timestamp:  2020-06-23T17:12:45Z
  Finalizers:
    external-attacher/csi-vsphere-vmware-com
  Managed Fields:
    API Version:  storage.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        f:attacher:
        f:nodeName:
        f:source:
          f:inlineVolumeSpec:
            .:
            f:accessModes:
            f:csi:
              .:
              f:driver:
              f:fsType:
              f:volumeHandle:
            f:volumeMode:
    Manager:      kube-controller-manager
    Operation:    Update
    Time:         2020-06-23T17:12:45Z
    API Version:  storage.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:csi.alpha.kubernetes.io/node-id:
        f:finalizers:
          .:
          v:"external-attacher/csi-vsphere-vmware-com":
      f:status:
        f:attached:
        f:attachmentMetadata:
          .:
          f:diskUUID:
          f:type:
    Manager:         csi-attacher
    Operation:       Update
    Time:            2020-06-23T17:12:57Z
  Resource Version:  6189386
  Self Link:         /apis/storage.k8s.io/v1/volumeattachments/csi-49012ea45cfb0100fa19d9d590c76814cb853375b3f9c1348f66a727f948678a
  UID:               ac680a9b-abcb-477b-a752-5c19172a70ad
Spec:
  Attacher:   csi.vsphere.vmware.com
  Node Name:  k8s-node1
  Source:
    Inline Volume Spec:
      Access Modes:
        ReadWriteOnce
      Csi:
        Driver:         csi.vsphere.vmware.com
        Fs Type:        ext4
        Volume Handle:  [vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk
      Volume Mode:      Filesystem
Status:
  Attached:  true
  Attachment Metadata:
    Disk UUID:  6000c29a8dfbdecd9788e70763c1653f
    Type:       vSphere CNS Block Volume
Events:         <none>


# kubectl get Cnsvspherevolumemigrations
NAME                                   AGE
31fb23f2-15f1-4fb9-94aa-214999668649   3m15s




# kubectl describe Cnsvspherevolumemigrations
Name:         31fb23f2-15f1-4fb9-94aa-214999668649
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  cns.vmware.com/v1alpha1
Kind:         CnsVSphereVolumeMigration
Metadata:
  Creation Timestamp:  2020-06-23T17:12:47Z
  Generation:          1
  Managed Fields:
    API Version:  cns.vmware.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:volumeid:
        f:volumepath:
    Manager:         vsphere-csi
    Operation:       Update
    Time:            2020-06-23T17:12:47Z
  Resource Version:  6189353
  Self Link:         /apis/cns.vmware.com/v1alpha1/cnsvspherevolumemigrations/31fb23f2-15f1-4fb9-94aa-214999668649
  UID:               8e6eac22-a6ae-404a-997e-b13cdacf6d80
Spec:
  Volumeid:    31fb23f2-15f1-4fb9-94aa-214999668649
  Volumepath:  [vsanDatastore] 0c6ad55e-88c2-e431-bccb-02000359156d/kubernetes-dynamic-pvc-b1b87f44-7f4c-453a-b740-5c307a6a956b.vmdk
Events:        <none>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment