Hyper-v aks-engine

notes.md

ctr -n k8s.io image list

difference between hyperv and isolated

• [plugins.cri.containerd.runtimes.runhcs-wcow-hypervisor]
• [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runhcs-wcow-process]

https://github.com/Azure/aks-engine/compare/master...marosset:hyper-v https://github.com/Azure/aks-engine/blob/master/parts/k8s/windowscontainerdfunc.ps1

https://k8swin.blob.core.windows.net/k8s-windows/aks-engine-ce5c82940-marosset-hyperv.tar.gz

How to debug?

hyperv: wpr -start ContainerPlatform.wprp wpr -stop trace.etl

Get-WinEvent -LogName Microsoft-Windows-Hyper-V-Compute-Operational | Select-Object -Property TimeCreated, Id, LevelDisplayName, Message | Sort-Object TimeCreated | Export-Csv -Path "$ENV:TEMP\$($timeStamp)_hyper-v-compute-operational.csv"

collect all the logs:

• Invoke-WebRequest -UseBasicParsing https://raw.githubusercontent.com/Azure/aks-engine/master/scripts/collect-windows-logs.ps1 | Invoke-Expression

• curl https://raw.githubusercontent.com/kubernetes-sigs/windows-testing/master/scripts/win-ci-logs-collector.sh (which uses aks-engine collector script)

find a uvm that a container is associated too:

ctr -n k8s.io containers info ae51fa4e4966ad8d050f3d489e0d9978732c5390bacba6b40d9b20136b2a11ea

....
 "annotations": {
            "io.kubernetes.container.hash": "649f8e94",
            "io.kubernetes.container.ports": "[{\"containerPort\":80,\"protocol\":\"TCP\"}]",
            "io.kubernetes.container.restartCount": "0",
            "io.kubernetes.container.terminationMessagePath": "/dev/termination-log",
            "io.kubernetes.container.terminationMessagePolicy": "File",
            "io.kubernetes.cri.container-type": "container",
            "io.kubernetes.cri.sandbox-id": "71c185f8f27fd1c2c9402cc5e7cba55784e48f97a75150fbccd97a7dd89170bf",
            "io.kubernetes.pod.terminationGracePeriod": "30"
        },
....

notice 71c185f8f27fd1c2c9402cc5e7cba55784e48f97a75150fbccd97a7dd89170bf is a VM

hcsdiag list
0bac649da1d1a8057cc06529b2328e18d88a213259336f34a2bab47555500f8b    Windows Server Container,   Running,                                     , containerd-shim-runhcs-v1.exe
71c185f8f27fd1c2c9402cc5e7cba55784e48f97a75150fbccd97a7dd89170bf@vm VM,                       Running, 60A8670F-AC19-5C1B-9CBC-40E54E9CF746, containerd-shim-runhcs-v1.exe
daf84a7fd018900e342884d63324785fc7c9f59d0247d90f09bf4841b8aa9a54     Windows Server Container,   Running,                                     , containerd-shim-runhcs-v1.exe

See pod sandbox level

• 0 (default) is process isolated
• 1 is hyperv

crictl --runtime-endpoint "npipe:////./pipe/containerd-containerd" inspect 2663742ccc4c0 | Select-String sandbox_isolation

 "sandbox_isolation": 1