Firecracker vs QEMU Boot Differences

gistfile1.md

Most of the difference comes from the virtio-blk boot. Systemd to kata-agent boot times are similar

[    1.002949] Freeing unused kernel image memory: 2016K                                                                                                                           |[    0.844629] Freeing unused kernel image memory: 2016K
[    1.018758] Freeing unused kernel image memory: 1204K                                                                                                                           |[    0.844878] Freeing unused kernel image memory: 1204K
[    1.018838] Run /usr/lib/systemd/systemd as init process                                                                                                                        |[    0.845072] Run /usr/lib/systemd/systemd as init process
[    1.248975] systemd[1]: systemd 241 running in system mode. (+PAM +AUDIT -SELINUX +IMA -APPARMOR -SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +B|[    1.004513] systemd[1]: systemd 241 running in system mode. (+PAM +AUDIT -SELINUX +IMA -APPARMOR -SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +B
LKID +ELFUTILS +KMOD -IDN2 -IDN -PCRE2 default-hierarchy=legacy)                                                                                                                   |LKID +ELFUTILS +KMOD -IDN2 -IDN -PCRE2 default-hierarchy=legacy)
[    1.248989] systemd[1]: No virtualization found in DMI                                                                                                                          |[    1.005417] systemd[1]: No virtualization found in DMI
[    1.249051] systemd[1]: Virtualization found, CPUID=KVMKVMKVM                                                                                                                   |[    1.005679] systemd[1]: Virtualization found, CPUID=KVMKVMKVM
[    1.249054] systemd[1]: Found VM virtualization kvm                                                                                                                             |[    1.006121] systemd[1]: Found VM virtualization kvm
[    1.249057] systemd[1]: Detected virtualization kvm.                                                                                                                            |[    1.006742] systemd[1]: Detected virtualization kvm.
[    1.249070] systemd[1]: Detected architecture x86-64.                                                                                                                           |[    1.007170] systemd[1]: Detected architecture x86-64.
[    1.249219] systemd[1]: Mounting cgroup to /sys/fs/cgroup/net_cls,net_prio of type cgroup with options net_cls,net_prio.                                                        |[    1.007487] systemd[1]: Mounting cgroup to /sys/fs/cgroup/perf_event of type cgroup with options perf_event.
[    1.249470] systemd[1]: Mounting cgroup to /sys/fs/cgroup/devices of type cgroup with options devices.                                                                          |[    1.008095] systemd[1]: Mounting cgroup to /sys/fs/cgroup/cpuset of type cgroup with options cpuset.
[    1.249616] systemd[1]: Mounting cgroup to /sys/fs/cgroup/pids of type cgroup with options pids.                                                                                |[    1.008793] systemd[1]: Mounting cgroup to /sys/fs/cgroup/cpu,cpuacct of type cgroup with options cpu,cpuacct.
[    1.249760] systemd[1]: Mounting cgroup to /sys/fs/cgroup/freezer of type cgroup with options freezer.                                                                          |[    1.014317] systemd[1]: Mounting cgroup to /sys/fs/cgroup/freezer of type cgroup with options freezer.
[    1.755766] systemd-bless-boot-generator[65]: Skipping generator, not an EFI boot.                                                                                              |[    1.121243] systemd-fstab-generator[72]: Parsing /etc/fstab
[    1.927883] systemd-gpt-auto-generator[70]: Failed to chase block device '/', ignoring: No such file or directory                                                               |[    1.131252] systemd-gpt-auto-generator[74]: Failed to chase block device '/', ignoring: No such file or directory
[    1.931486] systemd-fstab-generator[68]: Parsing /etc/fstab                                                                                                                     |[    1.160818] systemd-bless-boot-generator[69]: Skipping generator, not an EFI boot.
[    1.947422] systemd-gpt-auto-generator[70]: vda1: Root device /dev/vda.                                                                                                         |[    1.165828] systemd-gpt-auto-generator[74]: pmem0p1: Root device /dev/pmem0.
[    2.027787] systemd-gpt-auto-generator[70]: No suitable partition table found, ignoring.                                                                                        |[    1.186432] systemd-getty-generator[73]: Automatically adding serial getty for /dev/hvc0.
[    2.486653] systemd[72]: systemd-sysctl.service: Failed to connect stdout to the journal socket, ignoring: No such file or directory                                            |[    1.194498] systemd-getty-generator[73]: Automatically adding serial getty for /dev/hvc0.
[    2.486728] systemd[72]: systemd-sysctl.service: Executing: /usr/lib/systemd/systemd-sysctl                                                                                     |[    1.218441] systemd-gpt-auto-generator[74]: No suitable partition table found, ignoring.
[    2.491182] systemd-sysctl[72]: Parsing /usr/lib/sysctl.d/50-coredump.conf                                                                                                      |[    1.360185] systemd[76]: systemd-sysctl.service: Failed to connect stdout to the journal socket, ignoring: No such file or directory
[    2.491424] systemd-sysctl[72]: Parsing /usr/lib/sysctl.d/50-default.conf                                                                                                       |[    1.361103] systemd[76]: systemd-sysctl.service: Executing: /usr/lib/systemd/systemd-sysctl
[    2.491914] systemd-sysctl[72]: Setting 'kernel/core_pattern' to '|/usr/lib/systemd/coredump-wrapper %E %P %u %g %s %t %c %h %e'.                                               |[    1.366309] systemd-sysctl[76]: Parsing /usr/lib/sysctl.d/50-coredump.conf
[    2.491925] systemd-sysctl[72]: Setting 'kernel/sysrq' to '16'.                                                                                                                 |[    1.366637] systemd-sysctl[76]: Parsing /usr/lib/sysctl.d/50-default.conf
[    2.491932] systemd-sysctl[72]: Couldn't write '16' to 'kernel/sysrq', ignoring: No such file or directory                                                                      |[    1.368742] systemd-sysctl[76]: Setting 'kernel/core_pattern' to '|/usr/lib/systemd/coredump-wrapper %E %P %u %g %s %t %c %h %e'.
[    2.491934] systemd-sysctl[72]: Setting 'kernel/core_uses_pid' to '1'.                                                                                                          |[    1.370046] systemd-sysctl[76]: Setting 'kernel/sysrq' to '16'.
[    2.491941] systemd-sysctl[72]: Setting 'net/ipv4/conf/all/rp_filter' to '2'.                                                                                                   |[    1.371051] systemd-sysctl[76]: Couldn't write '16' to 'kernel/sysrq', ignoring: No such file or directory
[    2.492018] systemd-sysctl[72]: Setting 'net/ipv4/conf/all/accept_source_route' to '0'.                                                                                         |[    1.371402] systemd-sysctl[76]: Setting 'kernel/core_uses_pid' to '1'.
[    2.492026] systemd-sysctl[72]: Setting 'net/ipv4/conf/all/promote_secondaries' to '1'.                                                                                         |[    1.371634] systemd-sysctl[76]: Setting 'net/ipv4/conf/all/rp_filter' to '2'.
[    2.492033] systemd-sysctl[72]: Setting 'net/core/default_qdisc' to 'fq'.                                                                                                       |[    1.371926] systemd-sysctl[76]: Setting 'net/ipv4/conf/all/accept_source_route' to '0'.
[    2.492660] systemd-sysctl: 4 output lines suppressed due to ratelimiting                                                                                                       |[    1.374050] systemd-sysctl[76]: Setting 'net/ipv4/conf/all/promote_secondaries' to '1'.
[    2.500007] systemd[78]: kata-agent.service: Executing: /usr/bin/kata-agent                                                                                                     |[    1.374470] systemd-sysctl[76]: Setting 'net/core/default_qdisc' to 'fq'.
[    3.730852] virtio_blk virtio1: [vdb] new size: 0 512-byte logical blocks (0 B/0 B)                                                                                             |[    1.376157] systemd-sysctl: 4 output lines suppressed due to ratelimiting
[    3.732979] virtio_blk virtio1: [vdb] new size: 0 512-byte logical blocks (0 B/0 B)                                                                                             |[    1.396514] systemd[80]: kata-agent.service: Executing: /usr/bin/kata-agent
[    3.777937] XFS (vdb): Mounting V5 Filesystem                                                                                                                                   |[    2.259826] scsi 0:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
[    3.845988] XFS (vdb): Ending clean mount                                                                                                                                       |[    2.260448] sd 0:0:0:0: Attached scsi generic sg0 type 0
                                                                                                                                                                                   |[    2.260656] sd 0:0:0:0: Power-on or device reset occurred
                                                                                                                                                                                   |[    2.261459] sd 0:0:0:0: [sda] 20971520 512-byte logical blocks: (10.7 GB/10.0 GiB)
                                                                                                                                                                                   |[    2.261743] sd 0:0:0:0: [sda] Write Protect is off
                                                                                                                                                                                   |[    2.261891] sd 0:0:0:0: [sda] Mode Sense: 63 00 00 08
                                                                                                                                                                                   |[    2.262288] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
                                                                                                                                                                                   |[    2.266875] sd 0:0:0:0: [sda] Attached SCSI disk
                                                                                                                                                                                   |[    2.282785] pci 0000:00:02.0: PCI bridge to [bus 01]
                                                                                                                                                                                   |[    2.283206] pci 0000:00:02.0:   bridge window [io  0xc000-0xcfff]
                                                                                                                                                                                   |[    2.284469] pci 0000:00:02.0:   bridge window [mem 0xfe400000-0xfe5fffff]
                                                                                                                                                                                   |[    2.285420] pci 0000:00:02.0:   bridge window [mem 0xfe800000-0xfe9fffff 64bit pref]
                                                                                                                                                                                   |[    2.290822] XFS (sda): Mounting V5 Filesystem
                                                                                                                                                                                   |[    2.322011] XFS (sda): Ending clean mount
                                                                                                                                                                                   
                                                                                                                                                                                   ```

MP Boot Sequence 0.2s difference. But 0.4 extra in SMP Setup in firecracker. So FC is much faster till it does SMP boot

[    0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'                                                                                                            |[    0.379360] MTRR fixed ranges enabled:
[    0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'                                                                                                        |[    0.379361]   00000-9FFFF write-back
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256                                                                                                              |[    0.379361]   A0000-BFFFF uncachable
[    0.000000] x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64                                                                                                              |[    0.379362]   C0000-FFFFF write-protect
[    0.000000] x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64                                                                                                              |[    0.379362] MTRR variable ranges enabled:
[    0.000000] x86/fpu: xstate_offset[5]:  960, xstate_sizes[5]:   64                                                                                                              |[    0.379363]   0 base 0080000000 mask FF80000000 uncachable
[    0.000000] x86/fpu: xstate_offset[6]: 1024, xstate_sizes[6]:  512                                                                                                              |[    0.379363]   1 disabled
[    0.000000] x86/fpu: xstate_offset[7]: 1536, xstate_sizes[7]: 1024                                                                                                              |[    0.379364]   2 disabled
[    0.000000] x86/fpu: Enabled xstate features 0xff, context size is 2560 bytes, using 'compacted' format.                                                                        |[    0.379364]   3 disabled
[    0.000000] BIOS-provided physical RAM map:                                                                                                                                     |[    0.379365]   4 disabled
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable                                                                                                       |[    0.379365]   5 disabled
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007fffffff] usable                                                                                                       |[    0.379365]   6 disabled
[    0.000000] NX (Execute Disable) protection: active                                                                                                                             |[    0.379366]   7 disabled
[    0.000000] Hypervisor detected: KVM                                                                                                                                            |[    0.379547] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00                                                                                                                         |[    0.383690] found SMP MP-table at [mem 0x000f5ac0-0x000f5acf] mapped at [(____ptrval____)]
[    0.115955] kvm-clock: cpu 0, msr 24ae001, primary cpu clock                                                                                                                    |[    0.383697] Base memory trampoline at [(____ptrval____)] 99000 size 24576
[    0.115956] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns                                                            |[    0.383806] Using GB pages for direct mapping
[    0.116015] tsc: Detected 2592.016 MHz processor                                                                                                                                |[    0.383829] BRK [0x02600000, 0x02600fff] PGTABLE
[    0.117624] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved                                                                                                        |[    0.383831] BRK [0x02601000, 0x02601fff] PGTABLE
[    0.117627] e820: remove [mem 0x000a0000-0x000fffff] usable                                                                                                                     |[    0.383831] BRK [0x02602000, 0x02602fff] PGTABLE
[    0.117629] last_pfn = 0x80000 max_arch_pfn = 0x400000000                                                                                                                       |[    0.383846] BRK [0x02603000, 0x02603fff] PGTABLE
[    0.118206] MTRR default type: uncachable                                                                                                                                       |[    0.383898] BRK [0x02604000, 0x02604fff] PGTABLE
[    0.118207] MTRR fixed ranges disabled:                                                                                                                                         |[    0.383981] ACPI: Early table checksum verification disabled
[    0.118208]   00000-FFFFF uncachable                                                                                                                                            |[    0.384016] ACPI: RSDP 0x00000000000F58C0 000014 (v00 BOCHS )
[    0.118209] MTRR variable ranges disabled:                                                                                                                                      |[    0.384022] ACPI: RSDT 0x000000007FFE1E63 00003C (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.118209]   0 disabled                                                                                                                                                        |[    0.384026] ACPI: FACP 0x000000007FFE16CA 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.118210]   1 disabled                                                                                                                                                        |[    0.384029] ACPI: DSDT 0x000000007FFDF040 00268A (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.118210]   2 disabled                                                                                                                                                        |[    0.384031] ACPI: FACS 0x000000007FFDF000 000040
[    0.118210]   3 disabled                                                                                                                                                        |[    0.384033] ACPI: APIC 0x000000007FFE17BE 0000B0 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.118211]   4 disabled                                                                                                                                                        |[    0.384035] ACPI: HPET 0x000000007FFE186E 000038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
[    0.118211]   5 disabled                                                                                                                                                        |[    0.384037] ACPI: SRAT 0x000000007FFE18A6 000150 (v01 BOCHS  BXPCSRAT 00000001 BXPC 00000001)
[    0.118211]   6 disabled                                                                                                                                                        |[    0.384039] ACPI: SSDT 0x000000007FFE19F6 00038D (v01 BOCHS  NVDIMM   00000001 BXPC 00000001)
[    0.118212]   7 disabled                                                                                                                                                        |[    0.384041] ACPI: NFIT 0x000000007FFE1D83 0000E0 (v01 BOCHS  BXPCNFIT 00000001 BXPC 00000001)
[    0.118372] Disabled                                                                                                                                                            |[    0.384045] ACPI: Local APIC address 0xfee00000
[    0.118528] CPU MTRRs all blank - virtualized system.                                                                                                                           |[    0.385443] Zone ranges:
[    0.118549] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC                                                                                                        |[    0.385445]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.118684] found SMP MP-table at [mem 0x0009fc00-0x0009fc0f] mapped at [(____ptrval____)]                                                                                      |[    0.385446]   DMA32    [mem 0x0000000001000000-0x000000007ffdefff]
[    0.118984] Base memory trampoline at [(____ptrval____)] 99000 size 24576                                                                                                       |[    0.385447]   Normal   empty
[    0.119156] Using GB pages for direct mapping                                                                                                                                   |[    0.385447]   Device   empty
[    0.119188] BRK [0x02600000, 0x02600fff] PGTABLE                                                                                                                                |[    0.385448] Movable zone start for each node
[    0.119191] BRK [0x02601000, 0x02601fff] PGTABLE                                                                                                                                |[    0.385448] Early memory node ranges
[    0.119192] BRK [0x02602000, 0x02602fff] PGTABLE                                                                                                                                |[    0.385449]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.119748] Zone ranges:                                                                                                                                                        |[    0.385450]   node   0: [mem 0x0000000000100000-0x000000007ffdefff]
[    0.119807]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]                                                                                                              |[    0.385598] Reserved but unavailable: 98 pages
[    0.119810]   DMA32    [mem 0x0000000001000000-0x000000007fffffff]                                                                                                              |[    0.385599] Initmem setup node 0 [mem 0x0000000000001000-0x000000007ffdefff]
[    0.119811]   Normal   empty                                                                                                                                                    |[    0.385600] On node 0 totalpages: 524157
[    0.119837]   Device   empty                                                                                                                                                    |[    0.385601]   DMA zone: 64 pages used for memmap
[    0.119841] Movable zone start for each node                                                                                                                                    |[    0.385602]   DMA zone: 21 pages reserved
[    0.119841] Early memory node ranges                                                                                                                                            |[    0.385602]   DMA zone: 3998 pages, LIFO batch:0
[    0.119842]   node   0: [mem 0x0000000000001000-0x000000000009efff]                                                                                                             |[    0.385640]   DMA32 zone: 8128 pages used for memmap
[    0.119843]   node   0: [mem 0x0000000000100000-0x000000007fffffff]                                                                                                             |[    0.385641]   DMA32 zone: 520159 pages, LIFO batch:63
[    0.119936] Reserved but unavailable: 98 pages                                                                                                                                  |[    0.399497] ACPI: Local APIC address 0xfee00000
[    0.119989] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffffff]                                                                                                    |[    0.399541] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.119994] On node 0 totalpages: 524190                                                                                                                                        |[    0.399811] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.120030]   DMA zone: 64 pages used for memmap                                                                                                                                |[    0.399813] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.120031]   DMA zone: 21 pages reserved                                                                                                                                       |[    0.399814] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.120032]   DMA zone: 3998 pages, LIFO batch:0                                                                                                                                |[    0.399815] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.122054]   DMA32 zone: 8128 pages used for memmap                                                                                                                            |[    0.399815] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.122055]   DMA32 zone: 520192 pages, LIFO batch:63                                                                                                                           |[    0.399816] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.520655] Intel MultiProcessor Specification v1.4                                                                                                                             |[    0.399817] ACPI: IRQ0 used by override.
[    0.520664] MPTABLE: OEM ID: FC                                                                                                                                                 |[    0.399818] ACPI: IRQ5 used by override.
[    0.520665] MPTABLE: Product ID: 000000000000                                                                                                                                   |[    0.399818] ACPI: IRQ9 used by override.
[    0.520666] MPTABLE: APIC at: 0xFEE00000                                                                                                                                        |[    0.399819] ACPI: IRQ10 used by override.
[    0.520789] Processor #0 (Bootup-CPU)                                                                                                                                           |[    0.399819] ACPI: IRQ11 used by override.
[    0.521050] IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23                                                                                                      |[    0.399820] Using ACPI (MADT) for SMP configuration information
[    0.521052] Processors: 1                                                                                                                                                       |[    0.399822] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.521086] smpboot: Allowing 1 CPUs, 0 hotplug CPUs                                                                                                                            |[    0.399825] smpboot: Allowing 8 CPUs, 7 hotplug CPUs
[    0.521348] [mem 0x80000000-0xffffffff] available for PCI devices                                                                                                               |[    0.399928] [mem 0x80000000-0xfeffbfff] available for PCI devices
[    0.521350] Booting paravirtualized kernel on KVM                                                                                                                               |[    0.399929] Booting paravirtualized kernel on KVM
[    0.521352] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns                                                             |[    0.399931] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[    0.579117] setup_percpu: NR_CPUS:240 nr_cpumask_bits:240 nr_cpu_ids:1 nr_node_ids:1                                                                                            |[    0.448397] setup_percpu: NR_CPUS:240 nr_cpumask_bits:240 nr_cpu_ids:8 nr_node_ids:1
[    0.602757] percpu: Embedded 42 pages/cpu @(____ptrval____) s132888 r8192 d30952 u2097152                                                                                       |[    0.448698] percpu: Embedded 42 pages/cpu @(____ptrval____) s132888 r8192 d30952 u262144
[    0.602767] pcpu-alloc: s132888 r8192 d30952 u2097152 alloc=1*2097152                                                                                                           |[    0.448703] pcpu-alloc: s132888 r8192 d30952 u262144 alloc=1*2097152
[    0.602768] pcpu-alloc: [0] 0                                                                                                                                                   |[    0.448704] pcpu-alloc: [0] 0 1 2 3 4 5 6 7
[    0.602939] KVM setup async PF for cpu 0                                                                                                                                        |[    0.448771] KVM setup async PF for cpu 0
[    0.602998] kvm-stealtime: cpu 0, msr 7da14e80                                                                                                                                  |[    0.448828] kvm-stealtime: cpu 0, msr 7da14e80
[    0.603101] Built 1 zonelists, mobility grouping on.  Total pages: 515977                                                                                                       |[    0.448832] Built 1 zonelists, mobility grouping on.  Total pages: 515944
[    0.603105] Kernel command line: init=/usr/lib/systemd/systemd systemd.unit=kata-containers.target systemd.mask=systemd-networkd.service systemd.mask=systemd-networkd.socket sy|[    0.448834] Kernel command line: tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=
stemd.mask=systemd-journald.service systemd.mask=systemd-journald.socket systemd.mask=systemd-journal-flush.service systemd.mask=systemd-udevd.service systemd.mask=systemd-udevd.s|hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro ro rootfstype=ext4 debug systemd.show_stat
ocket systemd.mask=systemd-udev-trigger.service systemd.mask=systemd-timesyncd.service systemd.mask=systemd-update-utmp.service systemd.mask=systemd-tmpfiles-setup.service systemd|us=true systemd.log_level=debug panic=1 nr_cpus=8 agent.use_vsock=true init=/usr/lib/systemd/systemd systemd.unit=kata-containers.target systemd.mask=systemd-networkd.service syst
.mask=systemd-tmpfiles-cleanup.service systemd.mask=systemd-tmpfiles-cleanup.timer systemd.mask=tmp.mount systemd.mask=systemd-random-seed.service agent.log=debug root=/dev/vda1 r|emd.mask=systemd-networkd.socket systemd.mask=systemd-journald.service systemd.mask=systemd-journald.socket systemd.mask=systemd-journal-flush.service systemd.mask=systemd-udevd.s
ootflags=data=ordered,errors=remount-ro ro rootfstype=ext4 tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplac|ervice systemd.mask=systemd-udevd.socket systemd.mask=systemd-udev-trigger.service systemd.mask=systemd-timesyncd.service systemd.mask=systemd-update-utmp.service systemd.mask=sys
e-smp reboot=k iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug systemd.show_status=true systemd                                                                      |temd-tmpfiles-setup.service systemd.mask=systemd-tmpfiles-cleanup.service systemd.mask=systemd-tmpfiles-clean