The Ubuntu Core installation fails if one of the required seeded snaps fail to start.
When a required snap fails, the system starts shutting down, snapd stops, and we end up with the following strange error:
cannot obtain system details: cannot communicate with server: Get http://localhost/v2/system-info: dial unix /run/snapd.socket: connect: connection refused
or
/usr/share/subiquity/console-conf-wrapper: line 32: snap: command not found
This makes it very hard to debug the installation issues and fix them.
The following change increases the verbosity of logs on Ubuntu Core and opens a shell console to allow interactive debugging after a failure on QEMU. For details on how to boot an Ubuntu Core image with QEMU, refer to Testing Ubuntu Core with QEMU and Image building.
Note that this requires changes to the gadget snap and has been tested with Core20 AMD64 PC Gadget.
Create a directory and file named kernel-options/cmdline.full with the following content:
# this is the first serial console, pick a suitable one for your device
console=ttyS0
# set to get more output from snapd
#snapd.debug=1
# these will output more device status/logging to our console,
# which is kinda annoying in that it goes to the same place we
# have our shell, but I think it's more useful to be able to see
# than have a perfectly pristine console, if you have multiple
# consoles you could of course use a different console for
# systemd output and your debug shell
rd.systemd.journald.forward_to_console=1
systemd.journald.forward_to_console=1
# to disable console-conf from running on ttyS0 so we can setup
# our debug shell there, this is the most key part here, without
# this console-conf will kill/prevent from starting our debug shell
systemd.mask=serial-getty@ttyS0.service
# create our debug shell on ttyS0
systemd.debug-shell=ttyS0
# needed in order to allow the debug-shell to run on UC20
dangerous
# standard Ubuntu Core kernel command line parameters
panic=-1Make it executable:
chmod +x kernel-options/cmdline.fullAdd the following as a part to snapcraft.yaml file of the gadget snap:
# Add kernel options for extended logging and a debugging shell
kernel-options:
source: kernel-options/
plugin: dumpNow, whenever things go wrong, press enter in the terminal and start typing commands!
The snap commands won't work because snapd has stopped.
You can query the snap logs using journalctl.
Warning
This is only works when using QEMU and should not be used elsewhere.Before flashing the image on disk, make sure to (1) comment out the
kernel-optionspart, (2) clean the gadget build, (3) re-build the gadget, and (4) re-build image. Otherwise, you will end up with no logs during the installation.