Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Ubuntu 20.04 user-data files for autoinstall ISO
#cloud-config
autoinstall:
version: 1
refresh-installer: # start with an up-to-date installer
update: yes
interactive-sections: # Install groups listed here will wait for user input
- storage
storage: # should set the interactive default but doesn't seem to work??
layout:
name: direct
locale: en_US.UTF-8
keyboard:
layout: us
identity: # This is section you may want to add to interactive-sections (user name and password are ubuntu here)
hostname: puget-000
password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
username: ubuntu
ssh:
allow-pw: true
install-server: true
apt:
sources:
ignored1: # This is here to get the yaml formatting right when adding a ppa
source: ppa:graphics-drivers/ppa
packages:
- build-essential
- network-manager
- dkms
- emacs-nox
- ubuntu-desktop-minimal
package_update: true
package_upgrade: true
late-commands:
# Changing from networkd to NetworkManager
# move existing config out of the way
- find /target/etc/netplan/ -name "*.yaml" -exec sh -c 'mv "$1" "$1-orig"' _ {} \;
# Create a new netplan and enable it
- |
cat <<EOF | sudo tee /target/etc/netplan/01-netcfg.yaml
network:
version: 2
renderer: NetworkManager
EOF
- curtin in-target --target /target netplan generate
- curtin in-target --target /target netplan apply
- curtin in-target --target /target systemctl enable NetworkManager.service
# Write a script that can take care of some post install setup "late-commands" cannot be interactive unfortunately"
# - |
# cat <<EOF | sudo tee /target/etc/finish-install-setup.sh
# #!/usr/bin/env bash
# echo *************************
# echo **** Finish Setup ****
# echo *************************
# echo 'Enter the hostname for this system: '
# read NEW_HOSTNAME
# hostnamectl set-hostname \${NEW_HOSTNAME}
# echo
# echo 'Enter the timezone for this system: '
# echo 'America/Los_Angeles America/Denver America/Chicago America/New_York'
# read NEW_TIMEZONE
# timedatectl set-timezone \${NEW_TIMEZONE}
# echo *************************
# echo
# echo *************************
# echo 'Restarting to finish ...'
# shutdown -r 3
# EOF
# - curtin in-target --target /target chmod 744 /etc/finish-install-setup.sh
- cp /target/cdrom/extras/Puget_Systems.png /target/usr/share/backgrounds/
user-data: # Commands here run during first boot (cannot be interactive)
runcmd:
# Install the NVIDIA driver from the ppa we setup earlier
- [apt-get, update]
- [apt-get, dist-upgrade, --yes]
- [apt, autoremove, --yes]
- [apt-get, install, --yes, nvidia-driver-470] #, --no-install-recommends]
- [sudo, -u, ubuntu, dbus-launch, gsettings, set, org.gnome.desktop.background, picture-uri, file:///usr/share/backgrounds/Puget_Systems.png]
# - |
# #!/usr/bin/env bash
# echo ''
# echo '***************************************'
# echo ' To complete install setup please run, '
# echo ' sudo /etc/finish-install-setup.sh'
# echo '***************************************'
# echo ''
#cloud-config
autoinstall:
version: 1
refresh-installer: # start with an up-to-date installer
update: yes
interactive-sections: # Install groups listed here will wait for user input
- storage
storage: # should set the interactive default but doesn't seem to work??
layout:
name: direct
locale: en_US.UTF-8
keyboard:
layout: us
identity: # This is section you may want to add to interactive-sections (user name and password are ubuntu here)
hostname: puget-000
password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
username: ubuntu
ssh:
allow-pw: true
install-server: true
apt:
sources:
ignored1: # This is here to get the yaml formatting right when adding a ppa
source: ppa:graphics-drivers/ppa
packages:
- build-essential
- network-manager
- dkms
- emacs-nox
#- ubuntu-desktop-minimal^
package_update: true
package_upgrade: true
late-commands:
# Changing from networkd to NetworkManager
# move existing config out of the way
- find /target/etc/netplan/ -name "*.yaml" -exec sh -c 'mv "$1" "$1-orig"' _ {} \;
# Create a new netplan and enable it
- |
cat <<EOF | sudo tee /target/etc/netplan/01-netcfg.yaml
network:
version: 2
renderer: NetworkManager
EOF
- curtin in-target --target /target netplan generate
- curtin in-target --target /target netplan apply
- curtin in-target --target /target systemctl enable NetworkManager.service
# Write a script that can take care of some post install setup "late-commands" cannot be interactive unfortunately"
# - |
# cat <<EOF | sudo tee /target/etc/finish-install-setup.sh
# #!/usr/bin/env bash
# echo *************************
# echo **** Finish Setup ****
# echo *************************
# echo 'Enter the hostname for this system: '
# read NEW_HOSTNAME
# hostnamectl set-hostname \${NEW_HOSTNAME}
# echo
# echo 'Enter the timezone for this system: '
# echo 'America/Los_Angeles America/Denver America/Chicago America/New_York'
# read NEW_TIMEZONE
# timedatectl set-timezone \${NEW_TIMEZONE}
# echo *************************
# echo
# echo *************************
# echo 'Restarting to finish ...'
# shutdown -r 3
# EOF
# - curtin in-target --target /target chmod 744 /etc/finish-install-setup.sh
- ls -l > /target/root/ls.out
- pwd > /target/root/pwd.out
- mount > /target/root/mount.out
- touch afile
- cp afile /target/root/
- ls -l / > /target/root/ls-root.out
- ls -l /target > /target/root/ls-target.out
- ls -l /target/cdrom > /target/root/ls-target-cdrom.out
user-data: # Commands here run during first boot (cannot be interactive)
runcmd:
# Install the NVIDIA driver from the ppa we setup earlier
- [apt-get, install, --yes, nvidia-driver-470, --no-install-recommends]
- |
#!/usr/bin/env bash
echo ''
echo '***************************************'
echo ' To complete install setup please run, '
echo ' sudo /etc/finish-install-setup.sh'
echo '***************************************'
echo ''
@zero-pytagoras
Copy link

zero-pytagoras commented Mar 22, 2022

is there any example of setting basic LVM ?

@cantoni
Copy link

cantoni commented Apr 8, 2022

is there any example of setting basic LVM ?

This stuff is really hard to write manually. The easiest solution I've found is to configure one machine manually, then use the storage section from the file generated at /var/log/installer/autoinstall-user-data.

@zero-pytagoras
Copy link

zero-pytagoras commented Apr 11, 2022

that's the hard part - it does not shows the structure I created in logs, and documentation is fuzzy and not clear at all: Not with examples and not with explanation itself

@dbkinghorn
Copy link
Author

dbkinghorn commented Apr 11, 2022

In general I find cloud init difficult to use, but preseed wasn't simple either. When I did my post (where this gist came from) I had to do a lot of trial and error. But, I think the more it gets used folks will work out the rough edges. I'm going to try it again in a few weeks with Ubuntu 22.04 and write up another post. Hopefully it will be less of a struggle to use this next time :-) (I'm not betting on that)

What cantoni suggested is probably the best way to get a storage config worked out.

  • Do a basic auto install with c-i using interactive storage config
  • Log into that system and pull the config from /var/log/installer/autoinstall-data
  • Use that in your new autoinstall-user-data file

I found this post which gives a more detailed example of doing exactly that.

https://www.golinuxcloud.com/generate-user-data-file-ubuntu-20-04/

The storage section with LVM config is below. You would probably need to modify something like that to be more general, and there may some parts that can be left out. But it would be a good start for a workable config.

  storage:
    config:
    - ptable: gpt
      serial: 3600508b1001c576619b6670156e25877
      wwn: '0x600508b1001c576619b6670156e25877'
      path: /dev/sda
      wipe: superblock
      preserve: false
      name: ''
      grub_device: true
      type: disk
      id: disk-sda
    - device: disk-sda
      size: 1048576
      flag: bios_grub
      number: 1
      preserve: false
      grub_device: false
      type: partition
      id: partition-3
    - device: disk-sda
      size: 1073741824
      wipe: superblock
      flag: ''
      number: 2
      preserve: false
      grub_device: false
      type: partition
      id: partition-4
    - fstype: ext4
      volume: partition-4
      preserve: false
      type: format
      id: format-2
    - device: disk-sda
      size: 899074228224
      wipe: superblock
      flag: ''
      number: 3
      preserve: false
      grub_device: false
      type: partition
      id: partition-5
    - name: ubuntu-vg
      devices:
      - partition-5
      preserve: false
      type: lvm_volgroup
      id: lvm_volgroup-1
    - name: ubuntu-lv
      volgroup: lvm_volgroup-1
      size: 107374182400B
      wipe: superblock
      preserve: false
      type: lvm_partition
      id: lvm_partition-1
    - fstype: ext4
      volume: lvm_partition-1
      preserve: false
      type: format
      id: format-3
    - path: /
      device: format-3
      type: mount
      id: mount-3
    - path: /boot
      device: format-2
      type: mount
      id: mount-2

@zero-pytagoras
Copy link

zero-pytagoras commented Apr 12, 2022

first of thank you for the answer.
I had chat with the golinux post writer, and he didn't have too much information to share either, yet I was able to find eventually a small example that enabled me to build my LVM as follows below:

   config:
     - {ptable: msdos, wipe: superblock-recursive, preserve: false, grub_device: true, path: /dev/ROOT_DEV, type: disk, id: disk-sda}
     - {type: partition, wipe: superblock-recursive, number: 1, device: disk-sda, flag: bios_grub, size: 1M, id: sda-grub}
     - {type: partition, wipe: superblock-recursive, number: 2, device: disk-sda, flag: boot, size: 1G, id: sda-boot}
     - {type: partition, wipe: superblock-recursive, number: 3, device: disk-sda, size: -1, id: sda-lvm}
     - {type: lvm_volgroup, name: vg-my, devices: [sda-lvm], id: vg-my}
     - {type: lvm_partition, volgroup: vg-my, id: lv-root, name: lv-root, size: 15G}
     - {type: lvm_partition, volgroup: vg-my, id: lv-var, name: lv-var, size: 10G}
     - {type: lvm_partition, volgroup: vg-my, id: lv-var-lib-docker, name: lv-var-lib-docker, size: 20G}
     - {type: lvm_partition, volgroup: vg-my, id: lv-var-log, name: lv-var-log, size: 10G}
     - {type: lvm_partition, volgroup: vg-my, id: lv-var-log-audit, name: lv-var-log-audit, size: 5G}
     - {type: lvm_partition, volgroup: vg-my, id: lv-var-tmp, name: lv-var-tmp, size: 10G}
     - {type: lvm_partition, volgroup: vg-my, id: lv-home, name: lv-home, size: -1} # this means to use all the left storage
     - {type: format, fstype: ext4, volume: sda-boot, id: sda-boot-fs}
     - {type: format, fstype: ext4, volume: lv-root, id: lv-root-fs}
     - {type: format, fstype: ext4, volume: lv-home, id: lv-home-fs}
     - {type: format, fstype: ext4, volume: lv-var, id: lv-var-fs}
     - {type: format, fstype: ext4, volume: lv-var-lib-docker, id: lv-var-lib-docker-fs}
     - {type: format, fstype: ext4, volume: lv-var-log, id: lv-var-log-fs}
     - {type: format, fstype: ext4, volume: lv-var-log-audit, id: lv-var-log-audit-fs}
     - {type: format, fstype: ext4, volume: lv-var-tmp, id: lv-var-tmp-fs}
     - {type: mount, path: /, id: m-root, device: lv-root-fs}
     - {type: mount, path: /boot, id: m-boot, device: sda-boot-fs}
     - {type: mount, path: /home, id: m-home, device: lv-home-fs}
     - {type: mount, path: /var, id: m-var, device: lv-var-fs}
     - {type: mount, path: /var/lib/docker, id: m-varllibdocker, device: lv-var-lib-docker-fs}
     - {type: mount, path: /var/log, id: m-varlog, device: lv-var-log-fs}
     - {type: mount, path: /var/log/audit, id: m-var-log-audit, device: lv-var-log-audit-fs}
     - {type: mount, path: /var/tmp, id: m-vartmp, device: lv-var-tmp-fs}

ROOT_DEV is a variable that i am substituting with sed with shell script that am running with early-commands of cloud-init. it was only way I was able to make it dynamic.I have tried to use manual install c-i file, but for some reason, it does not work, mainly lvm, write-file and additional parameters for the user, were not implemented with c-i.
Anyway - thanks again for response and guidance.

@dbkinghorn
Copy link
Author

dbkinghorn commented Apr 12, 2022

Thanks for posting that back here! That's a nice use of early-commands and good reference for a difficult aspect of using c-i

@vgervais
Copy link

vgervais commented Apr 25, 2022

Hi all. @dbkinghorn I used your How-to make a custom iso using cloud-init for a on prem server. We would like to simplify installing a crashed server obviously. Everything is fine and thank you for that. I cannot find any other how to and I have searched.
I have a few questions (because I am also new to yaml & cloud-init) and was wondering if you would assist?
My first question is simple: Does cloud-init (c-i) allow adding and installing from other repositories at initilal install with a yaml like yours?
Question 2, can I add multiple users at the same time during initial install.
Thanks in advance.

@zero-pytagoras
Copy link

zero-pytagoras commented Apr 26, 2022

Does cloud-init (c-i) allow adding and installing from other repositories at initilal install with a yaml like yours?
yes: you can add apt tag with address to your repository e.g.

apt:
    sources:
        source: deb.address.to.your.repo

@vgervais
Copy link

vgervais commented Apr 27, 2022

Thanks for the info @zero-pytagoras

@finalls
Copy link

finalls commented Apr 28, 2022

Hello, @zero-pytagoras !
ROOT_DEV is a variable that i am substituting with sed with shell script that am running with early-commands of cloud-init.
Could you tell me more about how to use this variable and give me please some examples of it.
Thank you very much!

@zero-pytagoras
Copy link

zero-pytagoras commented May 1, 2022

I am using shell script that runs sed command to substitute the ROOT_DEV as part of early_commands tags.
essentially i run this script

ROOT_DEV=$(lsblk -x TYPE|grep disk|awk '{print $1}')

sed -i 's/ROOT_DEV/${ROOT_DEV}/g'

as part of user-data file

- early-commands:
  - swap-drive-name.sh  

for my use case that was good enough, but might not suit a system where you have more then one drive

@vgervais
Copy link

vgervais commented May 2, 2022

@zero-pytagoras: After I have successfully added the repo, how do I update it, or install the software from it? In my case, I want to install the latest postgresql-client.

@finalls
Copy link

finalls commented May 2, 2022

Hello, @zero-pytagoras !
What file do you change seed to make to work the variable in the config section?
let's give an example, it doesn't work

#cloud-config
autoinstall:
  version: 1
  early-commands:
    - /bin/bash /tmp/hdd.sh
    - /usr/bin/echo "Disk for install - $DISK"

  storage:
    layout:
      name: lvm
      match:
        path: $DISK

Thank you very much!

@zero-pytagoras
Copy link

zero-pytagoras commented May 8, 2022

@vgervais
use tag packages and add to it list of packages you would like to install, for example:

apt:
  repo: link-toyour-repo
packages:
  - ipython
  - postgresql-client

@zero-pytagoras
Copy link

zero-pytagoras commented May 8, 2022

@finalls
not sure i understand what you are trying to do. who is giving you value of $DISK ?
in my case i used early-commands to run script and to edit with sed the user-data file itself. kind of editing script from i am running myself

@vgervais
Copy link

vgervais commented May 10, 2022

@zero-pytagoras Thanks for the info. My yaml is now working 100%.

@sedasdas
Copy link

sedasdas commented May 31, 2022

@zero-pytagoras Thanks for the info. My yaml is now working 100%.

@finalls not sure i understand what you are trying to do. who is giving you value of ? in my case i used to run script and to edit with the user-data file itself. kind of editing script from i am running myself$DISK``early-commands``sed

#cloud-config
autoinstall:
early-commands:
- ROOT_DEV=$(lsblk -x TYPE|grep disk|grep G|awk '{print $1}')
it is my script , but the ROOT_DEV does not worker ? why

@zero-pytagoras
Copy link

zero-pytagoras commented May 31, 2022

because it is yaml - it can not run shell scripts directly.
you need to run it separate file/script and redirect its value to your current user-data file

@Bartsch462
Copy link

Bartsch462 commented Jun 14, 2022

@zero-pytagoras
I'm also trying to define a path using a script in the early commands. Would you know which user-data file to edit with sed so that it is used in the installation? I've tried some of the files in
/var/lib/cloud/instances/nocloud and /var/lib/cloud/instance
but I've had no luck yet.

@zero-pytagoras
Copy link

zero-pytagoras commented Jun 14, 2022

the default file to be update needs to /automation, but you could test it on failed run.

@Bartsch462
Copy link

Bartsch462 commented Jun 14, 2022

the default file to be update needs to /automation, but you could test it on failed run.

I needed to update the /autoinstall.yaml but thanks a lot for pointing me in the right direction! Works great now!

@SohMahmood
Copy link

SohMahmood commented Sep 10, 2022

Thank you so much for this template.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment