Skip to content

Instantly share code, notes, and snippets.

@mgerdts
Last active June 16, 2020 20:55
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mgerdts/a37e671461df84e0c94857ecdabd6122 to your computer and use it in GitHub Desktop.
Save mgerdts/a37e671461df84e0c94857ecdabd6122 to your computer and use it in GitHub Desktop.
Upgrade cloud-init on CentOS 6

This shows how to upgrade cloud-init from the one found in the early CentOS 6 bhyve images and create a new image from that.

Create a source instance

First, we will install an instance using image 11891f44-a374-11e8-9c35-c7bfcc724881.

The vmadm payload

{
  "alias": "c6",
  "hostname": "c6",
  "brand": "bhyve",
  "resolvers": [
    "8.8.8.8",
    "8.8.4.4"
  ],
  "ram": "1024",
  "vcpus": "2",
  "nics": [
    {
      "nic_tag": "admin",
      "ip": "10.88.88.215",
      "netmask": "255.255.255.0",
      "gateway": "10.88.88.2",
      "model": "virtio",
      "primary": true
    },
    {
      "nic_tag": "admin",
      "ip": "addrconf",
      "model": "virtio"
    }
  ],
  "disks": [
    {
      "image_uuid": "11891f44-a374-11e8-9c35-c7bfcc724881",
      "boot": true,
      "model": "virtio"
    }
  ],
  "customer_metadata": {
    "root_authorized_keys": "ssh-rsa AAAAB..."
  }
}

Create the source instance

[root@buglets ~]# vmadm create -f c6.json
Successfully created VM ac685e55-5b81-6f19-bb31-a6b87dae44a8

Upgrade cloud-init in the source instance

First, log in. Duh.

[root@buglets ~]# ssh root@10.88.88.215
The authenticity of host '10.88.88.215 (10.88.88.215)' can't be established.
RSA key fingerprint is SHA256:2+jIDO1fdh396FYg72MFvZXnYjnbUYdEz5f3dBCrduA.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.88.88.215' (RSA) to the list of known hosts.
   __        .                   .
 _|  |_      | .-. .  . .-. :--. |-
|_    _|     ;|   ||  |(.-' |  | |
  |__|   `--'  `-' `;-| `-' '  ' `-'
                   /  ;  Instance (CentOS 6.9 20180819)
                   `-'   https://docs.joyent.com/images/linux/centos

Remove cloud-init and cloud-init-joyent. The latter was needed to prevent some future update of centos cloud-init packages from "upgrading" to a version of cloud-init that does not have recent SmartOS fixes.

[root@c6 ~]# rpm -e cloud-init cloud-init-joyent

Install cloud-init 18.4. This was built using these instructions from the cloud-init 18.4 tag in the official repo.

[root@c6 ~]# yum install -y https://us-east.manta.joyent.com/mgerdts/public/cloud-init/centos-6.9/cloud-init-18.4-1.el6.noarch.rpm
Loaded plugins: fastestmirror
Setting up Install Process
cloud-init-18.4-1.el6.noarch.rpm                                                             | 920 kB     00:12
Examining /var/tmp/yum-root-8PDf5M/cloud-init-18.4-1.el6.noarch.rpm: cloud-init-18.4-1.el6.noarch
Marking /var/tmp/yum-root-8PDf5M/cloud-init-18.4-1.el6.noarch.rpm to be installed
Determining fastest mirrors
epel/metalink                                                                                |  13 kB     00:00
 * base: repo1.dal.innoscale.net
 * epel: mirror.math.princeton.edu
 * extras: repo1.dal.innoscale.net
 * updates: mirrors.xtom.com
base                                                                                         | 3.7 kB     00:00
base/primary_db                                                                              | 4.7 MB     00:14
epel                                                                                         | 3.2 kB     00:00
epel/primary                                                                                 | 3.2 MB     00:09
epel                                                                                                    12497/12497
extras                                                                                       | 3.4 kB     00:00
extras/primary_db                                                                            |  26 kB     00:00
updates                                                                                      | 3.4 kB     00:00
updates/primary_db                                                                           | 1.9 MB     00:01
Resolving Dependencies
--> Running transaction check
---> Package cloud-init.noarch 0:18.4-1.el6 will be installed
--> Processing Dependency: python-devel for package: cloud-init-18.4-1.el6.noarch
--> Processing Dependency: python-jinja2 for package: cloud-init-18.4-1.el6.noarch
--> Processing Dependency: python-jsonschema for package: cloud-init-18.4-1.el6.noarch
--> Processing Dependency: python-oauthlib for package: cloud-init-18.4-1.el6.noarch
--> Running transaction check
---> Package python-devel.x86_64 0:2.6.6-66.el6_8 will be installed
---> Package python-jinja2.x86_64 0:2.2.1-3.el6 will be installed
--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.2.1-3.el6.x86_64
---> Package python-jsonschema.noarch 0:2.3.0-1.el6 will be installed
---> Package python-oauthlib.noarch 0:0.6.0-4.el6 will be installed
--> Processing Dependency: python-crypto2.6 for package: python-oauthlib-0.6.0-4.el6.noarch
--> Running transaction check
---> Package python-babel.noarch 0:0.9.4-5.1.el6 will be installed
---> Package python-crypto2.6.x86_64 0:2.6.1-2.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================
 Package                    Arch            Version                    Repository                              Size
====================================================================================================================
Installing:
 cloud-init                 noarch          18.4-1.el6                 /cloud-init-18.4-1.el6.noarch          3.0 M
Installing for dependencies:
 python-babel               noarch          0.9.4-5.1.el6              base                                   1.4 M
 python-crypto2.6           x86_64          2.6.1-2.el6                epel                                   513 k
 python-devel               x86_64          2.6.6-66.el6_8             base                                   173 k
 python-jinja2              x86_64          2.2.1-3.el6                base                                   466 k
 python-jsonschema          noarch          2.3.0-1.el6                epel                                    62 k
 python-oauthlib            noarch          0.6.0-4.el6                epel                                   119 k

Transaction Summary
====================================================================================================================
Install       7 Package(s)

Total size: 5.7 M
Total download size: 2.7 M
Installed size: 14 M
Downloading Packages:
(1/6): python-babel-0.9.4-5.1.el6.noarch.rpm                                                 | 1.4 MB     00:01
(2/6): python-crypto2.6-2.6.1-2.el6.x86_64.rpm                                               | 513 kB     00:00
(3/6): python-devel-2.6.6-66.el6_8.x86_64.rpm                                                | 173 kB     00:00
(4/6): python-jinja2-2.2.1-3.el6.x86_64.rpm                                                  | 466 kB     00:00
(5/6): python-jsonschema-2.3.0-1.el6.noarch.rpm                                              |  62 kB     00:00
(6/6): python-oauthlib-0.6.0-4.el6.noarch.rpm                                                | 119 kB     00:00
--------------------------------------------------------------------------------------------------------------------
Total                                                                               756 kB/s | 2.7 MB     00:03
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : python-crypto2.6-2.6.1-2.el6.x86_64                                                              1/7
  Installing : python-oauthlib-0.6.0-4.el6.noarch                                                               2/7
  Installing : python-jsonschema-2.3.0-1.el6.noarch                                                             3/7
  Installing : python-babel-0.9.4-5.1.el6.noarch                                                                4/7
  Installing : python-jinja2-2.2.1-3.el6.x86_64                                                                 5/7
  Installing : python-devel-2.6.6-66.el6_8.x86_64                                                               6/7
  Installing : cloud-init-18.4-1.el6.noarch                                                                     7/7
  Verifying  : python-jinja2-2.2.1-3.el6.x86_64                                                                 1/7
  Verifying  : python-devel-2.6.6-66.el6_8.x86_64                                                               2/7
  Verifying  : python-babel-0.9.4-5.1.el6.noarch                                                                3/7
  Verifying  : python-oauthlib-0.6.0-4.el6.noarch                                                               4/7
  Verifying  : python-jsonschema-2.3.0-1.el6.noarch                                                             5/7
  Verifying  : python-crypto2.6-2.6.1-2.el6.x86_64                                                              6/7
  Verifying  : cloud-init-18.4-1.el6.noarch                                                                     7/7

Installed:
  cloud-init.noarch 0:18.4-1.el6

Dependency Installed:
  python-babel.noarch 0:0.9.4-5.1.el6  python-crypto2.6.x86_64 0:2.6.1-2.el6   python-devel.x86_64 0:2.6.6-66.el6_8
  python-jinja2.x86_64 0:2.2.1-3.el6   python-jsonschema.noarch 0:2.3.0-1.el6  python-oauthlib.noarch 0:0.6.0-4.el6

Complete!

Clean up some stale stuff from the old cloud-init installation.

[root@c6 ~]# rm -f /etc/udev/rules.d/70-persistent-net.rules
[root@c6 ~]# cloud-init clean -ls

All done in the source image. Shut it down head back to the global zone.

[root@c6 ~]# poweroff

Broadcast message from root@c6
	(/dev/pts/0) at 14:15 ...

The system is going down for power off NOW!
[root@c6 ~]# Connection to 10.88.88.215 closed by remote host.
Connection to 10.88.88.215 closed.

If creating from scratch...

If instead of creating a fresh image instead of deriving one from a Joyent image, you will also need to install pyserial.

To prevent cloud-init from trying a bunch of data sources that will take forever and a day to time out, stick the following into /etc/cloud/cloud.cfg.d/90_smartos.cfg. Only the first line is strictly necessary.

datasource_list: [ SmartOS ]
# Preserve traditional root@<ip> login that was possible with rc.local
disable_root: false
# Do not create the centos/ubuntu/debian user
users: [ ]

mounts:
- [ vdb, /data, auto, "defaults,nofail" ]

Create new image

This will use imgadm to create a new incremental image.

A workaround

If imgadm create below fails creating a snapshot, you will need to work around the lack of a fix for OS-7343. That fix should be present in the 20181108 SmartOS release.

[root@buglets ~]# zfs set quota=none zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8

Create the image

The UUID must match the UUID of the instance that was prepared above.

[root@buglets ~]# imgadm create -i ac685e55-5b81-6f19-bb31-a6b87dae44a8 name=centos6 version=test1
Inheriting from origin image 11891f44-a374-11e8-9c35-c7bfcc724881 (centos-6 20180819)
Manifest:
    {
      "v": 2,
      "uuid": "2e21ea4e-9854-41cb-82b0-6883b5dc7383",
      "name": "centos6",
      "version": "test1",
      "type": "zvol",
      "os": "linux",
      "requirements": {
        "networks": [
          {
            "name": "net0",
            "description": "public"
          }
        ],
        "ssh_key": true,
        "brand": "bhyve"
      },
      "users": [
        {
          "name": "root"
        }
      ],
      "nic_driver": "virtio",
      "disk_driver": "virtio",
      "cpu_type": "host",
      "image_size": 10240,
      "origin": "11891f44-a374-11e8-9c35-c7bfcc724881"
    }
Moving existing @final snapshot out of the way to "zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8/disk0@final-1541600336135"
Snapshotting to "zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8/disk0@final"
Sending image file to "centos6-test1.zvol"
Saving manifest to "centos6-test1.imgmanifest"

Install the new image

[root@buglets ~]# imgadm install -f centos6-test1.zvol -m centos6-test1.imgmanifest
Installing image 2e21ea4e-9854-41cb-82b0-6883b5dc7383 (centos6@test1)
...854-41cb-82b0-6883b5dc7383 [==================================================>] 100% 103.46MB  46.29MB/s     2s
Installed image 2e21ea4e-9854-41cb-82b0-6883b5dc7383 (centos6@test1)

Test the new image

The test instance vmadm payload

Notice that this specifies an IPv6 address.

{
  "alias": "c6test1",
  "hostname": "c6test1",
  "brand": "bhyve",
  "resolvers": [
    "8.8.8.8",
    "8.8.4.4"
  ],
  "ram": "1024",
  "vcpus": "2",
  "nics": [
    {
      "nic_tag": "admin",
      "ips": ["10.88.88.217/24", "fd12:3456:789a:1::7/64"],
      "gateway": "10.88.88.2",
      "model": "virtio",
      "primary": true
    },
    {
      "nic_tag": "admin",
      "ip": "addrconf",
      "model": "virtio"
    }
  ],
  "disks": [
    {
      "image_uuid": "2e21ea4e-9854-41cb-82b0-6883b5dc7383",
      "boot": true,
      "model": "virtio"
    }
  ],
  "customer_metadata": {
    "root_authorized_keys": "ssh-rsa AAAAB..."
  }
}

Create the instance

[root@buglets ~]# vmadm create -f c6-test1.json
zSuccessfully created VM 8aed29de-d993-cd53-8ea1-80a76f82398d

On the console we see:

Starting cloud-init: Cloud-init v. 18.4-1.el6 running 'init' at Wed, 07 Nov 2018 14:23:38 +0000. Up 14.64 seconds.
ci-info: +++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
ci-info: | Device |  Up  |           Address            |      Mask     | Scope  |     Hw-Address    |
ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
ci-info: |   lo   | True |          127.0.0.1           |   255.0.0.0   |  host  |         .         |
ci-info: |   lo   | True |          127.0.0.1           | 255.255.255.0 |  host  |         .         |
ci-info: |   lo   | True |           ::1/128            |       .       |  host  |         .         |
ci-info: |  net0  | True |         10.88.88.217         | 255.255.255.0 | global | b2:65:e1:4a:27:63 |
ci-info: |  net0  | True |    fd12:3456:789a:1::7/64    |       .       | global | b2:65:e1:4a:27:63 |
ci-info: |  net0  | True | fe80::b065:e1ff:fe4a:2763/64 |       .       |  link  | b2:65:e1:4a:27:63 |
ci-info: |  net1  | True | fe80::c0fb:70ff:fedf:e6e2/64 |       .       |  link  | c2:fb:70:df:e6:e2 |
ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
ci-info: ++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++
ci-info: +-------+-------------+------------+---------------+-----------+-------+
ci-info: | Route | Destination |  Gateway   |    Genmask    | Interface | Flags |
ci-info: +-------+-------------+------------+---------------+-----------+-------+
ci-info: |   0   |  10.88.88.0 |  0.0.0.0   | 255.255.255.0 |    net0   |   U   |
ci-info: |   1   | 169.254.0.0 |  0.0.0.0   |  255.255.0.0  |    net0   |   U   |
ci-info: |   2   |   0.0.0.0   | 10.88.88.2 |    0.0.0.0    |    net0   |   UG  |
ci-info: +-------+-------------+------------+---------------+-----------+-------+
ci-info: ++++++++++++++++++++++++Route IPv6 info++++++++++++++++++++++++
ci-info: +-------+-----------------------+---------+-----------+-------+
ci-info: | Route |      Destination      | Gateway | Interface | Flags |
ci-info: +-------+-----------------------+---------+-----------+-------+
ci-info: |   9   | fd12:3456:789a:1::/64 |    ::   |    net0   |   U   |
ci-info: |   10  |       fe80::/64       |    ::   |    net0   |   U   |
ci-info: |   11  |       fe80::/64       |    ::   |    net1   |   U   |
ci-info: |   15  |        ff00::/8       |    ::   |    net0   |   U   |
ci-info: |   16  |        ff00::/8       |    ::   |    net1   |   U   |
ci-info: +-------+-----------------------+---------+-----------+-------+
[root@c6test1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 127.0.0.1/24 brd 127.0.0.255 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: net0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether b2:65:e1:4a:27:63 brd ff:ff:ff:ff:ff:ff
    inet 10.88.88.217/24 brd 10.88.88.255 scope global net0
    inet6 fd12:3456:789a:1::7/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::b065:e1ff:fe4a:2763/64 scope link
       valid_lft forever preferred_lft forever
3: net1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether c2:fb:70:df:e6:e2 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::c0fb:70ff:fedf:e6e2/64 scope link
       valid_lft forever preferred_lft forever

Oops, I forgot

I mean to use the linux-prepare-image script. Had I remembered, I would have uttered:

# curl --insecure -O https://raw.githubusercontent.com/joyent/sdc-imgapi/master/tools/prepare-image/linux-prepare-image
# imgadm create -s linux-prepare-image -i $uuid name=$name version=$version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment