Build v15.0 beta Core (on v14.2 TKLDev)

tkldev-build-for-stretch-on-jessie.md

Please note that this process will add a third party TurnKey repo that I personally manage. I had to do that as I do not have direct access to the offical TurnKey repos and we need updated packages for a Stetch base.

Starting point: v14.2 TKLDev running in a VM (or bare metal if you prefer...), logged into terminal as root.

---

# checkout appropriate branch of common and core
cd common
git fetch origin # assumes you stil have turnkey as origin - can be skipped on fresh v14.2 install
git checkout -t origin/stretch-dev
cd core
git fetch origin # assumes you stil have turnkey as origin - can be skipped on fresh v14.2 install
git checkout -t origin/stretch-dev

# build new ISO
export RELEASE=debian/stretch
export DEV_TEST=y
make clean
make

@JedMeister, apparently you also need to install the bootstraps for stretch. Followed your instructions and got

# make
deck /turnkey/fab/bootstraps/stretch build/bootstrap
fatal: source `/turnkey/fab/bootstraps/stretch' is not a directory
/usr/share/fab/product.mk:526: recipe for target 'build/stamps/bootstrap' failed
make: *** [build/stamps/bootstrap] Error 1

Since I couldn't find a TurnKey bootstrap for stretch, I took a chance and just renamed the jessie bootstrap. I figured that all it needed was a bootable system and that it would automatically be upgraded to stretch during the build. It seems to have worked as the core build completed.

@JedMeister,
I've been working on creating a 'bootstrapped' TKLdev version 15.0 running in a container. I've had pretty good success so far, and have been able to build core, ansible, and lxc appliances. When it comes to building TKLdev itself, I'm blocked because there are not yet versions of 'deck' or 'pool' for stretch in the repository. I'm working on documenting the procedure, but I didn't want to post it without your approval because it would potentially increase use of your private repository, http://dev-apt.jeremydavis.org/. If users follow my recommendation to install apt-cacher-ng, the extra load shouldn't be great, but I wanted to ask anyway.
In the process, I created a new set of bootstraps for stretch. I'm posting that part here in the hope it will save you some time. The package list will need to be reviewed and approved by Alon or Liraz and the tarball signed.

---

Create Bootstraps for Debian Stretch

Bootstraps for Debian Stretch have not yet been provided upstream by TurnKey.
This documents the procedure I used to create my own set of bootstraps for Stretch.

On the tkldev v.15.0 container appliance, do the following:

cd bootstraps
debootstrap --arch=amd64 stable stretch

When debootstrap is complete, we get a list of the package selections to compare with the TurnKey selections from the Jessie bootstrap.

fab-chroot jessie -- dpkg --get-selections > turnkey-selections.list
fab-chroot stretch -- dpkg --get-selections > debootstrap-selections.list
diff turnkey-selections.list debootstrap-selections.list

Carefully compare the two selection lists. Basically we want to start with the turnkey-selections.list from the Jessie bootstraps and modify it, updating package versions where the name changed e.g. libprocps3:amd64 --> libprocps6:amd64. Next we need to remove SysInitV and replace with SystemD.

< startpar					install
< sysv-rc					install
---
> systemd					install
> systemd-sysv				install

Finally, we need to add the remaining files from debootstrap-selections.list changing the 'install' to 'purge' so they will be removed. I found it very helpful to use the Linux application, meld, to provide a side-by-side visual comparison of the two files. The results of some trial and error are shown below. You must make sure that packages initramfs-tools and insserv are not removed. The former required the addition of initramfs-tools-core to satisfy a dependency.

Before we can upgrade the packages, we must update dpkg's available list.

fab-chroot stretch -- apt-cache dumpavail > avail
fab-chroot stretch -- dpkg --merge-avail < avail
rm -f avail

Next we clear dpkg's existing selections and replace them with our new stretch-selections.list. Finally we run a dselect-upgrade to purge the unwanted packages.

fab-chroot stretch -- dpkg --clear-selections
fab-chroot stretch -- dpkg --set-selections < stretch-selections.list
fab-chroot stretch -- apt-get dselect-upgrade

After the upgrade completes, pull a dpkg list and make sure all the necessary are installed, and that all packages have status 'ii'. If any packages have a different status, then your stretch-selections.list needs adjustment. It took me several tries to get a clean installation.

fab-chroot stretch -- dpkg --list

Next, clean up any '-old' files left behind, clear dpkg's available list, and clean apt's cache.

find stretch/ -name '*-old' -delete
fab-chroot stretch -- dpkg --clear-avail
fab-chroot stretch -- apt-get clean

Finally, complete the process by creating a tar backup of the completed stretch bootstrap.

tar czvf bootstrap-stretch-amd64.tar.gz stretch/

Test by making sure that running a make build of the core appliance completes successfully.

cd core
make

---

Appendix

./stretch-selections.list

adduser                         install
apt                             install
apt-utils                       install
base-files                      install
base-passwd                     install
bash                            install
bsdmainutils                    purge
bsdutils                        install
coreutils                       install
cpio                            install
cron                            purge
dash                            install
debconf                         install
debconf-i18n                    purge
debian-archive-keyring          install
debianutils                     install
dialog                          install
diffutils                       install
dmidecode                       purge
dmsetup                         install
dpkg                            install
e2fslibs:amd64                  install
e2fsprogs                       install
findutils                       install
gcc-6-base:amd64                install
gnupg                           purge
gnupg-agent                     purge
gpgv                            install
grep                            install
gzip                            install
hostname                        install
ifupdown                        install
init                            install
init-system-helpers             install
initramfs-tools                 install
initramfs-tools-core            install
insserv                         install
iproute2                        install
iptables                        purge
iputils-ping                    purge
isc-dhcp-client                 purge
isc-dhcp-common                 purge
kmod                            install
libacl1:amd64                   install
libapparmor1:amd64              install
libapt-inst2.0:amd64            install
libapt-pkg5.0:amd64             install
libassuan0:amd64                purge
libattr1:amd64                  install
libaudit-common                 install
libaudit1:amd64                 install
libblkid1:amd64                 install
libbsd0:amd64                   install
libbz2-1.0:amd64                install
libc-bin                        install
libc-l10n                       install
libc6:amd64                     install
libcap-ng0:amd64                install
libcap2:amd64                   install
libcomerr2:amd64                install
libcryptsetup4:amd64            install
libdb5.3:amd64                  install
libdebconfclient0:amd64         install
libdevmapper1.02.1:amd64        install
libdns-export162                install
libedit2:amd64                  install
libelf1:amd64                   install
libestr0                        purge
libfastjson4:amd64              purge
libfdisk1:amd64                 install
libffi6:amd64                   purge
libgcc1:amd64                   install
libgcrypt20:amd64               install
libgdbm3:amd64                  purge
libgmp10:amd64                  purge
libgnutls30:amd64               purge
libgpg-error0:amd64             install
libgssapi-krb5-2:amd64          install
libhogweed4:amd64               purge
libidn11:amd64                  install
libidn2-0:amd64                 purge
libip4tc0:amd64                 install
libip6tc0:amd64                 install
libiptc0:amd64                  purge
libisc-export160                install
libk5crypto3:amd64              install
libklibc                        install
libkeyutils1:amd64              install
libkmod2:amd64                  install
libkrb5-3:amd64                 install
libkrb5support0:amd64           install
libksba8:amd64                  purge
liblocale-gettext-perl          purge
liblogging-stdlog0:amd64        purge
liblognorm5:amd64               purge
liblz4-1:amd64                  install
liblzma5:amd64                  install
libmnl0:amd64                   install
libmount1:amd64                 install
libncurses5:amd64               install
libncursesw5:amd64              install
libnetfilter-conntrack3:amd64   purge
libnettle6:amd64                purge
libnewt0.52:amd64               purge
libnfnetlink0:amd64             purge
libnpth0:amd64                  purge
libp11-kit0:amd64               purge
libpam-modules-bin              install
libpam-modules:amd64            install
libpam-runtime                  install
libpam0g:amd64                  install
libpcre3:amd64                  install
libpipeline1:amd64              purge
libpopt0:amd64                  purge
libprocps6:amd64                install
libpsl5:amd64                   purge
libreadline7:amd64              purge
libseccomp2:amd64               install
libselinux1:amd64               install
libsemanage-common              install
libsemanage1:amd64              install
libsepol1:amd64                 install
libslang2:amd64                 purge
libsmartcols1:amd64             install
libsqlite3-0:amd64              purge
libss2:amd64                    install
libssl1.0.2:amd64               install
libssl1.1:amd64                 purge
libstdc++6:amd64                install
libsystemd0:amd64               install
libtasn1-6:amd64                purge
libtext-charwidth-perl          purge
libtext-iconv-perl              purge
libtext-wrapi18n-perl           purge
libtinfo5:amd64                 install
libudev1:amd64                  install
libunistring0:amd64             purge
libustr-1.0-1:amd64             install
libuuid1:amd64                  install
libwrap0:amd64                  install
libxapian30:amd64               purge
libxtables12:amd64              purge
linux-base                      install
locales                         install
login                           install
logrotate                       purge
lsb-base                        install
mawk                            install
mount                           install
multiarch-support               install
nano                            purge
ncurses-base                    install
ncurses-bin                     install
net-tools                       install
netbase                         install
openssh-client                  purge
openssh-server                  purge
openssh-sftp-server             purge
passwd                          install
perl-base                       install
pinentry-curses                 purge
procps                          install
readline-common                 purge
rsyslog                         purge
sed                             install
sensible-utils                  install
systemd                         install
systemd-sysv                    install
sysvinit-utils                  install
tar                             install
tasksel                         purge
tasksel-data                    purge
tzdata                          install
ucf                             install
udev                            install
util-linux                      install
vim-common                      purge
vim-tiny                        purge
wget                            purge
whiptail                        purge
xxd                             purge
zlib1g:amd64                    install