Unprivileged LCX containers on Gentoo

UnprivilegedContainersOnGentoo.md

Getting unprivileged LCX containers to work on Gentoo

So many references on the Internet describe how to setup unprivileged containers on Ubuntu, but I've found that it is astonishingly cryptic to get them to work on other Linux distros without knowing about LXC internals. In my case, my distro of choice is Gentoo, and here's how I managed to run my first unprivileged container. (Big thanks to Lord on #gentoofr for spending just a few minutes to find the reference I needed to get it working; it helped tremendously.)

The Gentoo Wiki helped, but was incomplete for some of the crucial steps, namely the following two requirements:

• app-admin/cgmanager-0.37 or newer is necessary. It's not currently present in portage as of now (June 2nd 2015), so I had to use a local overlay to install it in a clean way. See below for a quick how-to.
• A cgroup hierarchy is needed, but no online guide mentions how to create one. That's because on Ubuntu systemd is responsible for automatically managing it. Gentoo doesn't run systemd, but rather openrc for system initialization. Again, see below for details.

Basics

Follow Stephane's excellent and detailed guide to setup the basic requirements for unprivileged containers. In two words, the following are needed:

• a recent kernel
• subuid and subgid ranges in /etc/subuid and /etc/subgid (use a recent version of usermod, from the shadow package)
• lxc-checkconfig to indicate all features to be enabled
• /etc/lxc/default.conf to contain your general networking setup
• /etc/lxc/lxc-usernet to contain user limits on net devices creation
• ~/.config/lxc/default.conf to contain lxc.id_map = u 0 100000 65536 and lxc.id_map = g 0 100000 65536
• ~/.local/share/lxc to exist and be traversable (+x) all along

Gentoo requirements

For Gentoo specific items, you need:

• >=sys-apps/shadow-4.2 (keyworded on amd64 at the moment, but that's ok)
• >=sys-auth/pambase-20150213 (stable)
• app-emulation/lxc[cgmanager] (set the cgmanager USE flag for lxc)
• >=app-admin/cgmanager-0.37 (version 0.36 is buggy, will not start/stop correctly, etc.)

cgmanager-0.37 can be added to a local overlay easily.

mkdir -p /usr/local/portage/app-admin/cgmanager
cd /usr/local/portage/app-admin/cgmanager
cp -r /usr/portage/app-admin/cgmanager/{cgmanager*.ebuild,files} .
mv cgmanager*.ebuild cgmanager-0.37.ebuild
ebuild cgmanager-0.37.ebuild digest
echo "PORTDIR_OVERLAY='$(pwd)'" >> /etc/portage/make.conf
emerge -va =cgmanager-0.37

The program cgmanager needs to be running when lxc-start is invoked. Add it to the startup scripts if needed: rc-update add cgmanager default. (You do not need to add/run cgproxy which comes with cgmanager.) For debugging purposes, you can run it manually with sudo cgmanager --debug.

Creating cgroup hierarchy manually

The cgroup hierarchy must be manually created if systemd is not installed and running on the system (most Gentoo systems, as it uses OpenRC). It is the critical part of the process or otherwise cgmanager will fail to move process IDs to the correct cgroup on behalf of the unprivileged user.

The web site http://chbrauner.blogspot.fr/2014/12/extended-introduction-to-unprivileged.html?m=1, section Creating cgroups > With cgmanager, is the only online reference that I've come across that tells how to do it. (I've later learned that the Gentoo Wiki page for LXC references that page at the very bottom in a P.S. note.)

quad@omexis ~ $ sudo cgm create all $USER
quad@omexis ~ $ sudo cgm chown all $USER $(id -u) $(id -g)
quad@omexis ~ $ cgm movepid all $USER $$

After completing the above commands, cgroups will be adjusted accordingly for the user's current process:

quad@omexis ~ $ cat /proc/self/cgroup
15:name=systemd:/quad
11:net_prio:/quad
10:perf_event:/quad
9:blkio:/quad
8:net_cls:/quad
7:freezer:/quad
6:devices:/quad
5:memory:/quad
4:cpuacct:/quad
3:cpu:/quad
2:cpuset:/quad
1:name=openrc:/quad

Normally, on Gentoo, the contents of /proc/self/cgroup looks like this (when cgmanager -m name=systemd is running).

13:name=systemd:/
11:net_prio:/
10:perf_event:/
9:blkio:/
8:net_cls:/
7:freezer:/
6:devices:/
5:memory:/
4:cpuacct:/
3:cpu:/
2:cpuset:/
1:name=openrc:/

If everything went well until now, you should now be able to run unprivileged containers.

quad@omexis ~ $ lxc-start -n ub1
quad@omexis ~ $ lxc-ls -f
NAME  STATE    IPV4  IPV6  GROUPS  AUTOSTART  
--------------------------------------------
ub1   RUNNING  -     -     -       NO         
quad@omexis ~ $ lxc-info -n ub1
Name:           ub1
State:          RUNNING
PID:            24229
CPU use:        1.61 seconds
Memory use:     3.97 MiB
KMem use:       0 bytes
quad@omexis ~ $ lxc-attach -n ub1
root@ub1:/# id
uid=0(root) gid=0(root) groups=0(root)
root@ub1:/# exit
quad@omexis ~ $ lxc-ls -f
NAME  STATE    IPV4  IPV6  GROUPS  AUTOSTART
--------------------------------------------
ub1   RUNNING  -     -     -       NO
quad@omexis ~ $ ps -fu "$(echo {100000..100200})"
UID        PID  PPID  C STIME TTY          TIME CMD
100000   24229 24209  0 03:08 ?        00:00:00 /sbin/init
100000   24555 24229  0 03:08 ?        00:00:00 upstart-udev-bridge --daemon
100000   24604 24229  0 03:08 ?        00:00:00 /lib/systemd/systemd-udevd --daemon
100102   24734 24229  0 03:08 ?        00:00:00 rsyslogd
100000   24765 24229  0 03:08 ?        00:00:00 upstart-file-bridge --daemon
100000   24766 24229  0 03:08 ?        00:00:00 upstart-socket-bridge --daemon
100000   29288 24229  0 03:10 pts/3    00:00:00 /sbin/getty -8 38400 tty4
100000   29290 24229  0 03:10 pts/1    00:00:00 /sbin/getty -8 38400 tty2
100000   29291 24229  0 03:10 pts/2    00:00:00 /sbin/getty -8 38400 tty3
100000   29300 24229  0 03:10 ?        00:00:00 cron
100000   29334 24229  0 03:10 pts/4    00:00:00 /sbin/getty -8 38400 console
100000   29336 24229  0 03:10 pts/0    00:00:00 /sbin/getty -8 38400 tty1
quad@omexis ~ $