A reference for running containers with non-root permissions

containers-and-permissions.md

When implementing development environments inside a container, usually I want to:

1. Run as non-root inside the container (many tools do not like being run as root).
2. Behave as the non-root user executing the container when reading/writing from mounted volumes

There are some additional (small) security benefits from running as non-root inside a container.

The solution to these problems is complicated by the way that different container runtimes behave:

keep-id

Across the different container runtimes, there is often a flag such as userns=keep-id that maps the UID:GID of the host to the container. Linux creates users by default with UID 1000, so default non-root users will often map directly to host UID.

Docker

Mostly summarised here.

1. Effectively, docker run --user and dockerd --userns-remap are orthogonal (one determines inside container, the other outside).
   • Setting --user=UID:GID means that user runs as non-root inside container
     • Overrides USER directive
     • Can't create HOME or other directories, need to mount them as volumes
   • Using user namespaces requires restarting Docker daemon
2. When container runs as ROOT, it has ROOT privileges on the host (besides the sandboxing), so it can delete root-owned host files when mounted.
3. Building in the UID and GID means users need to recompile image
   • Not able to read user-level mounts if GID and UID don't match
4. Mounted volumes have the same UID and GID as the host.

Singularity

Containers build as root, and execute as the host user. It is not possible to run as a non-user without invoking su inside the container. There is a fakeroot feature that maps the host UID to 0 in the container, but this doesn't permit unprivileged remapping unless one has access to the host /etc/subuid//etc/subgid files.

Podman

1. Similarly to docker, the --user flag is orthogonal to the UID remapping.
2. By default, the external UID maps to UID 0 inside the container, and so mounted volumes are owned by UID 0 irrespective of --user.

UID mapping

The --userns=keep-id flag maps the external UID and GID to the same values inside of the container. This flag also sets the UID of the user in the container.

However, this means that the container UID depends upon the host executing the container, i.e. it is not predictable. If one needs predictable user IDs, then a UID mapping must be used.

To achieve our original aims, one could

• Run as a container-defined USER with a hard-coded UID
• Establish a uidmap for the user

  # Simple explanation in three steps:
  # CONTAINER_UID = 4
  # host_id      = 0 1 2 3 4 5 6 7 8 9
  #                ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 
  # container_id = x[0 1 2 3]x x x x x  (step 1)
  # container_id =[4]x x x x x x x x x  (step 2)
  # container_id = x x x x x[5 6 7 8 9] (step 3)
  podman run ... \
  # Map the container 0->$CONTAINER_UID-1 onto the host, starting at an arbitrary host UID
  --uidmap="0:1:$CONTAINER_UID" \
  # Map the container $CONTAINER_UID to the host UID (0)
  --uidmap="$CONTAINER_UID:0:1" \
  # Map the remaining UIDs to the host, starting after the $CONTAINER_UID
  --uidmap=$(($CONTAINER_UID+1)):$(($CONTAINER_UID+1)):$((65536 - $CONTAINER_UID - 1)) \
  

• Create the $HOME directory and other configuration using useradd, and then set the user with USER user

Unlike --userns=keep-id, the UID mapping does not change the default user of the container.

So, if one wants a solution that will work correctly between Docker, podman, and Singularity, then one needs to use the keep-id on podman, and --user=UID:GID on Docker. On singularity, the default is correct.

Then, the HOME and other storage needs to be made available. On both Docker and podman, this can be done using volumes. On Singularity, this could be done using contain, or a host directory if it needs to be permanent.

Finally, any chown-pedantic software needs to be installed in the entrypoint.