Skip to content

Instantly share code, notes, and snippets.

@FreddieOliveira
Last active November 30, 2024 11:26
Show Gist options
  • Save FreddieOliveira/efe850df7ff3951cb62d74bd770dce27 to your computer and use it in GitHub Desktop.
Save FreddieOliveira/efe850df7ff3951cb62d74bd770dce27 to your computer and use it in GitHub Desktop.
This tutorial shows how to run docker natively on Android, without VMs and chroot.

Docker on Android 🐋📱

Edit 🎉

All packages, except for Tini have been added to termux-root. To install them, simply pkg install root-repo && pkg install docker. This will install the whole docker suite, left only Tini to be compiled manually.


Summary

  1. Intro
  2. Building
    1. Rooting
    2. Kernel
      1. General compiling instructions
      2. Modifications
      3. Patching
    3. Docker
      1. dockercli
      2. dockerd
      3. tini
      4. libnetwork
      5. containerd
      6. runc
  3. Running
    1. Caveats
      1. Internet access
      2. Shared volumes
    2. GUI
      1. X11 Forwarding
      2. VNC server within the container
    3. Steam (work in progress)
  4. Attachments
    1. Kernel patches
    2. docker-cli patches
    3. dockerd patches
    4. containerd patches
  5. Aknowledgements
  6. Final notes

1. Intro

This tutorial presents a step by step guide on how to run docker containers directly on Android. By directly I mean there's no VM involved nor chrooting inside a GNU/Linux rootfs. This is docker purely in Android. Yes, it is possible.

Bear in mind that you'll have to root your phone, mess with and compile your phone's kernel and docker suite. So, be prepared to get your hands dirty.

2. Building

2.1. Rooting

This step is pretty device specific, so there's no way to write a generic tutorial here. You'll need to google for instructions for your device and follow them.

Just be aware that you may lose your phone's warrant and all your data will be erased after unlocking the bootloader, so make a backup of your important stuff.

2.2. Kernel

2.2.1. General compiling instructions

Compiling the phone's kernel is also device specific, but some major tips may help you out.

First, google about instructions for your phone. Start by compiling the kernel without any modification. Flash it and hope for the best. If everything went well, then you can proceed to the modifications.

Note that flashing the kernel won't erase any data in your phone. The worst that can happen is you get stuck in a boot loop. In this case, you can flash a kernel that's known to be working or just flash a working ROM, since it contains a kernel with it. None of these operations erase any data in your phone.

2.2.2. Modifications

Now that you (hopefully) are able to compile the kernel, let's talk about what matters. Docker needs a lot of features that are disabled by default in Android's kernel.

To check the necessary features list, first install the Termux app in your phone. This is the terminal emulator that we're going to use throughout this guide. It has a package manager with many tools compiled for Android.

Next, open Termux and download a script to check your kernel:

$ pkg install wget
$ wget https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh
$ chmod +x check-config.sh
$ sed -i '1s_.*_#!/data/data/com.termux/files/usr/bin/bash_' check-config.sh
$ sudo ./check-config.sh

Now, in your computer, open the kernel's configuration menu. This menu is a modified version of dialog, a ncurses window menu, in which you can enable and disable the kernel features. To look for some item in particular, you can press the / key and type the item name and hit Enter. This will show the description and location of the item.

For now, we want to enable the Generally Necessary items, the Network Drivers items and some Optional Features. For the Storage Drivers we'll be using the overlay.

2.2.3. Patching

Before compiling the kernel there are two files that need to be patched.

kernel/Makefile

The first one is the kernel/Makefile. Although not strictly necessary to modify this file, it will help by making it possible to check if your kernel has all the necessary features docker needs.

If you do not apply this patch, the output of the check-config.sh script used above won't be reliable after recompiling the kernel.

Check the patch at the attachments section and modify your Makefile accordingly.

net/netfilter/xt_qtaguid.c

This second file needs to be patched because of a bug introduced by Google. After you run any container, a seg fault will be generated due to a null pointer dereference and your phone will freeze and reboot. If you work at Google or know someone who does, warn him/her about it.

Check the patch at the attachments section and modify your xt_qtaguid.c accordingly.


Now that everything is setup, compile and flash the kernel. If you applied the Makefile patch, you'll see this warning everytime your phone boots:

IMG_20210110_203818

Don't worry though, this is a harmless warning remembering you that you're using a modified kernel.

2.3. Docker

See Edit.

Once you have a supported kernel, it's time to compile the docker suite. It's a suite because it's not just one program, but rather a set of different programs that we'll need to compile separately. So hands on.

Firts, let's install the packages we're gonna use to build docker in Termux:

$ pkg install go make cmake ndk-multilib tsu

Now we're ready to start compiling things. Create a work directory where the packages will be downloaded and built:

$ mkdir $TMPDIR/docker-build
$ cd $TMPDIR/docker-build

Download all the patches files into there and let's begin. All commands for the differents packages that'll be compiled next is meant to be executed inside this folder.

2.3.1. dockercli

See Edit.

This is the docker client, which will talk to the docker daemon. This package will compile a binary named docker and all docker man pages. To build and install it:

$ cd $TMPDIR/docker-build
$ wget https://github.com/docker/cli/archive/v20.10.2.tar.gz -O cli-20.10.2.tar.gz
$ tar xf cli-20.10.2.tar.gz
$ mkdir -p src/github.com/docker
$ mv cli-20.10.2 src/github.com/docker/cli
$ export GOPATH=$(pwd)
$ export VERSION=v20.10.2-ce
$ export DISABLE_WARN_OUTSIDE_CONTAINER=1
$ cd src/github.com/docker/cli
$ xargs sed -i 's_/var/\(run/docker\.sock\)_/data/docker/\1_g' < <(grep -R /var/run/docker\.sock | cut -d':' -f1 | sort | uniq)
$ patch vendor/github.com/containerd/containerd/platforms/database.go ../../../../database.go.patch.txt
$ patch scripts/docs/generate-man.sh ../../../../generate-man.sh.patch.txt
$ patch man/md2man-all.sh ../../../../md2man-all.sh.patch.txt
$ patch cli/config/config.go ../../../../config.go.patch.txt
$ make dynbinary
$ make manpages
$ install -Dm 0700 build/docker-android-* $PREFIX/bin/docker
$ install -Dm 600 -t $PREFIX/share/man/man1 man/man1/*
$ install -Dm 600 -t $PREFIX/share/man/man5 man/man5/*
$ install -Dm 600 -t $PREFIX/share/man/man8 man/man8/*

2.3.2. dockerd

See Edit.

The docker daemon is the most problematic binary that's gonna be compiled. It needs so many patches that's easier to modify the code in a batch with sed. Despite the need of modifying a lot of files, the modifications by themselfs are rather simple:

  1. Substitute every occurrence of runtime.GOOS by the string "linux";
  2. Remove unneeded imports of the runtime lib.

By doing that, we are in essence spoofing our operating system as a Linux one: everytime the code would do the runtime.GOOS == "linux" comparison (which would become "android" == "linux", and thus false) it will now do "linux" == "linux" and thus true.

To make the substitution across every file, we'll run a sed command. After that, some files will now give the extremely annoying unturnable-off go lang "feature" imported and not used error, because the only function these files were using from the runtime package was the runtime.GOOS. So, to fix it we'll use an horrible but simple solution: we'll keep trying to compile the code and at each failed attempt we'll fix the reported files till we get it to compile successfully.

$ cd $TMPDIR/docker-build
$ wget https://github.com/moby/moby/archive/v20.10.2.tar.gz -O moby-20.10.2.tar.gz
$ tar xf moby-20.10.2.tar.gz
$ cd moby-20.10.2
$ export DOCKER_GITCOMMIT=8891c58a43
$ export DOCKER_BUILDTAGS='exclude_graphdriver_btrfs exclude_graphdriver_devicemapper exclude_graphdriver_quota selinux exclude_graphdriver_aufs'
$ patch cmd/dockerd/daemon.go ../daemon.go.patch
$ xargs sed -i "s_\(/etc/docker\)_$PREFIX\1_g" < <(grep -R /etc/docker | cut -d':' -f1 | sort | uniq)
$ xargs sed -i 's_\(/run/docker/plugins\)_/data/docker\1_g' < <(grep -R '/run/docker/plugins' | cut -d':' -f1 | sort | uniq)
$ xargs sed -i 's/[a-zA-Z0-9]*\.GOOS/"linux"/g' < <(grep -R '[a-zA-Z0-9]*\.GOOS' | cut -d':' -f1 | sort | uniq)
$ (while ! IFS='' files=$(AUTO_GOPATH=1 PREFIX='' hack/make.sh dynbinary 2>&1 1>/dev/null); do if ! xargs sed -i 's/\("runtime"\)/_ \1/' < <(echo $files | grep runtime | cut -d':' -f1 | cut -c38-); then echo $files; exit 1; fi; done)
$ install -Dm 0700 bundles/dynbinary-daemon/dockerd $PREFIX/bin/dockerd-dev

A binary called dockerd-dev was compiled and installed, but in order to it run correctly, the cgroups need to be mounted. Since Android mounts the cgroups in a non standard location we need to fix this. To do so, a script named dockerd will be created that will mount crgoups in the correct path if needed and call dockerd-dev next.

$ cat << "EOF" > $PREFIX/bin/dockerd
#!/data/data/com.termux/files/usr/bin/bash

export PATH="${PATH}:/system/xbin:/system/bin"
opts='rw,nosuid,nodev,noexec,relatime'
cgroups='blkio cpu cpuacct cpuset devices freezer memory pids schedtune'

# try to mount cgroup root dir and exit in case of failure
if ! mountpoint -q /sys/fs/cgroup 2>/dev/null; then
  mkdir -p /sys/fs/cgroup
  mount -t tmpfs -o "${opts}" cgroup_root /sys/fs/cgroup || exit
fi

# try to mount cgroup2
if ! mountpoint -q /sys/fs/cgroup/cg2_bpf 2>/dev/null; then
  mkdir -p /sys/fs/cgroup/cg2_bpf
  mount -t cgroup2 -o "${opts}" cgroup2_root /sys/fs/cgroup/cg2_bpf
fi

# try to mount differents cgroups
for cg in ${cgroups}; do
  if ! mountpoint -q "/sys/fs/cgroup/${cg}" 2>/dev/null; then
    mkdir -p "/sys/fs/cgroup/${cg}"
    mount -t cgroup -o "${opts},${cg}" "${cg}" "/sys/fs/cgroup/${cg}" \
    || rmdir "/sys/fs/cgroup/${cg}"
  fi
done

# start the docker daemon
$PREFIX/bin/dockerd-dev $@
EOF

Make the script executable:

$ chmod +x $PREFIX/bin/dockerd

And finally configure some dockerd options:

$ mkdir -p $PREFIX/etc/docker
$ cat << "EOF" > $PREFIX/etc/docker/daemon.json
{
    "data-root": "/data/docker/lib/docker",
    "exec-root": "/data/docker/run/docker",
    "pidfile": "/data/docker/run/docker.pid",
    "hosts": [
        "unix:///data/docker/run/docker.sock"
    ],
    "storage-driver": "overlay2"
}
EOF

Warning: dockerd will store all its files, like containers, images, volumes, etc inside the /data/docker folder, which means you'll lose everything if you format the phone (flash a ROM). This folder was chosen instead of storing things inside Termux installation folder, because dockerd fails when setting up the overlay storage driver there. It seems Android mounts the /data/data folder with some options that prevent overlayfs to work, or the filesystem doesn't support it.

2.3.3. tini

tini is an optional dependency of dockerd in case you want the init process to be the first process of the container being ran (for this use the --init flag when creating a container). Having init as the parent of all other proccess ensures that a proper clean up inside the container is made regarding zombie processes. For a detailed explanation on its benefits and when to use it, check here: krallin/tini#8

$ cd $TMPDIR/docker-build
$ wget https://github.com/krallin/tini/archive/v0.19.0.tar.gz
$ tar xf v0.19.0.tar.gz
$ cd tini-0.19.0
$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PREFIX ..
$ make -j8
$ make install
$ ln -s $PREFIX/bin/tini-static $PREFIX/bin/docker-init

2.3.4. libnetwork

See Edit.

Another dockerd dependency needed when using the -p flag while creating a container:

$ cd $TMPDIR/docker-build
$ wget https://github.com/moby/libnetwork/archive/448016ef11309bd67541dcf4d72f1f5b7de94862.tar.gz
$ tar xf 448016ef11309bd67541dcf4d72f1f5b7de94862.tar.gz
$ mkdir -p src/github.com/docker
$ mv libnetwork-448016ef11309bd67541dcf4d72f1f5b7de94862 src/github.com/docker/libnetwork
$ export GOPATH="$(pwd)"
$ cd src/github.com/docker/libnetwork
$ go build -o docker-proxy github.com/docker/libnetwork/cmd/proxy
$ strip docker-proxy
$ install -Dm 0700 docker-proxy $PREFIX/bin/docker-proxy

2.3.5. containerd

See Edit.

This is a dockerd dependency. Some patches are needed to fix path locations, build the manuals correctly and compile extra binaries used by dockerd that are not build by default by the Makefile:

$ cd $TMPDIR/docker-build
$ wget https://github.com/containerd/containerd/archive/v1.4.3.tar.gz
$ tar xf v1.4.3.tar.gz
$ mkdir -p src/github.com/containerd
$ mv containerd-1.4.3 src/github.com/containerd/containerd
$ export GOPATH=$(pwd)
$ cd src/github.com/containerd/containerd
$ xargs sed -i "s_\(/etc/containerd\)_$PREFIX\1_g" < <(grep -R /etc/containerd | cut -d':' -f1 | sort | uniq)
$ patch runtime/v1/linux/bundle.go ../../../../bundle.go.patch.txt
$ patch runtime/v2/shim/util_unix.go ../../../../util_unix.go.patch.txt
$ patch Makefile ../../../../Makefile.patch
$ patch platforms/database.go ../../../../database.go.patch.txt
$ patch vendor/github.com/cpuguy83/go-md2man/v2/md2man.go ../../../../md2man.go.patch.txt
$ BUILDTAGS=no_btrfs make -j8
$ make -j8 man
$ DESTDIR=$PREFIX make install
$ DESTDIR=$PREFIX/share make install-man

Lastly, some configurations:

$ mkdir -p $PREFIX/etc/containerd
$ cat << "EOF" > $PREFIX/etc/containerd/config.toml
root = "/data/docker/var/lib/containerd"
state = "/data/docker/run/containerd"
imports = ["$PREFIX/etc/containerd/runtime_*.toml", "./debug.toml"]

[grpc]
  address = "/data/docker/run/containerd/containerd.sock"

[debug]
  address = "/data/docker/run/containerd/debug.sock"

[plugins]
  [plugins.opt]
    path = "/data/docker/opt"
  [plugins.cri.cni]
    bin_dir = "/data/docker/opt/cni/bin"
    conf_dir = "/data/docker/etc/cni/net.d"
EOF

Note: unfortunately containerd files also can't be stored inside Termux installation folder, failing with an error when creating the socket it uses.

2.3.6. runc

See Edit.

runc is a dependency of containerd. Conveniently for us, it's already provided by Termux's repository. Install it by simply:

$ pkg install root-repo
$ pkg install runc

3. Running

Now comes the truth time. To run the containers, first we need to start the daemon manually. To do so, it's advisable to install a terminal multiplexer so you can run the daemon in one pane and the container in others panes:

$ pkg install tmux

In one pane start dockerd:

$ sudo dockerd --iptables=false

And in others panes you can run the containers:

$ sudo docker run hello-world

Note: Teaching how to use tmux is out of the scope of this guide, you can find good tutorials on YouTube. If you don't wanna use a terminal multiplexer, you can run dockerd in the background instead, with sudo dockerd &>/dev/null &.

3.1. Caveats

3.1.1. Internet access

The two network drivers tested so far are bridge and host. Here's how to get each of them working.

bridge

This is the default netwok driver. If you don't specify a driver, this is the type of network you are creating. Bridge networks isolate the container network by editing the iptables rules and creating a network interface called Docker0 that serves as a bridge. All containers created with the bridge driver will use this interface. This is analogous to creating a VLAN and running the containers inside it.

But, there's a catch in Android: iptables rules policy is different here than on a conventional GNU/Linux system (more info here). For the bridge driver to work, you'll have to manually edit the iptable by running;

$ sudo ip route add default via 192.168.1.1 dev wlan0
$ sudo ip rule add from all lookup main pref 30000

Note: change 192.168.1.1 according to your gateway IP.

Unfortunately, this means that changing networks will require you to re-configure the rules again.

host

Using the host driver, means to remove network isolation between the container and the Docker host, and use the host’s networking directly. This way, the container will use the same network interface as your device (e.g. wlan0) and thus will share the same IP address.

To use this driver give the --net=host --dns=8.8.8.8 flags when running a container.

3.1.2. Shared volumes

An easy way to share folders and files between containers and the host is to use a shared volume. For example, using the -v ~/Documents/docker-share:/root/docker-share flag when running a container, will make the ~/Documents/docker-share folder from the host to be accessible inside the container /root/docker-share folder.

But, when talking about Android, things seems to never be as easy and straightforward as expected. Due to Android file system encryption, if you ls the /root/docker-share folder inside the container you might see a bunch of random letters, numbers and symbols instead of the folders and files names:

# ls /root/docker-share
+2xKy7JIRrcGrCf+o6KSeB  T6BJkyIa5OedXNrSyRKLbB  cwoDh,Nzt1l,5BsKA4hH8D
2aHRCQEyK8yYiiK9PEI9SA  Ue39lJVm4kIxGrS1bV07zB  lEpWZhTY9dNqJxCu+GqBuA
5ZRDLfHMwyik6RMe,f0WPA  X+yGLxXSgwxbCsFGRXuczC  y4ZWVvVBBjcxSWlJ9conED
GljgSZK5gFr7D4Fk7BHNeB  X1ATNoqhp,,ZsKjFXqKFiA
I3N5j0R4zmaQPKCWwKBlxD  Yzi+KmovJmIYFOCHtDCXkB

and if you try to read or create a file inside the volume you might get the Required key not available error.

No definitive solution was discovered so far, but a workaround is to cat the files from within the host to give the container temporary access to them. You can cat an individual file by:

$ sudo cat ~/Documents/docker-share/file.pdf >/dev/null

or all of them by:

$ sudo find ~/Documents/docker-share -exec cat {} >/dev/null \;

3.2. GUI

Yes, it's possible to run GUI programs inside a container! There's basically two ways of accomplishing it in a simple manner:

3.2.1. X11 Forwarding

Description

This method has the advantage of not making necessary the installation nor configuration of any additional programs inside the container; all you'll have to do is to setup the X inside termux and share its sockets with the container.

This is advisable to be used when you intend to run various containers with GUI, since you'll only have to install and setup a VNC once in the host, instead of doing it for each container. This will save storage space and time.

Steps

The first step is to enable the X11 repository in termux, this will allow installation of graphical interface related programs, like the VNC server we'll be using.

$ pkg install x11-repo

Then install a VNC server in termux:

$ pkg install tigervnc

Note: These installations steps need to be executed only once.

Now, just run it:

$ vncserver -noxstartup -localhost

Note: It's advisable to pass the -geometry HEIGHTxWEIGHT flag substituting HEIGHT and WEIGHT by your phone's screen resolution or some multiple of it.

Note: The very first time you run it, you'll be prompted to setup a password. Note that passwords are not visible when you are typing them and it's maximal length is 8 characters. If you don't wanna use a passwd, use the -SecurityTypes none flag.

If everything is okay, you will see this message:

New 'localhost:1 ()' desktop is localhost:1

It means that X (vnc) server is available on display 'localhost:1'. Finally, export the DISPLAY environment variable according to that value:

$ export DISPLAY=:1

Now that the VNC server is configured and running in the host, start the container sharing the X related files as volumes:

$ sudo docker run -ti \
    --net="host" \
    --dns="8.8.8.8" \
    -e DISPLAY=$DISPLAY \
    -v $TMPDIR/.X11-unix:/tmp/.X11-unix \
    -v $HOME/.Xauthority:/root/.Xauthority \
    ubuntu

Note: If by any reason you forget to export the DISPLAY before starting the container, you can still export it from inside it.

You'll now be able to launch GUI programs from inside the container, e.g.:

# echo 'APT::Sandbox::User "root";' > /etc/apt/apt.conf
# apt update
# apt install x11-apps
# xeyes

To check the GUI, you'll need to install a VNC client app in your Android phone, like VNC Viewer (developed by RealVNC Limited). Unfortunately it's not open source, but it's a good and intuitive VNC client for Android.

Note: There's also an open source alternative developed by @pelya called XServer XSDL, which will not be covered by this guide (for now).

After installing the VNC Viewer app, open it and setup a new connection using 127.0.0.1 (or localhost) as the IP, 5901 as the port (the port is calculated as 5900 + {display number}) and when/if prompted, type the password choosen when running vnctiger for the first time.

3.2.2. VNC server within the container

Description

This method is very similar to the previous, with the difference that the X server will be installed inside the container instead of in the termux host.

The advantages are:

  1. you aren't changing your host system by installing softwares on it (like the VNC server);
  2. security, since you won't be sharing your host's X (this is only relevant when you are not the one running the container).

The main disadvantage is that you'll need to install and config the VNC server for each container you'd run a GUI program, thus making these containers big and time consuming to setup.

Steps

First, start a container:

$ sudo docker run -ti \
    --net="host" \
    --dns="8.8.8.8" \
    ubuntu

Now, a VNC server needs to be installed and configured inside the container. You can choose between TigerVNC or x11vnc:

TigerVNC

The same VNC server used above in termux. To install it:

# echo 'APT::Sandbox::User "root";' > /etc/apt/apt.conf
# apt update
# apt install tigervnc-standalone-server

Next, start it with:

# vncserver -noxstartup -localhost -SecurityTypes none

Here we disabled password (-SecurityTypes none) because using it causes things to crash as described in this issue report TigerVNC/tigervnc#800

If everything is okay, you will see this message:

New 'localhost:1 (root)' desktop at :1 on machine localhost

Export the DISPLAY environment variable according to that value:

# export DISPLAY=:1

From now on, you can already run GUI programs and access them using the VNC Viewer client as already described in the end of X11 Forwarding steps.

x11vnc

Install the x11vnc and the virtual fake X (since x11vnc can't emulate a X11 by itself):

# echo 'APT::Sandbox::User "root";' > /etc/apt/apt.conf
# apt update
# apt install x11vnc xvfb

Now, start it:

# x11vnc -nopw -forever -noshm -create

If everything is okay, you will see this message:

The VNC desktop is:      localhost:0
PORT=5900

This will open a xterm terminal which can be acessed by the VNC Viewer client as already described in the end of X11 Forwarding steps. From that terminal you can open the desired GUI program.

3.3. Steam (work in progress)

I'm not talking about running the useless steam app for Android, but about running the Desktop version and play the games inside a docker container. Yes, you read it right, it's possible to play your Steam games on Android!

(ACTUALLY NOT YET, BECAUSE I DIDN'T MANAGE TO GET OPENGL TO WORK, THAT'S WHY THIS IS A WORK IN PROGRESS. TO CONTRIBUTE OR STAY UP TO DATE ABOUT THE PROGRESS CHECK ptitSeb/box86#249)

To do so, we'll use an awesome x86 emulator for ARM developed by @ptitSeb called box86.

But first, you need to enable System V IPC under General Setup in the kernel config and recompile it again. That's because the steam binary needs some semaphore functions and will crash in case it can't use them.

Next, we hit a problem: box86 can only be compiled by a 32 bit toolchain. But, in fact, this can be easily circumvented by using a 32 bit container:

$ sudo docker run -ti \
    --net="host" \
    --dns="8.8.8.8" \
    -e DISPLAY=$DISPLAY \
    -w /root \
    -v $TMPDIR/.X11-unix:/tmp/.X11-unix \
    -v $HOME/.Xauthority:/root/.Xauthority \
    --platform=linux/arm \
    arm32v7/ubuntu

Note: if your system is 32 bit already (run uname -m to check), you don't need to specify the --platform=linux/arm flag and can simply use ubuntu instead of arm32v7/ubuntu.

Now that we are inside the container, let's install the tools we're gonna use, as well as the steam .deb installer:

# echo 'APT::Sandbox::User root;' >> /etc/apt/apt.conf
# apt update
# apt install wget binutils xterm libvdpau1 libappindicator1 libnm0 libdbusmenu-gtk4

Install steam:

# wget https://steamcdn-a.akamaihd.net/client/installer/steam.deb
# ar x steam.deb
# mkdir steam
# tar xf data.tar.xz -C steam
# find steam -type d -exec sh -c 'mkdir -p /${0#*/}' {} \;
# find steam \! -type d -exec sh -c 'mv $0 /${0#*/}' {} \;
# patch /usr/lib/steam/bin_steam.sh bin_steam.sh.patch
# rm -rf steam* *.tar* bin_steam.sh.patch
# steam

Steam will fail with a bunch of errors, but that's expected. The important thing is that it installed the necessary files under ~/.local/share/Steam, one of them being the steam binary. Finish the installation by adding it to the path:

# ln -sf /root/.local/share/Steam/ubuntu12_32/steam /usr/bin/steam

Now, we need to install the i386 version of some libs required by steam. For this, we're going to download them directly from Ubuntu packages. That's because if we instead simply apt install them we would be getting the arm32 version.

4. Attachments

4.1. kernel patches

  • kernel/Makefile
diff --git a/kernel/Makefile b/kernel/Makefile
index d5c1115..2dea801 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -121,7 +121,7 @@ $(obj)/configs.o: $(obj)/config_data.h
# config_data.h contains the same information as ikconfig.h but gzipped.
# Info from config_data can be extracted from /proc/config*
targets += config_data.gz
-$(obj)/config_data.gz: arch/arm64/configs/lavender_stock-defconfig FORCE
+$(obj)/config_data.gz: $(KCONFIG_CONFIG) FORCE
    $(call if_changed,gzip)

    filechk_ikconfiggz = (echo "static const char kernel_config_data[] __used = MAGIC_START"; cat $< | scripts/basic/bin2c; echo "MAGIC_END;")
  • net/netfilter/xt_qtaguid.c
--- orig/net/netfilter/xt_qtaguid.c     2020-05-12 12:13:14.000000000 +0300
+++ my/net/netfilter/xt_qtaguid.c       2019-09-15 23:56:45.000000000 +0300
@@ -737,7 +737,7 @@
{
        struct proc_iface_stat_fmt_info *p = m->private;
        struct iface_stat *iface_entry;
-       struct rtnl_link_stats64 dev_stats, *stats;
+       struct rtnl_link_stats64 *stats;
        struct rtnl_link_stats64 no_dev_stats = {0};  
@@ -745,13 +745,8 @@
        current->pid, current->tgid, from_kuid(&init_user_ns, current_fsuid()));
        iface_entry = list_entry(v, struct iface_stat, list);
+       stats = &no_dev_stats; 
-       if (iface_entry->active) {
-               stats = dev_get_stats(iface_entry->net_dev,
-                                     &dev_stats);
-       } else {
-               stats = &no_dev_stats;
-       }
        /*
         * If the meaning of the data changes, then update the fmtX
         * string.

4.2. docker-cli patches

4.3. dockerd patches

4.4. containerd patches

5. Aknowledgements

I'd like to thank the Termux Dev team for this wonderful app and @xeffyr for discovering about the bug in net/netfilter/xt_qtaguid.c and sharing the patch, as well as all the conversation we had here that led to docker finally working.

Also @yjwong, for figuring out how to use the bridge network driver.

6. Final notes

If you are a docker developer reading this, please consider adding an official support for Android. Look above the possibilities it opens for a smartphone. If you are not a docker developer, consider supporting this by showing interest here. If we annoy the devs enough, this may become official (of they may simply unsubscribe from the thread and let it rot in the Issues section ¯\_(ツ)_/¯ ).

@pranav-bhatt
Copy link

pranav-bhatt commented Apr 13, 2024

Cannot connect to the Docker daemon at unix:///var/run/docker.sock

This worked for me:
sudo DOCKER_HOST=<the docker.sock location spit out when you start dockerd> docker-compose up

@binhvo7794
Copy link

Has anyone encountered a problem where the system freezes and can't open any applications when running Docker?

@arminask
Copy link

arminask commented Jun 12, 2024

Hey, I'm using https://github.com/LineageOS/android_kernel_oneplus_sm8150 kernel, however it doesn't have xt_qtaguid.c file, so I cannot even patch it.
I've compiled the kernel either way, however when starting dockerd, it says error="getCPUInfo for OS android: not implemented".
Also not sure if I enabled something or what, but after my kernel changes Wi-Fi doesn't work on Android :/ Unrelated.

@brokeDude2901
Copy link

brokeDude2901 commented Jun 16, 2024

LinuxDeployPro docker info
Screenshot_20240616-220642_Linux Deploy Screenshot_20240616-220812_Termius
  • permission denied: use vfs storage driver or manually backport your overlayfs to newer commit (you should have Native Overlay Diff: true under docker info)

@brokeDude2901
Copy link

Hey, I'm using https://github.com/LineageOS/android_kernel_oneplus_sm8150 kernel, however it doesn't have xt_qtaguid.c file, so I cannot even patch it. I've compiled the kernel either way, however when starting dockerd, it says error="getCPUInfo for OS android: not implemented". Also not sure if I enabled something or what, but after my kernel changes Wi-Fi doesn't work on Android :/ Unrelated.

Has anyone encountered a problem where the system freezes and can't open any applications when running Docker?

might wanna check dmesg

@lawal1232
Copy link

lawal1232 commented Jun 16, 2024

I found it really helpful to learn how to run Docker on Android but it didn't work for me. However, I recently discovered an even better way to install Docker on my Samsung Galaxy A5. I followed a tutorial on this page https://crackoverflow.com/docs/system_administration/containerization/install-docker-natively-on-android-phone-and-use-it-as-a-home-server/ , and it provided a more streamlined process with additional features. You basically install linux on the phone and run docker on it. Now i have my phone as home server.
Nonetheless, I appreciate the effort you put into sharing your knowledge. Keep up the good work!

@james28909
Copy link

i have a US variant samsung galaxy note 9 (N960U) running docker. i was able to recompile my kernel and then downgraded to the previous version of dockerd and then it worked fine.

the problem i am facing is trying to connect to some kind of gui remotely. im not sure if it is because of my setup/hardware or some configuration error. i have built in wifi on my main computer, with hotspot turned on and the note 9 is connected to that wifi hotspot shared from my pc.

optimally i would like to use windows built in RDP if possible. i tried to connect to it via rdp (it connects it seems) but at the xorg login screen, i enter my credentials and the rdp window goes black for a split second then it just crashes back to windows rdp as if nothing ever even happened.where can i find log files? i think its because of the display isnt able to be found. ive tried :0 and :1 etc but i dont think it makes a difference. is there some way to forward rdp pass-through termux to the docker container?

any help would sure be appreciated! :)

@nemocrk
Copy link

nemocrk commented Jun 24, 2024

i'm having a strange problem.
I'm using docker from termux-root and containerd from reddit: link
this is my docker info:

Client:
 Version:    v1:24.0.6-ce
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: dev
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8.m
 runc version:
 init version: N/A
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 4.4.302-perf+
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 5.588GiB
 Name: localhost
 ID: d1e4889d-7110-42ad-ab80-4e69daebeafd
 Docker Root Dir: /data/docker/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

the only thing i modified is
$PREFIX/etc/docker/daemon.json
because of overlay error

{
    "data-root": "/data/docker/lib/docker",
    "exec-root": "/data/docker/run/docker",
    "pidfile": "/data/data/com.termux/files/usr/var/run/docker.pid",
    "hosts": [
        "unix:///data/data/com.termux/files/usr/var/run/docker.sock"
    ],
    "storage-driver": "overlay2"
}

when i simply start sudo docker run hello-world i get this error:
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: bpf_prog_query(BPF_CGROUP_DEVICE) failed: invalid argument: unknown.

googling around i found that as a solution :
The solution was to run: sudo mount -t tmpfs cgroup_root /sys/fs/cgroup which created directories for each of the cgroup resources and allowed Docker to run. #https://android.stackexchange.com/questions/249453/how-can-i-enable-cpu-memory-and-io-in-cgroupv2-attempt-to-run-docker-on-andro

after this now when i start sudo docker run hello-world it simply stuck without relevant messages in dmesg and in dockerd

any idea?

EDIT:

it's stranger then before.
i noticed that after a reboot and forcing to use graphdriver="vfs" when i launch the command, the first time starts but after it stop.

EDIT2:

Downgraded to LineageOS 19.1 and now it works.
probably a problema introduced by Android 14

@whyakari
Copy link

whyakari commented Jul 9, 2024

Error:

docker: Error response from daemon: failed to create task for container: failed to start shim: start failed: io.containerd.runc.v2: create new shim socket: listen unix /data/data/com.termux/files/usr/var/run/containerd/s/1da210b0827a1c772c72b223e4f1d42d8604457319c91bfc896ef8bb8919411e: bind: invalid argument: exit status 1: unknown.
ERRO[0000] error waiting for container:

There's an open bug report about it since last year -

termux/termux-packages#18359

The current workaround is to use previous version of containerd -

curl -o containerd_1.6.21-1_aarch64.deb -L "https://www.dropbox.com/scl/fi/9ihg5gnlzwtbookjs5szy/containerd_1.6.21-1_aarch64.deb?rlkey=vsywpuz3e2x3edu72yklwvod6&st=tzl93ri9&dl=1" && pkg install -y --allow-downgrades --allow-change-held-packages ./containerd_1.6.21-1_aarch64.deb && apt-mark hold containerd

Thanks to Near_Earth

@dxvx
Copy link

dxvx commented Jul 17, 2024

zuk z2 with lineageos 17.1 finally bootup the docker and run hello world. But system will crash in about 3 minutes without any measurable unusual signal.

@aikooo7
Copy link

aikooo7 commented Jul 27, 2024

Is it possible to use this in mobile data @FreddieOliveira ?

@bigfoxtail
Copy link

i'm having a strange problem. I'm using docker from termux-root and containerd from reddit: link this is my docker info:

Client:
 Version:    v1:24.0.6-ce
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: dev
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8.m
 runc version:
 init version: N/A
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 4.4.302-perf+
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 5.588GiB
 Name: localhost
 ID: d1e4889d-7110-42ad-ab80-4e69daebeafd
 Docker Root Dir: /data/docker/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

the only thing i modified is $PREFIX/etc/docker/daemon.json because of overlay error

{
    "data-root": "/data/docker/lib/docker",
    "exec-root": "/data/docker/run/docker",
    "pidfile": "/data/data/com.termux/files/usr/var/run/docker.pid",
    "hosts": [
        "unix:///data/data/com.termux/files/usr/var/run/docker.sock"
    ],
    "storage-driver": "overlay2"
}

when i simply start sudo docker run hello-world i get this error: docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: bpf_prog_query(BPF_CGROUP_DEVICE) failed: invalid argument: unknown.

googling around i found that as a solution : The solution was to run: sudo mount -t tmpfs cgroup_root /sys/fs/cgroup which created directories for each of the cgroup resources and allowed Docker to run. #https://android.stackexchange.com/questions/249453/how-can-i-enable-cpu-memory-and-io-in-cgroupv2-attempt-to-run-docker-on-andro

after this now when i start sudo docker run hello-world it simply stuck without relevant messages in dmesg and in dockerd

any idea?

EDIT:

it's stranger then before. i noticed that after a reboot and forcing to use graphdriver="vfs" when i launch the command, the first time starts but after it stop.

EDIT2:

Downgraded to LineageOS 19.1 and now it works. probably a problema introduced by Android 14

I have the same problem, is there a solution now?

@pejhar
Copy link

pejhar commented Aug 12, 2024

How can I Use Docker Compose on Android?

@yyqwe158
Copy link

how did you go about finding the bug in net/netfilter/xt_qtaguid.c ? I did the changes in my kernel source here but I'm still getting the freeze .. any leads would be appreciable!

I also encounter this problem...Do you find a way to solve this?

@whyakari
Copy link

After flashing a custom kernels with all necessary configs (except zfs and apparmor), i still cannot run sudo docker run hello-world . When trying to run this docker image, I get the following error:

INFO[2023-11-08T09:23:38.969808191Z] Daemon has completed initialization  
INFO[2023-11-08T09:23:39.021227102Z] API listen on /data/data/com.termux/files/usr/var/run/docker.sock       
ERRO[2023-11-08T09:23:48.152404522Z] copy shim log                                 error="read /proc/self/fd/14: file already closed"    
ERRO[2023-11-08T09:23:48.155390876Z] stream copy error: reading from a closed fifo      
ERRO[2023-11-08T09:23:48.155572335Z] stream copy error: reading from a closed fifo      
ERRO[2023-11-08T09:23:48.265803492Z] Handler for POST /v1.43/containers
/45907966238bc87b42a3c17a597203dc338e69516f5a3c3cb5d3c143d066d3fc/start returned error: failed to create task for container: failed
 to start shim: start failed: io.containerd.runc.v2: create new shim socket: listen unix /data/data/com.termux/files/usr/var/run/containerd
/s/a60e5bf84cdcea9929275c0fad94918c243db03df9d92ab7e518c494ffc3a294: bind: invalid argument: exit status 1: unknown

Using docker version v1:24.0.6-ce
What could be the issue?

Yes I also same error

broken package

You mean a broken docker package? Using another version should work?
EDIT: seems to be an issue in the termux package, termux/termux-packages#18359

yes, path is too long, package should be recompiled with old pathes at least for containerd
https://unix.stackexchange.com/questions/367008/why-is-socket-path-length-limited-to-a-hundred-chars

Hello, have you compiled it? I'm trying to compile, but I don't know where to start, and I can't find the archive for the previous Docker version.

the last couple push introduced the TERMUX_PREFIX, so basically moved the whole docker and containerd data root and state folder into the termux scope, instead of working in the root /data its now should work from /data/data/com.termux/files/usr/var/run/containerd/s + 64 random char which is around 117char which is more than the limit 108. The pusher either has raised its devices kernels path limit or havent tested it before releasing the pkg, or of course we just dont see something.
I have tried to recompile from before these commits ( from 30cb8aa) but there are pkgs it cant download as they are not hosted now. Also tried with modifying the config.toml, containerd.toml and the daemon.json to make it look for a shorter custom folder, no success, that path kind of baked in the docker daemon.
There might be a patch which modified what is the root folder, but not yet found, tried modifying the DefaultStateRoot which is being updated by a new patch from last commits but no success, so further investigation for fix is needed.

I found the historical versions 1:20.10.24 and 1:20.10.24-1 in the mirror repository mirrors.aliyun.com, and they are working fine :)

How did you install the historic version? Termux can't connect with the mirror. I tried downloading the deb package from https://mirrors.aliyun.com/termux/termux-root/pool/stable/d/docker/docker_1:20.10.24_aarch64.deb and installing locally, but then docker refuses to start (timeout on containerd).

docker 1:20.10.24 containerd 1.6.21.1-1
Due to some dependencies of Docker, you can first install Docker, then uninstall it, followed by uninstalling containerd. After that, install an older version of containerd and then an older version of Docker. Finally, don't forget to modify daemon.json. Since the Aliyun mirror is deployed in China, you may need some special means to access it. Also, remember to backup the older versions because you never know when the mirror will sync to the new version, and there is no archive for Termux software source.

Thanks for the help, but I can't get it working. :/

pkg install docker
pkg uninstall docker
pkg uninstall containerd
dpkg -i containerd_1.6.21-1_aarch64.deb
dpkg -i docker_1: 20.10.24_aarch64. deb

But I still get following error:

- $ sudo docker run hello-world
docker: Error response from daemon: failed to create shim task
OC I runtime create failed: runc create failed: unable to sta
r t container process: error during container init: error mount
ing "mqueue" to rootfs at "/dev/mqueue": mount mqueue: /dev/mqu
eue (via /proc/seIf/fd/6), flags: Oxe: device or resource busy
unknown.
[0000] error waiting for container: context cancel ed
ERRO

You mentioned modifying the daemon.json, but this is already correct when installing the older version?

"data-root": "/data/docker/lib/docker" ,
"exec-root": "/data/docker/run/docker" ,
"pidfile": "/data/docker/ run/docker. pid" ,
"hosts"
"unix : / / /data / docker/ run/ docker. sock"
"storage -driver": "overlay2"

try disabling the CONFIG_USER_NS and recompile kernel

Thank you very much!!
Sudo docker run hello-world runs fine. Now the next challenge is fixing docker-compose... :/

 $ sudo docker-compose up
Cannot connect to the Docker daemon at unix:///var/ run/docker .
sock. Is the docker daemon running?

did you manage to find a fix for docker compose?

to fix it, I used this:

mkdir -p /var/run # (in /)

and then run

sudo mount --bind /data/docker/run /var/run

@brokeDude2901
Copy link

Guys, beware there are NASTY bugs in your Docker kernel that could kill/brick your phone !

I've been running Docker on SM-N950 for 2-3 months with LXC on rooted Termux but then last week suddenly there are read/write errors, either the NAND flash or NAND controller is broken and flashing back to stock rom didn't work either.

Also, there are various bugs and cons:

  1. ARM64 openblas libraries will burn all CPU cores (ie: numpy, ...), OpenMathLib/OpenBLAS#4365
  2. Docker container randomly freeze/error after a day or so (?)
  3. Android just randomly remove/alter permission on your Docker/Termux folder after a few days (?)
  4. if the screen is off, Android turn off big cores, little cores are slow as F
  5. when the phone is hot or below 50% battery, wifi speed will be throttle to 50mbps

@aikooo7
Copy link

aikooo7 commented Aug 28, 2024

Guys, beware there are NASTY bugs in your Docker kernel that could kill/brick your phone !

I've been running Docker on SM-N950 for 2-3 months with LXC on rooted Termux but then last week suddenly there are read/write errors, either the NAND flash or NAND controller is broken and flashing back to stock rom didn't work either.

Also, there are various bugs and cons:

1. ARM64 openblas libraries will burn all CPU cores (ie: numpy, ...), [OpenMathLib/OpenBLAS#4365](https://github.com/OpenMathLib/OpenBLAS/issues/4365)

2. Docker container randomly freeze/error after a day or so (?)

3. Android just randomly remove/alter permission on your Docker/Termux folder after a few days (?)

4. if the screen is off, Android turn off big cores, little cores are slow as F

5. when the phone is hot or below 50% battery, wifi speed will be throttle to 50mbps

Hey,

About your NAND flash, sorry about that and I hope it doesn't happen with me or anyone else, for me as far never happened to me.

  1. My android is always cold and definitely not using all cores 100%
  2. That actually happened to me but now it stopped.
  3. Never happened to me.
  4. Maybe something to do with doze?
  5. Can't confirm, might be true.

@Gato-Chamuscador
Copy link

hey! im compiling the lineage os miatoll kernel but i can't find net/netfilter/xt_qtaguid.c in ~/android/lineage/kernel/xiaomi/sm6250/net/netfilter i think that there's no such file in my kernel

@whyakari
Copy link

whyakari commented Oct 23, 2024

hey! im compiling the lineage os miatoll kernel but i can't find net/netfilter/xt_qtaguid.c in ~/android/lineage/kernel/xiaomi/sm6250/net/netfilter i think that there's no such file in my kernel

You won't find it because it only exists in older kernel versions 4.9. Google changed it to eBPF-based.

Starting with Android 9, Android devices running on kernel 4.9 or above and originally shipped with the P release MUST use eBPF-based network traffic monitoring accounting instead of xt_qtaguid. The new infrastructure is more flexible and more maintainable and does not require any out-of-tree kernel code.

you can read the Google docs here
https://source.android.com/docs/core/data/ebpf-traffic-monitor

@Gato-Chamuscador
Copy link

A

hey! im compiling the lineage os miatoll kernel but i can't find net/netfilter/xt_qtaguid.c in ~/android/lineage/kernel/xiaomi/sm6250/net/netfilter i think that there's no such file in my kernel

You won't find it because it only exists in older kernel versions 4.9. Google changed it to eBPF-based.

Starting with Android 9, Android devices running on kernel 4.9 or above and originally shipped with the P release MUST use eBPF-based network traffic monitoring accounting instead of xt_qtaguid. The new infrastructure is more flexible and more maintainable and does not require any out-of-tree kernel code.

you can read the Google docs here https://source.android.com/docs/core/data/ebpf-traffic-monitor

So I don't have to modify any file right?

@whyakari
Copy link

A

hey! im compiling the lineage os miatoll kernel but i can't find net/netfilter/xt_qtaguid.c in ~/android/lineage/kernel/xiaomi/sm6250/net/netfilter i think that there's no such file in my kernel

You won't find it because it only exists in older kernel versions 4.9. Google changed it to eBPF-based.
Starting with Android 9, Android devices running on kernel 4.9 or above and originally shipped with the P release MUST use eBPF-based network traffic monitoring accounting instead of xt_qtaguid. The new infrastructure is more flexible and more maintainable and does not require any out-of-tree kernel code.
you can read the Google docs here https://source.android.com/docs/core/data/ebpf-traffic-monitor

So I don't have to modify any file right?

as far as I remember, no

Copy link

ghost commented Nov 6, 2024

Hi, I ported docker-stable for termux, check this repo termux-docker-stable, all packages is update now :D

@whyakari
Copy link

whyakari commented Nov 7, 2024

Hi, I ported docker-stable for termux, check this repo termux-docker-stable, all packages is update now :D

idk why but it looks the same as mine repo 🤷

Copy link

ghost commented Nov 7, 2024

Hi, I ported docker-stable for termux, check this repo termux-docker-stable, all packages is update now :D

idk why but it looks the same as mine repo 🤷

I don't know 😂, but I never copy and paste🤡

@NikMAX2077
Copy link

docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't get final child's PID from pipe: EOF: unknown.
ERRO[0000] error waiting for container: context canceled

I'm facing the same problem. Have you found a solution?

@TopNotchSushi
Copy link

For recent Android devices that doesn't use sdcardfs1, and with external_storage.casefold.enabled=1. Docker might complain about overlayfs driver failed and fallback to vfs driver.

This is because userdata partition is formatted with casefold option to support case insensitive /sdcard. Most parts of userdata partition is still case sensitive2, but overlayfs refuses to work because it detects case sensitivity by checking DCACHE_OP_{HASH,COMPARE} on direntry.

To continue to use overlayfs driver for docker, three options should be available:

  1. Revert to use sdcardfs. I don't like and never tried this approach.
  2. Disable casefold. See build/make/target/product/emulated_storage.mk for more details. This option is easy when building AOSP. But it will make /sdcard case sensitive, which might break some Android applications. The implication is not clear but I guess there's almost no implication.
  3. Disable the DCACHE_OP_{HASH,COMPARE} check in overlayfs. If I understand it correctly, this only shifts the responsibility of ensuring the filesystem used by overlayfs case-sensitive to the users. The kernel patch is here: android-kxxt/android_kernel_xiaomi_sm8450@ae700d3. USE IT AT YOUR OWN RISK because I know nothing about DCACHE_OP_{HASH,COMPARE} and I am not really clear if DCACHE_OP_{HASH,COMPARE} is only meant for case sensitivity check3.

Footnotes

  1. https://source.android.com/docs/core/storage/sdcardfs-deprecate
  2. https://android.googlesource.com/platform/system/vold/+/5b711b10dbbec26cd8157672f12566b525d0d2bb/model/PrivateVolume.cpp#178
  3. https://lore.kernel.org/all/CAKQB+fuYbWHm+CiYHDycrre5RZdCVKPRuqz83FnkGF2_0siz8w@mail.gmail.com/T/

Thanks for this! This is exactly what I needed to get my stock pixel 7 + configured kernel working with the latest version of docker (not an older package) + overlay 2 working. My last hurdle seems to be docker networking. None of the proposed solutions here seem to be working aside from running the container as host networking.

@0d129
Copy link

0d129 commented Nov 21, 2024

@TopNotchSushi
Hi, I got CONFIG_OVERLAY_FS: enabled in the check script, and I changed the daemon.json to mount files under /data/docker . Still met error when I exec sudo dockerd --iptables=false, it said:

INFO[2024-11-21T03:37:23.774876107Z] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2024-11-21T03:37:23.775936576Z] serving...                                    address=/data/docker/run/docker/containerd/containerd-debug.sock
INFO[2024-11-21T03:37:23.776428347Z] serving...                                    address=/data/docker/run/docker/containerd/containerd.sock.ttrpc
INFO[2024-11-21T03:37:23.776853190Z] serving...                                    address=/data/docker/run/docker/containerd/containerd.sock
INFO[2024-11-21T03:37:23.776962097Z] containerd successfully booted in 0.097642s
INFO[2024-11-21T03:37:23.826019966Z] [graphdriver] trying configured driver: overlay2
ERRO[2024-11-21T03:37:23.827942570Z] failed to mount overlay: invalid argument     storage-driver=overlay2
INFO[2024-11-21T03:37:23.842289551Z] stopping healthcheck following graceful shutdown  module=libcontainerd
INFO[2024-11-21T03:37:23.842930020Z] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: error initializing graphdriver: driver not supported: overlay2

is it the same issue as you said? May I know how did you solve it (which method you choose)

@TopNotchSushi
Copy link

TopNotchSushi commented Nov 22, 2024

@TopNotchSushi Hi, I got CONFIG_OVERLAY_FS: enabled in the check script, and I changed the daemon.json to mount files under /data/docker . Still met error when I exec sudo dockerd --iptables=false, it said:

INFO[2024-11-21T03:37:23.774876107Z] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2024-11-21T03:37:23.775936576Z] serving...                                    address=/data/docker/run/docker/containerd/containerd-debug.sock
INFO[2024-11-21T03:37:23.776428347Z] serving...                                    address=/data/docker/run/docker/containerd/containerd.sock.ttrpc
INFO[2024-11-21T03:37:23.776853190Z] serving...                                    address=/data/docker/run/docker/containerd/containerd.sock
INFO[2024-11-21T03:37:23.776962097Z] containerd successfully booted in 0.097642s
INFO[2024-11-21T03:37:23.826019966Z] [graphdriver] trying configured driver: overlay2
ERRO[2024-11-21T03:37:23.827942570Z] failed to mount overlay: invalid argument     storage-driver=overlay2
INFO[2024-11-21T03:37:23.842289551Z] stopping healthcheck following graceful shutdown  module=libcontainerd
INFO[2024-11-21T03:37:23.842930020Z] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: error initializing graphdriver: driver not supported: overlay2

is it the same issue as you said? May I know how did you solve it (which method you choose)

This one

@0d129
Copy link

0d129 commented Nov 22, 2024

@TopNotchSushi Thanks!
btw, for anyone who stuck on this, following @brokeDude2901 's way, changed it to vfs, could still functional, at least dockerd could be running.

@whyakari
Copy link

@TopNotchSushi Hi, I got CONFIG_OVERLAY_FS: enabled in the check script, and I changed the daemon.json to mount files under /data/docker . Still met error when I exec sudo dockerd --iptables=false, it said:

INFO[2024-11-21T03:37:23.774876107Z] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2024-11-21T03:37:23.775936576Z] serving...                                    address=/data/docker/run/docker/containerd/containerd-debug.sock
INFO[2024-11-21T03:37:23.776428347Z] serving...                                    address=/data/docker/run/docker/containerd/containerd.sock.ttrpc
INFO[2024-11-21T03:37:23.776853190Z] serving...                                    address=/data/docker/run/docker/containerd/containerd.sock
INFO[2024-11-21T03:37:23.776962097Z] containerd successfully booted in 0.097642s
INFO[2024-11-21T03:37:23.826019966Z] [graphdriver] trying configured driver: overlay2
ERRO[2024-11-21T03:37:23.827942570Z] failed to mount overlay: invalid argument     storage-driver=overlay2
INFO[2024-11-21T03:37:23.842289551Z] stopping healthcheck following graceful shutdown  module=libcontainerd
INFO[2024-11-21T03:37:23.842930020Z] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
failed to start daemon: error initializing graphdriver: driver not supported: overlay2

is it the same issue as you said? May I know how did you solve it (which method you choose)

This one

thanks! this worked. i can use everything now, on a 5.4 kernel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment