Skip to content

Instantly share code, notes, and snippets.

@unknown321
Last active February 5, 2023 15:01
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save unknown321/0890745aa101e5eaaf2ca50efb394a07 to your computer and use it in GitHub Desktop.
Save unknown321/0890745aa101e5eaaf2ca50efb394a07 to your computer and use it in GitHub Desktop.
docker on synology DS218j armv7l synology_armada38x_ds218j

You want to run docker on armv7l. There are no builds. You build it yourself. I build it on debian 10.

Dependencies:

Clone everything, create folder for binaries and libs:

mkdir synology-docker && cd synology-docker
mkdir bin
mkdir lib
git clone <reponame>

Build dockerd:

cd moby
git checkout v20.10.5
make DOCKER_CROSSPLATFORMS=linux/arm/v5 cross
cp -rv bundles/cross/linux/arm/v5/dockerd-dev ../bin/dockerd
cd ..

Build runc without seccomp:

cd runc
git checkout 59ad417c14143ae6b34e9cf88cf3f6e9c6d5f9e8
# there will be an error because you didn't install gcc-arm-linux-gnueabihf package
# we don't need it, so just ignore the error and grab binary
make BUILDTAGS= localcross
cp -v runc-armel ../bin/runc
cd ..

Patch and build containerd:

cd containerd
git checkout a72fe7da21237815731386d6b73a0e93700112f9
patch Makefile build_all_bins_for_armv7l.patch
make binaries
cp -rv bin/* ../bin
cd ..

Build docker cli:

cd cli
git checkout v20.10.5
make -f docker.Makefile cross
cp build/docker-linux-arm ../bin/docker
cd ..

Follow Synology DSM Developer Guide to build iptables package (most likely there will be problems with DSM version detection) or do it without toolkit utility. Let's assume that unpacked toolchain is in ./build_env/ds.armada38x-6.2 directory:

mkdir ./build_env/ds.armada38x-6.2/source/
cp -r iptables ./build_env/ds.armada38x-6.2/source/
sudo chroot ./build_env/ds.armada38x-6.2
  
# you are chrooted
# CHROOT@ds.armada38x[/]#
cd source/iptables
git checkout v1.6.0
./autogen.sh
./configure --disable-devel  --host=arm-unknown-linux-gnueabi
# make will throw an error, but it doesn't matter since we need only iptables extensions
make
exit
  
# not chrooted
cp -v build_env/ds.armada38x-6.2/source/iptables/extensions/libxt_{addrtype,conntrack}.so ../lib

Copy both iptables libs to /usr/lib/iptables/ on synology. You can start iptables service by enabling firewall service in settings or somehow figure out how to start it from console.

Copy binaries to /usr/local/bin

Run everything in different terminal sessions on synology as root:

containerd
echo 1 > /proc/sys/net/ipv4/ip_forward
synofirewall --enable
syno_iptables_common load_nat_mod

source /usr/syno/etc.defaults/iptables_modules_list
iptablestool --insmod docker ${KERNEL_MODULES_CORE} ${KERNEL_MODULES_COMMON} ${KERNEL_MODULES_NAT} ${IPV6_MODULES}
BRIDGE="stp.ko bridge.ko"
AUFS="aufs.ko"
IPTABLES="xt_conntrack.ko xt_addrtype.ko veth.ko"
MODULES="$BRIDGE $AUFS $IPTABLES"
for i in $MODULES; do 
  /sbin/insmod /lib/modules/$i
done

dockerd

Run something:

docker pull --platform linux/arm/7 alpine:3.12
docker run -it --rm -v /dev:/dev --network=host alpine:3.12

/ # cat /etc/alpine-release 
3.12.4
/ # uname -a
Linux syn 3.10.105 #25426 SMP Tue May 12 04:42:24 CST 2020 armv7l Linux
/ # exit

root@syn:~# uname -a
Linux syn 3.10.105 #25426 SMP Tue May 12 04:42:24 CST 2020 armv7l GNU/Linux synology_armada38x_ds218j

Issues:

  • no network unless --network=host
  • weird issue with /dev/ptmx, has to mount /dev into container
  • no seccomp

I was unable to compile runc with seccomp support in chroot. Note: Fedora has static libseccomp.

Running docker without seccomp means that container can make whatever syscall it wants. Don't use it in production.

diff --git Makefile Makefile
index 399c7db3a..345c9d9fd 100644
--- Makefile
+++ Makefile
@@ -198,25 +198,25 @@ benchmark: ## run benchmarks tests
FORCE:
define BUILD_BINARY =
-@echo "$(WHALE) $@"
-@go build ${DEBUG_GO_GCFLAGS} ${GO_GCFLAGS} ${GO_BUILD_FLAGS} -o $@ ${GO_LDFLAGS} ${GO_TAGS} ./$<
+echo "$(WHALE) $@"
+CC=arm-linux-gnueabi-gcc-8 CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=7 go build ${DEBUG_GO_GCFLAGS} ${GO_GCFLAGS} ${GO_BUILD_FLAGS} -o $@ ${GO_LDFLAGS} ${GO_TAGS} ./$<
endef
# Build a binary from a cmd.
bin/%: cmd/% FORCE
$(BUILD_BINARY)
-bin/containerd-shim: cmd/containerd-shim FORCE # set !cgo and omit pie for a static shim build: https://github.com/golang/go/issues/17789#issuecomment-258542220
- @echo "$(WHALE) bin/containerd-shim"
- @CGO_ENABLED=${SHIM_CGO_ENABLED} go build ${GO_BUILD_FLAGS} -o bin/containerd-shim ${SHIM_GO_LDFLAGS} ${GO_TAGS} ./cmd/containerd-shim
-
-bin/containerd-shim-runc-v1: cmd/containerd-shim-runc-v1 FORCE # set !cgo and omit pie for a static shim build: https://github.com/golang/go/issues/17789#issuecomment-258542220
- @echo "$(WHALE) bin/containerd-shim-runc-v1"
- @CGO_ENABLED=${SHIM_CGO_ENABLED} go build ${GO_BUILD_FLAGS} -o bin/containerd-shim-runc-v1 ${SHIM_GO_LDFLAGS} ${GO_TAGS} ./cmd/containerd-shim-runc-v1
-
-bin/containerd-shim-runc-v2: cmd/containerd-shim-runc-v2 FORCE # set !cgo and omit pie for a static shim build: https://github.com/golang/go/issues/17789#issuecomment-258542220
- @echo "$(WHALE) bin/containerd-shim-runc-v2"
- @CGO_ENABLED=${SHIM_CGO_ENABLED} go build ${GO_BUILD_FLAGS} -o bin/containerd-shim-runc-v2 ${SHIM_GO_LDFLAGS} ${GO_TAGS} ./cmd/containerd-shim-runc-v2
+#bin/containerd-shim: cmd/containerd-shim FORCE # set !cgo and omit pie for a static shim build: https://github.com/golang/go/issues/17789#issuecomment-258542220
+# @echo "$(WHALE) bin/containerd-shim"
+# @CGO_ENABLED=${SHIM_CGO_ENABLED} go build ${GO_BUILD_FLAGS} -o bin/containerd-shim ${SHIM_GO_LDFLAGS} ${GO_TAGS} ./cmd/containerd-shim
+#
+#bin/containerd-shim-runc-v1: cmd/containerd-shim-runc-v1 FORCE # set !cgo and omit pie for a static shim build: https://github.com/golang/go/issues/17789#issuecomment-258542220
+# @echo "$(WHALE) bin/containerd-shim-runc-v1"
+# @CGO_ENABLED=${SHIM_CGO_ENABLED} go build ${GO_BUILD_FLAGS} -o bin/containerd-shim-runc-v1 ${SHIM_GO_LDFLAGS} ${GO_TAGS} ./cmd/containerd-shim-runc-v1
+#
+#bin/containerd-shim-runc-v2: cmd/containerd-shim-runc-v2 FORCE # set !cgo and omit pie for a static shim build: https://github.com/golang/go/issues/17789#issuecomment-258542220
+# @echo "$(WHALE) bin/containerd-shim-runc-v2"
+# @CGO_ENABLED=${SHIM_CGO_ENABLED} go build ${GO_BUILD_FLAGS} -o bin/containerd-shim-runc-v2 ${SHIM_GO_LDFLAGS} ${GO_TAGS} ./cmd/containerd-shim-runc-v2
binaries: $(BINARIES) ## build binaries
@echo "$(WHALE) $@"
@huXgebo
Copy link

huXgebo commented Mar 30, 2021

Hi,
this is good article!!

Unfortunately, he writes to me:

[FATAL] Your Linux kernel version 3.2.40 is not supported for running docker. Please upgrade your kernel to 3.10.0 or newer.

Did you come across this when you tried?

My NAS:
Processor: Marvell PJ4Bv7 Processor rev 1 (v7l)
Hardware: Marvell Armada-370

Now i see i get another such error when my containerd starts
failed to load cni during init, please check CRI plugin status before setting up network for pods error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config"

but containerd is started:
containerd successfully booted in 0.451770s
Start subscribing containerd event
Start event monitor
Start snapshots syncer
Start cni network conf syncer
Start streaming server

when i start the dockerd, docker say error:
[FATAL] Your Linux kernel version 3.2.40 is not supported for running docker. Please upgrade your kernel to 3.10.0 or newer.

Could I have ruined something? Can you help me?

Maybe I took down a bad armada system ... I'll try the 370 ... o ..

... oo. .. unfortunately, this did not solve either!

There was a mistake I don't know why.
The insmod not find aufs.ko,xt_conntrack.ko and xt_addrtype.ko, veth.ko.

insmod: ERROR: could not load module /lib/modules/aufs.ko: No such file or directory
insmod: ERROR: could not load module /lib/modules/xt_conntrack.ko: No such file or directory
insmod: ERROR: could not load module /lib/modules/xt_addrtype.ko: No such file or directory
insmod: ERROR: could not load module /lib/modules/veth.ko: No such file or directory

When should these files be generated?

I will attach the output to see if anyone can help.


INFO[2021-03-30T23:47:52.109520464+02:00] Starting up
INFO[2021-03-30T23:47:52.632116317+02:00] libcontainerd: started new containerd process  pid=8758
INFO[2021-03-30T23:47:52.632923087+02:00] parsed scheme: "unix"                         module=grpc
INFO[2021-03-30T23:47:52.641734300+02:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2021-03-30T23:47:52.642446393+02:00] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}  module=grpc
INFO[2021-03-30T23:47:52.642992605+02:00] ClientConn switching balancer to "pick_first"  module=grpc
WARN[2021-03-30T23:47:53.650307303+02:00] grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout". Reconnecting...  module=grpc
INFO[2021-03-30T23:47:54.971531686+02:00] starting containerd                           revision=a72fe7da21237815731386d6b73a0e93700112f9.m version=v1.5.0-beta.3-4-ga72fe7da2.m
WARN[2021-03-30T23:47:56.439540580+02:00] grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout". Reconnecting...  module=grpc
WARN[2021-03-30T23:48:01.023255008+02:00] grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout". Reconnecting...  module=grpc
INFO[2021-03-30T23:48:01.912087120+02:00] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
INFO[2021-03-30T23:48:02.013882013+02:00] loading plugin "io.containerd.snapshotter.v1.aufs"...  type=io.containerd.snapshotter.v1
WARN[2021-03-30T23:48:02.014656565+02:00] failed to load plugin io.containerd.snapshotter.v1.aufs  error="invalid aufs configuration"
INFO[2021-03-30T23:48:02.015137220+02:00] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
INFO[2021-03-30T23:48:02.024479571+02:00] skip loading plugin "io.containerd.snapshotter.v1.btrfs"...  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2021-03-30T23:48:02.025265761+02:00] loading plugin "io.containerd.snapshotter.v1.devmapper"...  type=io.containerd.snapshotter.v1
WARN[2021-03-30T23:48:02.031605596+02:00] failed to load plugin io.containerd.snapshotter.v1.devmapper  error="devmapper not configured"
INFO[2021-03-30T23:48:02.032188738+02:00] loading plugin "io.containerd.snapshotter.v1.native"...  type=io.containerd.snapshotter.v1
INFO[2021-03-30T23:48:02.047405582+02:00] loading plugin "io.containerd.snapshotter.v1.overlayfs"...  type=io.containerd.snapshotter.v1
INFO[2021-03-30T23:48:02.059152288+02:00] loading plugin "io.containerd.snapshotter.v1.zfs"...  type=io.containerd.snapshotter.v1
WARN[2021-03-30T23:48:02.059933958+02:00] failed to load plugin io.containerd.snapshotter.v1.zfs  error="invalid zfs configuration"
INFO[2021-03-30T23:48:02.067803883+02:00] loading plugin "io.containerd.metadata.v1.bolt"...  type=io.containerd.metadata.v1
WARN[2021-03-30T23:48:02.068518208+02:00] could not use snapshotter aufs in metadata plugin  error="invalid aufs configuration"
WARN[2021-03-30T23:48:02.068886213+02:00] could not use snapshotter devmapper in metadata plugin  error="devmapper not configured"
WARN[2021-03-30T23:48:02.069202133+02:00] could not use snapshotter zfs in metadata plugin  error="invalid zfs configuration"
INFO[2021-03-30T23:48:02.069525060+02:00] metadata content store policy set             policy=shared
INFO[2021-03-30T23:48:02.112659976+02:00] loading plugin "io.containerd.differ.v1.walking"...  type=io.containerd.differ.v1
INFO[2021-03-30T23:48:02.113276568+02:00] loading plugin "io.containerd.gc.v1.scheduler"...  type=io.containerd.gc.v1
INFO[2021-03-30T23:48:02.113961159+02:00] loading plugin "io.containerd.service.v1.introspection-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.114558776+02:00] loading plugin "io.containerd.service.v1.containers-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.143865800+02:00] loading plugin "io.containerd.service.v1.content-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.144525319+02:00] loading plugin "io.containerd.service.v1.diff-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.145027209+02:00] loading plugin "io.containerd.service.v1.images-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.145546311+02:00] loading plugin "io.containerd.service.v1.leases-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.179030617+02:00] loading plugin "io.containerd.service.v1.namespaces-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.179447980+02:00] loading plugin "io.containerd.service.v1.snapshots-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.179823450+02:00] loading plugin "io.containerd.runtime.v1.linux"...  type=io.containerd.runtime.v1
INFO[2021-03-30T23:48:02.180876884+02:00] loading plugin "io.containerd.runtime.v2.task"...  type=io.containerd.runtime.v2
INFO[2021-03-30T23:48:02.228085688+02:00] loading plugin "io.containerd.monitor.v1.cgroups"...  type=io.containerd.monitor.v1
INFO[2021-03-30T23:48:02.269044104+02:00] loading plugin "io.containerd.service.v1.tasks-service"...  type=io.containerd.service.v1
INFO[2021-03-30T23:48:02.343026069+02:00] loading plugin "io.containerd.internal.v1.restart"...  type=io.containerd.internal.v1
INFO[2021-03-30T23:48:02.343948378+02:00] loading plugin "io.containerd.grpc.v1.containers"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.344396783+02:00] loading plugin "io.containerd.grpc.v1.content"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.344900381+02:00] loading plugin "io.containerd.grpc.v1.diff"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.371966458+02:00] loading plugin "io.containerd.grpc.v1.events"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.372695234+02:00] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.373241039+02:00] loading plugin "io.containerd.grpc.v1.images"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.373776730+02:00] loading plugin "io.containerd.grpc.v1.leases"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.374375387+02:00] loading plugin "io.containerd.grpc.v1.namespaces"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.382385675+02:00] loading plugin "io.containerd.internal.v1.opt"...  type=io.containerd.internal.v1
INFO[2021-03-30T23:48:02.393957542+02:00] loading plugin "io.containerd.grpc.v1.snapshots"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.394648160+02:00] loading plugin "io.containerd.grpc.v1.tasks"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.395108446+02:00] loading plugin "io.containerd.grpc.v1.version"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.395809286+02:00] loading plugin "io.containerd.grpc.v1.introspection"...  type=io.containerd.grpc.v1
INFO[2021-03-30T23:48:02.411657337+02:00] serving...                                    address=/var/run/docker/containerd/containerd-debug.sock
INFO[2021-03-30T23:48:02.454085947+02:00] serving...                                    address=/var/run/docker/containerd/containerd.sock.ttrpc
INFO[2021-03-30T23:48:02.455554968+02:00] serving...                                    address=/var/run/docker/containerd/containerd.sock
INFO[2021-03-30T23:48:02.643361114+02:00] containerd successfully booted in 7.963726s
FATA[2021-03-30T23:48:03.880277284+02:00] Your Linux kernel version 3.2.40 is not supported for running docker. Please upgrade your kernel to 3.10.0 or newer.

Thank you for replay
gebo

@unknown321
Copy link
Author

Thank you for replay
gebo

What's the model of your device, DS114? Are you sure you have the latest DSM version? Perhaps upgrading will help.

@huXgebo
Copy link

huXgebo commented Mar 31, 2021

I Update the latest dsm version, but not solved my problem.

My device DS213j, maybe this is causing the problem..?

Thank,
gebo

@unknown321
Copy link
Author

I Update the latest dsm version, but not solved my problem.

According to https://www.synology.com/en-global/releaseNote/DSM?model=DS213j latest version is 6.2.4-25556.

Also show your kernel version, uname -a

@huXgebo
Copy link

huXgebo commented Mar 31, 2021

My DSM version 6.2.4-25556

uname -a
3.2.40 #25556 Thu Mar 4 17:56:47 CST 2021 armv7l GNU/Linux synology_armada370_213j

Thanks,
gebo

@unknown321
Copy link
Author

Well, you can try to compile a new kernel.

@huXgebo
Copy link

huXgebo commented Apr 2, 2021

It would exceed my knowledge of a kernel translation, I wanted to run a password store in a docker, I try to solve it differently.

Thanks for the help!
gebo

@spermikromik
Copy link

Hi, I have the same Synology DS218j run on DSM 7.1. Please could you provide your done install package for docker? Many thanks

@dark0dev
Copy link

dark0dev commented Feb 5, 2023

Hi, I have the same Synology DS218j run on DSM 7.1. Please could you provide your done install package for docker? Many thanks

Me, too! Would be great 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment