tnk4on/podman-qemu-user-static.md

## podman-qemu-user-static.md

      
    Raw
  

              podman-qemu-user-static.md
            
          
    Podman on RHEL with multiarch/qemu-user-static

RHEL does not include qemu-user-static, consider alternatives.
Links


qemu-user-binfmt and qemu-user-static are not included in RHEL 8 - Red Hat Customer Portal
multiarch/qemu-user-static: /usr/bin/qemu-*-static

Environment(RHEL for Edge)

$ cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.1 (Plow)

$ uname -a
Linux edge 5.14.0-162.23.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 23 20:08:28 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

$ rpm-ostree status 
State: idle
Deployments:
● edge:rhel/9/x86_64/edge
                  Version: 9.1 (2023-04-20T01:41:02Z)
                   Commit: 5c6ea854b8e3a33a246c8f25cba3774c7c94bc8d6d0b18724fb1bf13dacf4df1

$ podman version
Client:       Podman Engine
Version:      4.2.0
API Version:  4.2.0
Go Version:   go1.18.9
Built:        Tue Feb  7 19:56:41 2023
OS/Arch:      linux/amd64


Before(don't work)

$ podman run --rm --arch arm64 ubi9/ubi uname -m
{"msg":"exec container process (missing dynamic library?) `/usr/bin/uname`: No such file or directory","level":"error","time":"2023-05-03T19:45:34.000483276Z"}

$ podman run --rm --arch arm64 --privileged ubi9/ubi uname -m
{"msg":"exec container process (missing dynamic library?) `/usr/bin/uname`: No such file or directory","level":"error","time":"2023-05-03T19:45:45.000351573Z"}

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
enabled
interpreter /usr/bin/qemu-aarch64-static
flags: 
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

Run multiarch/qemu-user-static

$ sudo podman run --rm --privileged multiarch/qemu-user-static --reset -p yes
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
Setting /usr/bin/qemu-hexagon-static as binfmt interpreter for hexagon

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
enabled
interpreter /usr/bin/qemu-aarch64-static
flags: F
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

After(rootless mode)

$ podman run --rm --arch arm64 --privileged ubi9/ubi uname -m
aarch64

but, it can't run without --privileged .
$ podman run --rm --arch arm64 ubi9/ubi uname -m
{"msg":"exec container process (missing dynamic library?) `/usr/bin/uname`: No such file or directory","level":"error","time":"2023-05-03T19:47:59.000475518Z"}

it also can't build.
$ podman build --rm -t test --platform linux/arm64 - <<EOF
FROM registry.access.redhat.com/ubi9/ubi
RUN dnf update -y
EOF
STEP 1/2: FROM registry.access.redhat.com/ubi9/ubi
STEP 2/2: RUN dnf update -y
container exited on segmentation fault
Error: error building at STEP "RUN dnf update -y": error while running runtime: exit status 1

$ podman build --rm -t test --cap-add ALL --platform linux/arm64 - <<EOF
FROM registry.access.redhat.com/ubi9/ubi
RUN dnf update -y
EOF
STEP 1/2: FROM registry.access.redhat.com/ubi9/ubi
STEP 2/2: RUN dnf update -y
container exited on segmentation fault
Error: error building at STEP "RUN dnf update -y": error while running runtime: exit status 1

After(root mode)

it need with --privileged .
$ sudo podman run --rm --arch arm64 --privileged ubi9/ubi uname -m
aarch64

it also can't build.
$ sudo podman build --rm -t test --platform linux/arm64 - <<EOF
FROM registry.access.redhat.com/ubi9/ubi
RUN dnf update -y
EOF
STEP 1/2: FROM registry.access.redhat.com/ubi9/ubi
STEP 2/2: RUN dnf update -y
container exited on segmentation fault
Error: error building at STEP "RUN dnf update -y": error while running runtime: exit status 1

$ sudo podman build --rm -t test --cap-add ALL --platform linux/arm64 - <<EOF
FROM registry.access.redhat.com/ubi9/ubi
RUN dnf update -y
EOF
STEP 1/2: FROM registry.access.redhat.com/ubi9/ubi
STEP 2/2: RUN dnf update -y
container exited on segmentation fault
Error: error building at STEP "RUN dnf update -y": error while running runtime: exit status 1

Reset

$ sudo podman run --rm --privileged multiarch/qemu-user-static --reset -p no

SElinux

$ sestatus 
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      33

$ podman run --rm --arch arm64 ubi9/ubi uname -m

$ sudo setenforce 0

$ podman run --rm --arch arm64 ubi9/ubi uname -m
aarch64


## qemu-user-static-on-arm64.md

      
    Raw
  

              qemu-user-static-on-arm64.md
            
          
    multiarch/qemu-user-static on arm64

Links


Need support for run x86_64 on linux/arm64 · Issue #174 · multiarch/qemu-user-static
Build Linux software for foreign CPU architectures on non-x86_64 | by Martin Grigorov | Medium
Building Linux packages for different CPU architectures with Docker and QEMU | by Martin Grigorov | Medium

Environment

# cat /etc/redhat-release
Red Hat Enterprise Linux release 9.2 Beta (Plow)

# uname -a
Linux rhel92 5.14.0-283.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Thu Feb 23 19:37:21 EST 2023 aarch64 aarch64 aarch64 GNU/Linux

# podman version
Client:       Podman Engine
Version:      4.4.0
API Version:  4.4.0
Go Version:   go1.19.4
Built:        Sat Feb  4 01:14:41 2023
OS/Arch:      linux/arm64

Before(not work)

# sudo podman run --rm --privileged multiarch/qemu-user-static --reset -p yes
✔ docker.io/multiarch/qemu-user-static:latest
Trying to pull docker.io/multiarch/qemu-user-static:latest...
Getting image source signatures
Copying blob 30c9c93f40b9 done
Copying blob 205dae5015e7 done
Copying blob 816739e52091 done
Copying blob 30abb83a18eb done
Copying blob 0657daef200b done
Copying config 3539aaa873 done
Writing manifest to image destination
Storing signatures
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)
{"msg":"exec container process `/register`: Exec format error","level":"error","time":"2023-05-06T15:33:43.244686Z"}

# sudo podman run --rm --privileged --arch arm64  multiarch/qemu-user-static --reset -p yes
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)
{"msg":"exec container process `/register`: Exec format error","level":"error","time":"2023-05-06T15:36:38.674713Z"}

Run aptman/qus

# podman run --rm --privileged aptman/qus -s -- -p
✔ docker.io/aptman/qus:latest
Trying to pull docker.io/aptman/qus:latest...
Getting image source signatures
Copying blob aa73ede2d57b done
Copying blob 24752f2a9b6d done
Copying blob b50100f25006 done
Copying blob 1bb83505e71f done
Copying blob b9922d81f8dd done
Copying config b82740c079 done
Writing manifest to image destination
Storing signatures
cat ./qemu-binfmt-conf.sh | sh -s -- --path=/qus/bin -p --suffix -static
Setting /qus/bin/qemu-i386-static as binfmt interpreter for i386
Setting /qus/bin/qemu-i386-static as binfmt interpreter for i486
Setting /qus/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /qus/bin/qemu-arm-static as binfmt interpreter for arm
Setting /qus/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /qus/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /qus/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /qus/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /qus/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /qus/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /qus/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /qus/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /qus/bin/qemu-mips-static as binfmt interpreter for mips
Setting /qus/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /qus/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /qus/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /qus/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /qus/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /qus/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /qus/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /qus/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /qus/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /qus/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /qus/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /qus/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /qus/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /qus/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /qus/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /qus/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /qus/bin/qemu-or1k-static as binfmt interpreter for or1k
Setting /qus/bin/qemu-x86_64-static as binfmt interpreter for x86_64
Setting /qus/bin/qemu-hexagon-static as binfmt interpreter for hexagon

# cat /proc/sys/fs/binfmt_misc/qemu-x86_64
enabled
interpreter /qus/bin/qemu-x86_64-static
flags: F
offset 0
magic 7f454c4602010100000000000000000002003e00
mask fffffffffffefe00fffffffffffffffffeffffff

## After(rootless mode)

$ podman run --rm --arch amd64 ubi9/ubi uname -m
x86_64

$ podman build --rm -t test --platform linux/amd64 - <<EOF
FROM registry.access.redhat.com/ubi9/ubi
RUN dnf update -y
EOF
STEP 1/2: FROM registry.access.redhat.com/ubi9/ubi
STEP 2/2: RUN dnf update -y
Updating Subscription Management repositories.
Unable to read consumer identity
Subscription Manager is operating in container mode.
Red Hat Universal Base Image 9 (RPMs) - BaseOS  361 kB/s | 576 kB     00:01
Red Hat Universal Base Image 9 (RPMs) - AppStre 1.1 MB/s | 1.8 MB     00:01
Red Hat Universal Base Image 9 (RPMs) - CodeRea  73 kB/s | 104 kB     00:01
Dependencies resolved.
Nothing to do.
Complete!
COMMIT test
--> 526befd3226
[Warning] one or more build args were not consumed: [TARGETARCH TARGETOS TARGETPLATFORM]
Successfully tagged localhost/test:latest
526befd3226a5f18a6909d9c91a43ef8a46dca0bc0eff85e6876c8767ef7c739

After(root mode)

# podman run --rm --arch amd64  ubi9/ubi uname -m
x86_64