Podman with docker-compose on MacOS.

podman_macos.md

Podman with docker-compose on MacOS.

Podman an alternative to Docker Desktop on MacOS

Getting podman installed and started is super easy.
Just use brew to install it.

> brew install podman

Now since podman uses a VM just like the Docker Client on MacOS we need to initialize that and start it.

> podman machine init
> podman machine start

Now we are set to go.

If you want you can create a symlink so podman can be executed with "docker" command.

> ln -s /usr/local/bin/podman /usr/local/bin/docker

Now most of the commands in podman are the same so try podman images and you will get a list of images.
Else the podman --help command list all the help you need.

To get docker-compose without the docker client for mac. You can install it using the brew command.

> brew install docker-compose

When that is done you now should have the ability to use docker-compose with podman.

On MacOS the podman project does not expose the podman.socket which is similar to docker.socket, by default. So to get docker-compose working one needs to expose the socket.

To get the socket running run the following commands.
First we need to find the port it is exposed on in the VM.

> podman system connection ls

Then we need to take that port and create a forward ssh connection to that.

> ssh -fnNT -L/tmp/podman.sock:/run/user/1000/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:<port to socket> -o StreamLocalBindUnlink=yes
> export DOCKER_HOST='unix:///tmp/podman.sock'

Second, we expose the DOCKER_HOST env variable that is used by docker-compose.

Be aware that if the connection is disconnected one needs to delete/overwrite the /tmp/podman.socket to run the forward command.

Overall findings is that if one only runs single images then it is fairly easy to get going using podman. But if you rely on the compose part to orchestrate the containers in a bigger setup of different images with networking etc. then podman is a lot less easy to get working "out of the box". There is a lot of googling involved and then it still seems that there are a lot of the features that are not too easy to get working. I did have a lot of issues getting the right permissions to mount drives into the images. One of the main features with podman is that it is rootless. Which is great but it means that you need to understand what permissions a container needs before it fully works. I have tried to use the podman-compose as the goto instead of docker-compose, but I had a hard time even getting it installed, and there were alot of issues where it could not load images from the local repository, so in the end that is why I decided to use docker-compose and not podman-compose. Another thing is that podman-compose is also developed by people not really part of the podman community it seems, or it is not set to be the frist choice by the podman community. So it seems that it is a project that has its own agenda, and is run by a few people and not as many as the podman community. For now I got it working but I will say that there are many wheels that need tuning and kept updated to have the setup running in a daily development environment. So if you, like me, just want to use the tools and not need to finetune all the time, it seems a little like there is a way to go before podman takes over the MacOS setup. Next for me is to try to setup everything on my linux laptop and see if this works easier out of the box.

I would like to add that if you come from docker desktop, you may see this problem:

error getting credentials - err: exec: "docker-credential-desktop": executable file not found in $PATH, out: ``

To solve this edit ~/.docker/config.json and remove "credsStore" : "desktop", line.

Reference to the original solution.

@alexrusenhack-loft, you are welcome and great comment. Thanks.

Thank you @kaaquist !

Here,

Be aware that if the connection is disconnected one needs to delete/overwrite the /tmp/podman.socket to run the forward command.

I had trouble to overwrite the tunnel, having the following message:

unix_listener: cannot bind to path /tmp/podman.sock: Address already in use
Could not request local forwarding.

But I found this option flag -o StreamLocalBindUnlink=yes enables the overwrite:

ssh -fnNT -L/tmp/podman.sock:/run/user/1000/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:<port to socket> -o StreamLocalBindUnlink=yes

• The explanation for the flag

Thanks for this, I did get this running but it doesn't seem to handle volumes correctly. You said you googled a lot, but did you figure it out for instance the simple docker-compose.yml:

main:
   image: alpine
   volumes:
        - ./data:/data

If I connect docker compose to the the podman socket as explained above then when I do a docker compose -f docker-compose.yml I get this error:

Error response from daemon: make cli opts(): error making volume mountpoint for
volume /User/rich/wsn/git/src/extern/netdrones-ip: mkdir /User: operation not permitted

nice sharing!
I am getting Server did not provide an image ID. Cannot write /var/folders/vq/.... error on docker-compose build. Anyone else getting the same?

Solution:
Basically two things. First of all, I'd recommend creating your VM w/ a more realistic set of resources, for me it was
podman machine init --cpus 4 --disk-size 50 --memory 4096
Second, COMPOSE_DOCKER_CLI_BUILD=0 solved the problem. There is an ongoing issue that is solved using this flag

@richtong, I did not elaborate too much about the volume mount and the "rootless" setup podman provides. But I found this here explanation that might help you on your way:
https://blog.christophersmart.com/2021/01/31/volumes-and-rootless-podman/
https://blog.christophersmart.com/2021/01/26/user-ids-and-rootless-containers-with-podman/

@luizanao sorry, I did not encounter that error. If you find a fix please report back 😄

found and commented in the same post @kaaquist ;)
I am running rootless btw. Selecting the root user in the connections list did the trick. Bear in your mind if you need to expose lower ports (such 443 or 80) you gonna need run rootless since it needs special permissions from the host OS

@luizanao Thanks a million.

Trying to spin up docker-compose from https://github.com/docker/awesome-compose/tree/master/react-express-mongodb

$ docker-compose up
[+] Running 3/3
 ⠿ Network react-express-mongodb_express-mongo  Created                                                                                                                                                                        0.0s
 ⠿ Network react-express-mongodb_react-express  Created                                                                                                                                                                        0.0s
 ⠿ Container mongo                              Created                                                                                                                                                                        0.0s
 ⠋ Container backend                            Creating                                                                                                                                                                       0.0s
Error response from daemon: fill out specgen: must set source volume

Seem the migration from docker to podman is a painful.

@quangthe good that you are trying 😀.
I don't know the project you are linking to, and I only briefly looked at the docker-compose file in it.
But from what I have experienced so far, is that podman is pretty strict on its syntax.
When looking at the docker-compose file from the project you linked, It can be that you will be able to add a few extra info lines to the docker-compose file, like what version it is. And also when one is using the Volume mount point it normally (or that is what I have encountered so fare) looks as follow:

version: "3"
services:
  db:
    image: postgres
    volumes:
      - data:/var/lib/postgresql/data
volumes:
  data:
    driver: mydriver

As you can see here, the volume is both referenced under each of the services and as a source volume later in the descriptor. So maybe that is what is missing, and docker with docker-composeis not as strict in the syntax.
I hope you get it working.

@kaaquist Thanks for your help. With docker we can mount current ./data directory (which may contains project configs, data...) into container. However, with podman, afaik it's not supported yet. Hope podman can support it soon :)

If you want to mount local directory then this here command would work

> podman run -it --rm --privileged --mount type=bind,source=/home/$USER/localFolder,target=/remoteFolder docker.io/debian:bullseye /bin/bash

But as you can see it uses --mount and also --privileged which does the following

--privileged=true|false

Give extended privileges to this container. The default is false.

By default, Podman containers are unprivileged (=false) and cannot, for example, modify parts of the operating system. This is because by default a container is only allowed limited access to devices. A "privileged" container is given the same access to devices as the user launching the container.

A privileged container turns off the security features that isolate the container from the host. Dropped Capabilities, limited devices, read-only mount points, Apparmor/SELinux separation, and Seccomp filters are all disabled.
Rootless containers cannot have more privileges than the account that launched them.

Source - Link

Thanks for sharing!

Contributing my two bytes regarding the following section:

If you want you can create a symlink so podman can be executed with "docker" command.

ln -s /usr/local/bin/podman /usr/local/bin/docker

So for those who install podman using Homebrew on M1 Macs, you may need to change /usr/local/bin to /opt/homebrew/bin. You may first check the location where podman is installed (using which podman) and replace the paths in the command with correct paths.

For example:

> ln -s /opt/homebrew/bin/podman /opt/homebrew/bin/docker

I solved it with

podman machine stop podman-machine-default || true
podman machine rm podman-machine-default --force
podman machine init -v ${HOME}:${HOME}
podman machine start

After that it just works, e.g.

services:
  app:
    image: python:alpine
    working_dir: /app/
    volumes: [.:/app/]

I tried with Podman v4.0. No more need to create a forward ssh connection.

nice sharing! I am getting Server did not provide an image ID. Cannot write /var/folders/vq/.... error on docker-compose build. Anyone else getting the same?

Solution: Basically two things. First of all, I'd recommend creating your VM w/ a more realistic set of resources, for me it was podman machine init --cpus 4 --disk-size 50 --memory 4096 Second, COMPOSE_DOCKER_CLI_BUILD=0 solved the problem. There is an ongoing issue that is solved using this flag

This is awesome! It fixed my issue. Thank you

Thank you for this. Tremendously helpful!

One note on the ssh command however. The user id in the for the socket won't always be 1000. In my case it was 501. It could be useful to add a placeholder to ensure folks enter the appropriate value when running the command.

> ssh -fnNT -L/tmp/podman.sock:/run/user/<userid>/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:<port to socket> -o StreamLocalBindUnlink=yes

I tried with Podman v4.0. No more need to create a forward ssh connection.

What do you mean? I'm on Podman 4.1.1 and get "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?" Can you describe the steps to make this work without SSH forwarding? My steps:

... (install Podman & docker-compose)
podman machine init
podman machine start

ln -s /opt/homebrew/bin/podman /usr/local/bin/docker

docker-compose up

Thanks for this, I did get this running but it doesn't seem to handle volumes correctly. You said you googled a lot, but did you figure it out for instance the simple docker-compose.yml:

main:
   image: alpine
   volumes:
        - ./data:/data

If I connect docker compose to the the podman socket as explained above then when I do a docker compose -f docker-compose.yml I get this error:

Error response from daemon: make cli opts(): error making volume mountpoint for
volume /User/rich/wsn/git/src/extern/netdrones-ip: mkdir /User: operation not permitted

@richtong I get the same thing, did you figure this out?

Works for me, I translate this gist and try to create a simple script

https://www.israel-perales.com/podman-con-docker-compose-en-macos/

brew install podman

podman machine init

podman machine start

ln -s /usr/local/bin/podman /usr/local/bin/docker

brew install docker-compose

sshPort=$(podman system connection ls --format="{{ .URI }}"| head -1 | awk -F'[^0-9]+' '{ print $2 }')

ssh -fnNT -L/tmp/podman.sock:/run/user/1000/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:$sshPort -o StreamLocalBindUnlink=yes

export DOCKER_HOST='unix:///tmp/podman.sock'

I was having the same problem as @trymzet and eventually realized it was due to not having installed the podman-mac-helper.

In other words, the following workflow worked for me (podman 4.2.1, docker-compose 2.11.1):

brew install podman
podman machine init
podman machine start
sudo /usr/local/bin/podman-mac-helper install # 👈
brew install docker-compose

docker-compose in a new shell/terminal worked fine after that.

@clintharris thanks for sharing 🙏 .. hope this will help you @trymzet 👍

Wow great find. Me and my company are actually using Rancher Desktop now because of these issues but I'm sure it will help others!

$ brew install podman docker-compose
$ podman machine init
$ podman machine set --rootful
$ podman machine start

Now, You can use docker-compose for pleasure

Great info man!

Thank you for sharing. I am getting below error, how do I solve ?

Attaching to dashboards-grafana-1
Error response from daemon: crun: write to `/proc/self/oom_score_adj`: Permission denied: OCI permission denied
Error: executing /usr/local/bin/docker-compose up grafana: exit status 1

I did these steps and trying to run grafana using docker compose

brew install podman
podman machine init # download linux vm
podman machine start #start vm


ln -s /usr/local/bin/podman /usr/local/bin/docker
brew install docker-compose #install docker compose
export podman_sock_port=`podman system connection ls | cut -d ":" -f 3 | cut -d "/" -f 1 | grep "[0-9]" | uniq` #get podman.socket
ssh -fnNT -L/tmp/podman.sock:/run/user/1000/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:${podman_sock_port} -o StreamLocalBindUnlink=yes #expose podman.socket
export DOCKER_HOST='unix:///tmp/podman.sock'
mv ~/.docker/config.json ~/.docker/config.json.old; cat ~/.docker/config.json.old | grep -v "\s*\"credsStore\"*" > ~/.docker/config.json 

@richtong
about the error Error response from daemon: make cli opts(): making volume mountpoint
use following command may help you:

podman machine init
podman machine set --rootful
podman machine start

I still get the same error (no permissions on mounted volume), setting rootful or adding userns_mode: "keep-id" doesn't help.

您的邮件我已经收到啦~我会尽快回复您的！ by何劲和

Please keep the conversation in English. Thanks.

Error response from daemon: make cli opts(): error making volume mountpoint for

This error can be fixed by starting podman machine with the --rootful flag:
podman machine set --rootful