Docker and Docker Compose on AMD OSX Hackintosh via Docker Machine

docker-compose-hackintosh.md

Introduction

Docker.app will complain about incompatible processor, so we will use Docker Machine.

Instalation

Download Docker for Mac (Docker.app). It contains some binaries that are necessary.

brew install virtualbox docker-machine

# Normally, those links are created automatically by running Docker.app,
# but it quits on us too early, so we need to do this manually
ln -s "/Applications/Docker.app/Contents//Resources/bin/docker-compose" /usr/local/bin/docker-compose
ln -s "/Applications/Docker.app/Contents//Resources/bin/docker-credential-desktop" /usr/local/bin/docker-credential-desktop
ln -s "/Applications/Docker.app/Contents//Resources/bin/docker-credential-osxkeychain" /usr/local/bin/docker-credential-osxkeychain

You can also use brew to install docker and doccker-compose and it should work without linking above.

Running

docker-machine create
eval $(docker-machine env)
docker run hello-world
docker-compose up

Hey @sergeycherepanov!
Yeah I tried rebooting a few times and didn't seem to solve the issue :(. Not sure why it happened...

Hey @adrienbaron, I checked it on the clean Ryzentosh setup and looks once manually run would be enough to be able approve permissions.

I added it to the readme.

Nice one! Thank you for taking the time to check that!

@sergeycherepanov i had done this in a more manual way previously - this is great, thank you!

i'm curious what the process is to allocate more storage to docker-machine. i was running out of disk space before for docker (not on the host - there's ~1.8TB available to the host). increasing the amount of ram is straight forward enough in virtual box, but i think i messed something up when trying to increase the amount of storage accessible before.

additionally, do you know how to bridge the networks so that localhost is available like it would be with just running docker on an intel mac?

@sergeycherepanov @slykar

Hi, first off thank you for taking the time to make running docker on AMD Hackintosh possible. I'd just like to ask if it is possible to have the docker app icon to show up on the macOS Menu bar (at the to next to ther WiFi/Bluetooth Icons and Time ) ?

I was able to get docker installed and running. But just wondering if these were possible. I'm fairly new to Docker.

Hi @shreyas1307

What features do you want from the icon? You need to understand this implementation based on the "Docker Machine", and some features available in the "Docker for mac" are not included.

Hi @j4ys0n
Currently, these limits depended on your machine size (25% of total memory) and cap hardcoded in the init script, need some effort to move it to a separate config file. You are able to edit it directly in the script or via Vbox interface. About the network, I used Gobetween to proxy ports fro the localhost. But currently, it works in a very simple way, only for TCP.

Keep in mind guys, that it's possible to use Vagrant to create a VM for use with Docker Machine generic driver. Then you can use Vagrant Manager to have an icon with status icon for running VMs.

@sergeycherepanov I keep running into these errors when trying to start a docker container, how should I check if everything is installed correctly?

00:00:22.887747 VMMDev: Heartbeat flatline timer set to trigger after 4 000 000 000 ns
00:00:22.887823 VMMDev: Guest Log: vgdrvHeartbeatInit: Setting up heartbeat to trigger every 2000 milliseconds
00:00:22.888007 VMMDev: Guest Log: vboxguest: misc device minor 57, IRQ 20, I/O port d020, MMIO at 00000000f0400000 (size 0x400000)
00:00:28.438310 NAT: IPv6 not supported
00:00:31.439179 NAT: DHCP offered IP address 10.0.2.15
00:00:31.439377 NAT: DHCP offered IP address 10.0.2.15
00:00:47.992780 VMMDev: Guest Log: VBoxService 5.2.34 r133893 (verbosity: 0) linux.amd64 (Oct 10 2019 20:19:38) release log
00:00:47.992807 VMMDev: Guest Log: 00:00:00.000056 main     Log opened 2020-06-18T12:46:11.093684000Z
00:00:47.993419 VMMDev: Guest Log: 00:00:00.000710 main     OS Product: Linux
00:00:47.993743 VMMDev: Guest Log: 00:00:00.001035 main     OS Release: 4.14.154-boot2docker
00:00:47.994061 VMMDev: Guest Log: 00:00:00.001355 main     OS Version: #1 SMP Thu Nov 14 19:19:08 UTC 2019
00:00:47.994396 VMMDev: Guest Log: 00:00:00.001689 main     Executable: /sbin/VBoxService
00:00:47.994408 VMMDev: Guest Log: 00:00:00.001689 main     Process ID: 2317
00:00:47.994415 VMMDev: Guest Log: 00:00:00.001690 main     Package type: LINUX_64BITS_GENERIC
00:00:47.996377 VMMDev: Guest Log: 00:00:00.003669 main     5.2.34 r133893 started. Verbose level = 0
00:00:47.998859 Guest Control: GUEST_MSG_REPORT_FEATURES: 0x1, 0x8000000000000000
00:00:48.003460 VMMDev: Guest Log: 00:00:00.010735 vminfo   rtldrNativeLoad: dlopen('libdbus-1.so.3', RTLD_NOW | RTLD_LOCAL) failed: libdbus-1.so.3: cannot open shared object file: No such file or directory
00:00:48.003533 VMMDev: Guest Log: 00:00:00.010824 vminfo   Error: Unable to connect to system D-Bus (1/3): D-Bus not installed
00:00:48.004286 VMMDev: Guest Additions capability report: (0x0 -> 0x0) seamless: no, hostWindowMapping: no, graphics: no
00:00:53.007235 VMMDev: Guest Log: 00:00:05.014507 vminfo   Error: Unable to connect to system D-Bus (2/3): D-Bus not installed
00:00:58.009439 VMMDev: Guest Log: 00:00:10.016711 vminfo   Error: Unable to connect to system D-Bus (3/3): D-Bus not installed

Hi @StefKors
Looks like an issue with your VirtualBox setup, you need to check the VM logs via UI and try a different version of VirtualBox.

https://stackoverflow.com/questions/46071615/error-unable-to-connect-to-system-d-bus-d-bus-not-installed

@sergeycherepanov
I'm on the latest virtualbox Version 6.1.10 r138449 (Qt5.6.3)

This is the error I get in the docker-virtualbox.log file, currently trying some different permissions, but also the syntax error looks suspicious

/usr/local/opt/docker-virtualbox/bin/docker-machine-init: line 51: [[: 1.36724e+06: syntax error: invalid arithmetic operator (error token is ".36724e+06")
error getting state for host docker: machine does not exist
error getting state for host docker: machine does not exist
===> Creating docker machine: docker
Docker machine "docker" already exists
eady exists
. Use "docker-machine create" to add a new one.
===> Creating docker machine: docker
Creating CA: /Users/stef/.docker/machine/certs/ca.pem
Creating client certificate: /Users/stef/.docker/machine/certs/cert.pem
Running pre-create checks...
(docker) Image cache directory does not exist, creating it at /Users/stef/.docker/machine/cache...
(docker) No default Boot2Docker ISO found locally, downloading the latest release...
(docker) Latest release for github.com/boot2docker/boot2docker is v19.03.5
(docker) Downloading /Users/stef/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v19.03.5/boot2docker.iso...
(docker) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%
Creating machine...
(docker) Copying /Users/stef/.docker/machine/cache/boot2docker.iso to /Users/stef/.docker/machine/machines/docker/boot2docker.iso...
(docker) Creating VirtualBox VM...
(docker) Creating SSH key...
(docker) Starting the VM...
(docker) Check network to re-create if needed...
(docker) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: /usr/local/opt/docker-machine/bin/docker-machine env docker
vm.max_map_count = 262144
�[1;34m[INFO] �[0mConfiguration:

	�[0;35m- Machine Name: docker �[0m
	�[0;35m- Shared Folder: /Users �[0m
	�[0;35m- Mount Options: noacl,async,nfsvers=3 �[0m
	�[0;35m- Force: false �[0m

�[1;34m[INFO] �[0mmachine presence ... 			�[0;32mOK �[0m
�[1;34m[INFO] �[0mmachine running ... 			�[0;32mOK �[0m
�[1;34m[INFO] �[0mLookup mandatory properties ... 
�[0;32mOK �[0m

	�[0;35m- Machine IP: 192.168.99.102 �[0m
	�[0;35m- Network ID: vboxnet0 �[0m
	�[0;35m- NFSHost IP: 192.168.99.1 �[0m

�[1;34m[INFO] �[0mConfigure NFS ... 
�[0;33m
 !!! Sudo will be necessary for editing /etc/exports !!! �[0m
tee: /etc/exports: Operation not permitted

Hi @StefKors

Ok, for now it looks like sudo permissions issue. Did you run docker-machine-init initialize and was approve permissions?

ps: I fixed the "line 51" issue, it wasn't critical for work

Hi everyone, thank you very much for your work since i am very dependent on Docker. Now after a successful installation i am facing an issue with docker-compose up -d - The download basically isn't continuing - googled some tips like setting max-concurrent-downloads 1 but had no luck - Any ideas on how to get this working? I can't start my projects :/

brew cask install virtualbox instead of brew install virtualbox

Docker was also missing from the path for me:

ln -s "/Applications/Docker.app/Contents//Resources/bin/docker /usr/local/bin/docker

Does anyone have connection issues between docker containers using docker-compose?

I tested running 2 alpine containers with docker run, and I was able to ping each other by container name. But when I try the following docker-compose file I get connection refused:

version: "3.8"
​
services:
​
  zookeeper:
    image: zookeeper:3.4.9
    restart: unless-stopped
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOO_MY_ID: 1
      ZOO_PORT: 2181
      ZOO_SERVERS: server.1=zookeeper:2888:3888
    volumes:
      - ./zookeeper/data:/data
      - ./zookeeper/datalog:/datalog
    networks:
      - my_network
​
  kafka:
    image: confluentinc/cp-kafka:5.1.0
    hostname: kafka
    container_name: kafka
    ports:
      - "9092:9092"
      - "29092:29092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      ALLOW_PLAINTEXT_LISTENER: "yes"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_LISTENERS: PLAINTEXT://:9092,PLAINTEXT_HOST://:29092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    volumes:
      - ./kafka/data:/var/lib/kafka/data
    depends_on:
      - zookeeper
    networks:
      - my_network
​
  kafka-connect:
    image: confluentinc/cp-kafka-connect:5.1.0
    hostname: kafka-connect
    container_name: kafka-connect
    ports:
      - "8083:8083"
    environment:
      CONNECT_BOOTSTRAP_SERVERS: "kafka:9092"
      CONNECT_REST_ADVERTISED_HOST_NAME: connect
      CONNECT_REST_PORT: 8083
      CONNECT_GROUP_ID: compose-connect-group
      CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs
      CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets
      CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: "1"
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: "1"
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: "1"
      CONNECT_PLUGIN_PATH: '/usr/share/java,/etc/kafka-connect/jars'
      CONNECT_CONFLUENT_TOPIC_REPLICATION_FACTOR: 1
    volumes:
      - ./kafka-connect/confluentinc-kafka-connect-mqtt-1.2.4/lib:/etc/kafka-connect/jars
    depends_on:
      - zookeeper
      - kafka
      - mosquitto
    networks:
      - my_network
​
  kafdrop:
    image: obsidiandynamics/kafdrop:3.27.0
    restart: "no"
    ports:
      - "9000:9000"
    environment:
      KAFKA_BROKERCONNECT: "kafka:9092"
      JVM_OPTS: "-Xms16M -Xmx48M -Xss180K -XX:-TieredCompilation -XX:+UseStringDeduplication -noverify"
    depends_on:
      - "kafka"
    networks:
      - my_network
​
networks: 
  my_network:
    driver: bridge

kafka            | [main-SendThread(zookeeper:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper/172.18.0.2:2181. Will not attempt to authenticate using SASL (unknown error)```

Hello guys,
docker working perfectly on my hackintosh. But i am facing one problem.
The com.apple.nfs prevents my hackintosh from going into sleepmode.
The 'pmset -g assertions' gives me:
pid 3439(nfsd): [0x0000013e000182e1] 00:03:30 PreventUserIdleSystemSleep named: "com.apple.nfsd"
Is there any workaround for that?
If i unload nfsd the container won't start, so thats not the solution.

None of this worked for me :(

Another approach would be to use dinghy with Virtualbox (https://github.com/codekitchen/dinghy) and create your dinghy docker-machine something like that:

$ dinghy create --provider virtualbox --disk 20000 --cpus 2 --memory 2048

Hi @chikko80
Yes, known issue, your machine can't go to sleep automatically if the NFS daemon runs

Hey @sergeycherepanov,
okay i unterstand.
But i am facing another problem with docker on virtualbox. If i start my docker-machine everything works fine - and if i use it consistently it also works fine.
But if its in idle mode for longer than 2-4 minutes, i cant reach the docker-machine and i get a timeout.
Its really annoying because i have to restart the docker-machine every time.

The output if i try to start a docker-compose with different containers:

ERROR: for mongo HTTPSConnectionPool(host='192.168.99.102', port=2376): Read timed out. (read timeout=60)

ERROR: for frontend  HTTPSConnectionPool(host='192.168.99.102', port=2376): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60)

With Verbose stuck on here:

compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=docker', 
'com.docker.compose.oneoff=False']})
urllib3.connectionpool._make_request: https://192.168.99.102:2376 "GET /v1.40/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Ddocker%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1" 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=docker', 'com.docker.compose.oneoff=False']})
urllib3.connectionpool._make_request: https://192.168.99.102:2376 "GET /v1.40/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Ddocker%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1" 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=docker', 'com.docker.compose.oneoff=False']})
urllib3.connectionpool._make_request: https://192.168.99.102:2376 "GET /v1.40/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Ddocker%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 5 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('6c9d4eba84ed6827d100a1454ea5a63229b4e0bb43cb365b59f5524208851f42')
ERROR: compose.cli.errors.log_timeout_error: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

hi @chikko80
Looks like a VM issue, you need to verify the vm directly in the vbox ui

if its responses to your actions just check the dmesg

@ALL after struggling with this approach and being unhappy i finally decided to install Ubuntu on my System AND then macOS via KVM. Works perfect now and with the help of OVMF you can passthrough your GPU if you need more graphical power!

I gave up too. Running Windows 10 with WSL2.

@mortensassi Yes, it's a possible workaround. Also, I recommend trying the Proxmox VE with the guides by Nick Sherlock. In this case, you will be able to install docker on the neighbor VM or directly on the host system.

I don't know why... But sometimes docker-machine get a disk corruption when building large images, even after recreate docker-machine the problem still there.
Soo now i use just a Ubuntu on VirtualBox and connect to him using DOCKER_HOST + ssh

Share the host ssh-key with VM, install docker, put user in docker group and add DOCKER_HOST in bash_profile

echo 'export DOCKER_HOST="ssh://user@host"' >> ~/.bash_profile

Since docker-machine is deprecated and archived, I recommend using minikube with virtualbox driver instead.

Installing minikube and docker:

brew install minikube docker

Creating the kubernets cluster using the virtualbox driver:

minikube start --driver=virtualbox --keep-context

Configures the Docker CLI in your current shell to use the minikube's docker:

eval $(minikube docker-env)

To use docker-compose just install it via homebrew:

brew install docker-compose

If you want to access a served application on a container (localhost:8080/route), you'll need the minikube IP to access it, using localhost won't work. And to retrieve the IP is just like that:

minikube ip

You can set this IP on /etc/hosts to access applications without typing the IP.

sudo echo "$(minikube ip) minikube" >> /etc/hosts

Now you can use docker without any problems on your Ryzentosh.

Since docker-machine is deprecated and archived, I recommend using minikube with virtualbox driver instead...

This worked perfectly for me. Thank you @jakoritarleite!!

thanks a lot @jakoritarleite

@jakoritarleite I am getting

docker-compose up
Error response from daemon: Client sent an HTTP request to an HTTPS server.

Any idea why?

Dude, THANK YOU! This solves the problem here!

I'm getting the error when running the minikube start command: