Skip to content

Instantly share code, notes, and snippets.

@kekru
Last active March 25, 2024 19:03
Show Gist options
  • Save kekru/4e6d49b4290a4eebc7b597c07eaf61f2 to your computer and use it in GitHub Desktop.
Save kekru/4e6d49b4290a4eebc7b597c07eaf61f2 to your computer and use it in GitHub Desktop.
Connect to another host with your docker client, without modifying your local Docker installation

Run commands on remote Docker host

This is how to connect to another host with your docker client, without modifying your local Docker installation or when you don't have a local Docker installation.

Enable Docker Remote API

First be sure to enable the Docker Remote API on the remote host.

This can easily be done with a container.
For HTTP connection use jarkt/docker-remote-api.
For HTTPS connection use kekru/docker-remote-api-tls.

You can also configure the Docker engine to expose the remote API. Read Enable Docker Remote API with TLS client verification for more information.

Download docker client

If you don't have a local Docker installation, you need to download the docker client (= docker cli), which is a simple executable.
And then add it to your PATH variable.

Here are some ways how to get the executable.
You only need one of the steps for you OS, not all:

  • Linux:
    • Either: (Any Linux)
      Download tgz file from download.docker.com/linux/static and unzip it. You only need the docker file, which must be added to your PATH.
      Maybe this script helps downloading it.
      Or just run:
      curl https://download.docker.com/linux/static/stable/x86_64/docker-19.03.8.tgz | tar xvz --directory /tmp && mv -v /tmp/docker/docker /usr/local/bin/docker && chmod +x /usr/local/bin/docker && rm -rf /tmp/docker
    • Or: (Ubuntu/Debian)
      $ apt-get update
      $ apt-get install -y apt-transport-https ca-certificates curl gnupg-agent software-properties-common
      $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
      # Verify correct gpg key
      $ apt-key fingerprint 0EBFCD88
      $ add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
      $ apt-get update && apt-get install docker-ce-cli
      From Install using the repository, but I would not recommend it, because there are many steps and much to install in the first steps
    • Or: (Centos)
      $ yum install -y yum-utils
      $ yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
      $ yum install -y docker-ce-cli
  • MacOS:
  • Windows

See also the official installation site and Install Docker Engine from binaries

HTTPS connection configuration

Docker's Remote API client authentication works with certificates.
See Protect the Docker daemon socket or my Enable Docker Remote API with TLS client verification on how to create server and client certificates.

For the following examples copy ca.pem (CA certificate), cert.pem (client certificate) and key.pem (client's private key) in /home/me/docker-tls/ or C:\users\me\docker-tls\.

Connect to remote api

Now we will see some ways on how to connect to a docker remote api.

Set alias (Linux and Mac)

For HTTP connection set the following alias:

alias dockerx="docker -H=your-remote-server.org:2375"

For HTTPS connection set the following alias:

alias dockerx="docker \
  --tlsverify \
  -H=your-remote-server.org:2376 \
  --tlscacert=/home/me/docker-tls/ca.pem \
  --tlscert=/home/me/docker-tls/cert.pem \
  --tlskey=/home/me/docker-tls/key.pem"

Now you can run commands on the remote machine with dockerx instead of docker.

Example:

dockerx ps

Create .bat file (Windows)

Create a file dockerx.bat.
For HTTP connection the content of the bat file should be:

docker -H=your-remote-server.org:2375 %*

For HTTPS connection the content of the bat file should be:

docker ^
  --tlsverify ^
  -H=your-remote-server.org:2376 ^
  --tlscacert=C:\users\me\docker-tls\ca.pem ^
  --tlscert=C:\users\me\docker-tls\cert.pem ^
  --tlskey=C:\users\me\docker-tls\key.pem %*

(If this does not work remove the carets (^) and the line breaks)

Now you can run commands on the remote machine with dockerx.bat instead of docker.

Example:

dockerx.bat ps

Set env var (Linux, Mac, Windows)

You can set environment vars to define the docker remote api that should be connected to.

For HTTP connection

# Linux/Mac
export DOCKER_HOST="tcp://your-remote-server.org:2375"

# Windows Powershell
$env:DOCKER_HOST="tcp://your-remote-server.org:2375"

For HTTPS connection

# Linux/Mac
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://your-remote-server.org:2376"
export DOCKER_CERT_PATH="/home/me/docker-tls"

# Windows Powershell
$env:DOCKER_TLS_VERIFY="1"
$env:DOCKER_HOST="tcp://your-remote-server.org:2376"
$env:DOCKER_CERT_PATH="C:\users\me\docker-tls"

Be sure that your DOCKER_CERT_PATH directory contains the following files:

  • ca.pem (CA certificate)
  • cert.pem (client certificate)
  • key.pem (client's private key)

Now any docker command will run against the remote api

docker ps

Do switch back to local docker, unset the env vars:

# Linux/Mac
unset DOCKER_HOST
unset DOCKER_TLS_VERIFY
unset DOCKER_CERT_PATH

# Windows Powershell
Remove-Item env:DOCKER_HOST
Remove-Item env:DOCKER_TLS_VERIFY
Remove-Item env:DOCKER_CERT_PATH

Reuse SSH connection (Linux, Mac, Windows(?))

If you already added an SSH public key to your remote server, then you can use this ssh credentials for your docker connection, too. You don't need to configure the remote api on the server for this approach.
(Should work on Windows, but I did only test on Linux yet)

Set the env var to a ssh address:

# Linux/Mac
export DOCKER_HOST="ssh://username@your-remote-server.org"

# Windows Powershell
$env:DOCKER_HOST="ssh://username@your-remote-server.org"

Now any docker command will run against the remote api

docker ps

Do switch back to local docker, unset the env vars:

# Linux/Mac
unset DOCKER_HOST

# Windows Powershell
Remove-Item env:DOCKER_HOST

Docker Context (Linux, Mac, Windows)

Since Docker 19.03 there is the docker context command. You can define multiple remote servers and switch between them.

Create a context for HTTPS
(Change paths for Windows)

docker context create example-server \
  --description "connection to example server" \
  --docker "host=tcp://your-remote-server.org:2376, \
     ca=/home/me/docker-tls/ca.pem, \
     cert=/home/me/docker-tls/cert.pem, \
     key=/home/me/docker-tls/key.pem"

(For HTTP connection remove ca, cert and key and switch port to 2375. For SSH connection use ssh address)

Now you can call the remote server with:

docker --context example-server ps

Or choose the context and then all following command will call the remote server

docker context use example-server
docker ps
@BartRobeyns
Copy link

The Install Docker from binaries link no longer provides info for installing the Windows Docker Client by itself. Instead, you can go to https://download.docker.com/win/static/stable/x86_64/, and grab the latest zip-file that contains the Docker Client, docker.exe.

@kekru
Copy link
Author

kekru commented Feb 22, 2018

Thanks @BartRobeyns, I updated the text

@solvingj
Copy link

solvingj commented Mar 9, 2018

This is what happens when i run docker.bat ps:

U:\>docker.bat ps
U:\>docker -H=TEMP-HYPERV:2375 ps
U:\>docker -H=TEMP-HYPERV:2375 -H=TEMP-HYPERV:2375 ps
U:\>docker -H=TEMP-HYPERV:2375 -H=TEMP-HYPERV:2375 -H=TEMP-HYPERV:2375 ps
U:\>docker -H=TEMP-HYPERV:2375 -H=TEMP-HYPERV:2375 -H=TEMP-HYPERV:2375 -H=TEMP-HYPERV:2375 ps

and it goes on to infinity until i cancel.

Basically, it happens if you name the script docker.bat specifically, AND run it in the folder where it exists. Name it something else, or run it from another directory.

@kekru
Copy link
Author

kekru commented Mar 19, 2018

Haha thats crazy. Thanks @solvingj, I updated the text to use dockerx.bat instead

@wxgeorge
Copy link

You also could set DOCKER_HOST in your session ...

Copy link

ghost commented Jan 16, 2019

root@dfmix-easycards01-pltf:/etc/gitlab-runner# docker run -p 80:2375 -v /var/run/docker.sock:/var/run/docker.sock jarkt/docker-remote-api

Hangs and no port is opened

@kekru
Copy link
Author

kekru commented Jan 17, 2019

@fusionfai
Try run with the -d option to make it run in background:
docker run -d -p 80:2375 -v /var/run/docker.sock:/var/run/docker.sock jarkt/docker-remote-api
And make sure, you're connecting to port 80 instead of 2375 by changing the port in -H, when running a docker command

@koganzjo
Copy link

@fusionfai
if you set port 80 instead of 2375 in the dockerx.bat I think that all works

@omidkrad
Copy link

Thanks! This worked great, I only needed to use port 2375 instead of 80.
docker run -d --name docker-remote-api -p 2375:2375 -v /var/run/docker.sock:/var/run/docker.sock jarkt/docker-remote-api

@steven87vt
Copy link

steven87vt commented Mar 6, 2020

FYI this does not work on docker 19.03.5, build 633a0ea

docker context create example-server --description "connection to example server" --docker "host=tcp://your-remote-server.org:2376, ca=/home/me/docker-tls/ca.pem, cert=/home/me/docker-tls/cert.pem, key=/home/me/docker-tls/key.pem"
unable to create docker endpoint config:  cert: unrecognized config key
 key: unrecognized config key
 ca: unrecognized config key

@tnusraddinov
Copy link

tnusraddinov commented Jun 5, 2020

SSH:
For example if you are using EC2 AWS instance you will have aws.pem key file. Adding this file to your ssh agent like:
ssh-add -K ~/.ssh/aws.pem

and creating context
docker context create aws-server --description "my aws server" --docker "host=ssh://your-remote-server.com"

you will be able to connect to remote instance and see running (or other commands) containers:
docker --context=ssh-box ps

@Forinil
Copy link

Forinil commented Jun 30, 2020

@steven87vt It does work - there can be no whitespace characters in the string passed to --docker option.

@steven87vt
Copy link

@Forinil Thanks for the update, im sure it will help someone else. I ended up going with standard docker remote host commands.

@Forinil
Copy link

Forinil commented Jul 1, 2020

@steven87vt You're welcome. For future reference this file contains a list of all keys available for --docker option with explanations of what they mean: options.go

@shortcord
Copy link

Wasn't even aware of the SSH support for $DOCKER_HOST, can confirm that it works flawlessly on OSX 10.15.7.

@krystofwoldrich
Copy link

krystofwoldrich commented May 27, 2021

Hi, thanks for the helpful gist.

Has anything changed or I've missed something. But there is no need to create *.bat file on windows anymore. After setting $env:DOCKER_HOST everything work.

Can someone confirm? (I'm running windows in docker mcr.microsoft.com/windows/servercore:ltsc2019)

@Forinil
Copy link

Forinil commented May 27, 2021

Hi, thanks for the helpful gist.

Has anything changed or I've missed something. But there is no need to create *.bat file on windows anymore. After setting $env:DOCKER_HOST everything work.

Can someone confirm? (I'm running windows in docker mcr.microsoft.com/windows/servercore:ltsc2019)

There was never any need to create a *.bat file. The purposes of the bat files was to create a separate command for every host, eg. docker ps would show dockerized processes on localhost and dockerx ps would do the same, but for some remote server.

You can absolutely pass appropriate parameters using just environment variables or command line switches, although if you regularly work with multiple Docker daemons I would recommend setting up appropriate contexts as described at the very end of this gist.

@krystofwoldrich
Copy link

Hi, thanks for the helpful gist.
Has anything changed or I've missed something. But there is no need to create *.bat file on windows anymore. After setting $env:DOCKER_HOST everything work.
Can someone confirm? (I'm running windows in docker mcr.microsoft.com/windows/servercore:ltsc2019)

There was never any need to create a *.bat file. The purposes of the bat files was to create a separate command for every host, eg. docker ps would show dockerized processes on localhost and dockerx ps would do the same, but for some remote server.

You can absolutely pass appropriate parameters using just environment variables or command line switches, although if you regularly work with multiple Docker daemons I would recommend setting up appropriate contexts as described at the very end of this gist.

Ahhh, thank you for explanation. I’ve missed that.

I’m setting up Windows ServerCore docker image with Docker CLI for use in Gitlab CD/CI. So I think the env will be fine.

Copy link

ghost commented Sep 29, 2021

It would be nice if the SSH-approache is verified for windows.
Since Docker updated ther subscription this would be an accepteable drop-in-replacement for Docker-Desktop on Windows.

Sure, you will need some configuration for WSL2 and Docker-cli, but this would be better then pay for featueres you don't need.

@AlexOdobesteanu
Copy link

Hi, @kekru @kekru
I tried using the dockerx.bat script , but the output that i get is "error during connect: Get "https://remote-api.example.com:2376/v1.24/containers/json": dial tcp: lookup remote-api.example.com: no such host".I have set CERT_HOSTNAME=remote-api.example.com in my docker-compose.yml, before i used the command "docker compose up -d", but now i can't connect to it.So you have any idea what the problem could be?

@kekru
Copy link
Author

kekru commented Mar 12, 2022

@AlexOdobesteanu
Did you replace remote-api.example.com with your real domain name? This is only an example, which will not work.

You can try it out with a curl to the url:

$ curl https://remote-api.example.com:2376/v1.24/containers/json
curl: (6) Could not resolve host: remote-api.example.com

If you dont have a real domain name, you could use remote-api.127-0-0-1.nip.io (replace 127-0-0-1 with the IP address of the docker server).
See also https://nip.io

@AlexOdobesteanu
Copy link

@AlexOdobesteanu Did you replace remote-api.example.com with your real domain name? This is only an example, which will not work.

You can try it out with a curl to the url:

$ curl https://remote-api.example.com:2376/v1.24/containers/json
curl: (6) Could not resolve host: remote-api.example.com

If you dont have a real domain name, you could use remote-api.127-0-0-1.nip.io (replace 127-0-0-1 with the IP address of the docker server). See also https://nip.io

@kekru @kekru

I fixed the issue with remote-api.example.com.Now i have another problem.How do i get the ip of the docker server ? And there is also something i don't understand.How am i supposed to use CERT_HOSTNAME=remote-api.x.x.x.x.nip.io(ip address of the docker server) in docker-compose.yml , if i don't know the server ip address ? I think i'm missing something.
Thank you very very much !

@kekru
Copy link
Author

kekru commented Mar 14, 2022

@AlexOdobesteanu
You don't know your server's IP address? It's the address of the server, where docker is installed. If you use Docker-Desktop, you can use 127.0.0.1. If you are in another linux machine try running ifconfig or curl ifconfig.me or just google for how to find out your IP address

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment