This document contains some notes I have gathered while I was trying to setup a redis service using boot2docker running on OS X. This won't cover what Docker is, see Docker website for details.
First, install Virtualbox and follow the steps at http://docs.docker.com/installation/mac/
Since Docker only runs on Linux, boot2docker runs a virtual machine in Virtualbox (or VMWare etc), however you can run the docker command on OS X command line as if it were running on OS X.
Next, run docker run dockerfile/redis
This will download dockerfile's version of redis (for some reason, I wasn't able to access data outside the container for the official redis repo, more details in the next sections) and run it in the foreground. Note that you cannot send an interrupt signal (Ctrl-C) to stop this.
One way to stop this is to open a new shell, run docker ps to verify that it
is running. You will see that docker has given the running instance
(container) a unique id (1st column) and a unique name (last column). You can
use any of these to kill the running container. docker kill 62ec63560d3b
where 62ec63560d3b is the ID reported in the first column will kill the
container. You can also abbreviate the id so docker kill 62e should work
too.
Run the container again, this time with -d switch so that it runs in
background. docker run -d dockerfile/redis docker ps should report that
it is running again. So far so good, but how do we access the redis instance
inside the container?
Running docker ps should show that the port 6379/tcp is not bound to any
port on the host of the container (in our case, the VM that boot2docker has
started for us.) This means that this port will also not be exposed outside
the VM, in the OS X system.
Again stop the docker instance (tip: docker stop $(docker ps -a -q) will
stop all containers) and re-run it, this time with -P which will publish all
the exposed ports on the host. So running docker run -P -d dockerfile/redis
will start the redis server, with its port exposed in the host of the
container, that is the VM.
Running docker ps should verify this, as this time, the ports section will
show the mapping: 0.0.0.0:49165->6379/tcp instead of just 6379/tcp. The IP
0.0.0.0 means that the host of the container, the VM will be listening on
all interfaces.
Now, back in OS X, let's try connecting to this Redis server running in the container. Before continuing, make sure that any natively running servers on OS X has been stopped, just to make sure that we are connecting to the redis inside the container.
Try connecting to the port outputted by docker ps, so issue: redis-cli -p 49165 where 49165 is the random port assigned due to usage of -P. You
will see a connection error because boot2docker does not map the port exposed
in the virtual machine on the OS X system.
To be able to connect to the redis server, you need to specify the IP of the virtual machine running inside virtualbox. This is a common mistake as boot2docker makes it seem as if docker is running inside OS X.
To get the ip of the virtual machine, issue: boot2docker ip This will report
192.168.59.103 for Virtualbox.
Therefore, to connect to redis instance running inside the container, issue:
redis-cli -h 192.168.59.103 -p 49165.
Remember that we have used -P to expose a random port on the outside. You can also use -p to manually expose the port, for example -p 10000:6379 will expose the port 10000 on the virtual machine so that redis-cli -h 192.168.59.103 -p 10000 should work.
Verify this by running:
docker run -p 10000:6379 -d dockerfile/redis
redis-cli -h 192.168.59.103 -p 10000
After exposing the port for external access, let's look at exposing the data
to the outside world. Volumes are used in docker for this, however the
container recipe has to be crafted to make use of volumes for the consumers of
the container to override the data directories. dockerfile/redis
(https://registry.hub.docker.com/u/dockerfile/redis/) exposes the data
directory of docker at /data, so running the following should create a
folder called /redis_data for storing redis dumps.
docker run -d -p 6379:6379 -v /redis_data:/data --name redis dockerfile/redis
One common mistake is to assume that this data directory will magically appear
in OS X. This currently is not the case, there is an ongoing effort to make
folder sharing as seamless as possible in the future, however, for now, this
folder will be inside the virtual machine, not in OS X. Verify this by ssh'ing
into the virtual machine using boot2docker ssh and ls /redis_data. The
folder should be there, but empty for now since we have not entered any data
to redis.
To verify that redis dump will be available inside the container, do the following:
Since we have exposed redis at port 6379, connect to redis instance just by specifying the ip of the VM:
redis-cli -h 192.168.59.103
Then issue the following:
sadd teams Brazil
sadd teams Germany
save
The last explicit save is important since redis will normally wait for some
time before persisting to disk. save will trigger the dump creation.
Now, ssh into the vm by running boot2docker ssh and issue ls /redis_data. dump.rdb should be available there.
Now that we have the data inside the virtualbox instance, there are several ways to access it. There is an ongoing work to make this seamless using fuse in the near future, see moby/moby#4023 The changes have been merged 5 days ago as of writing (Jun 1, 2014), so next versions of docker and boot2docker should make these much easier.
One way is to use virtualbox's native sharing capabilities by actually using a different boot2docker iso instead of the stock one. boot2docker project maintainers are currently opposed to this method since boot2docker is not specific to virtualbox and adding such functionality for each backend like vmware would bloat boot2docker.
The other method is to use samba or nfs sharing to expose the folder, which is pretty easy and does not require one to use a different boot2docker iso.
First, redis instance must be run with a name by specifying --name redis, we have already done so in the above examples:
docker run -d -p 6379:6379 -v /redis_data:/data --name redis dockerfile/redis
Next, issue docker run svendowideit/samba redis
This will prompt you to run (erronously)
docker run --rm -v $(which docker):/docker -v $(/var/run/docker.sock):/docker.sock -e DOCKER_HOST svendowideit/samba redis
Instead, issue
docker run --rm -v $(which docker):/docker -v /var/run/docker.sock:/docker.sock -e DOCKER_HOST svendowideit/samba redis
You should see the following output:
stopping and removing existing server
starting samba server container sharing redis:/data
# run 'docker logs samba-server' to view the samba logs
================================================
Your data volume (/data) should now be accessible at \\<docker ip>\ as 'guest' user (no password)
For example, on OSX, using a typical boot2docker vm:
goto Go|Connect to Server in Finder
enter 'cifs://192.168.59.103
hit the 'Connect' button
select the volumes you want to mount
choose the 'Guest' radiobox and connect
Or on Linux:
mount -t cifs //192.168.59.103/data /mnt/data -o username=guest
Or on Windows:
Enter '\\192.168.59.103\data' into Explorer
Log in as Guest - no password`
Follow the instructions there and you will now be able to access the data on OS X side. You can now access (for example backup) the redis dump folder.
What we want to achieve is to be able to access the dump after redis has exited. One interesting pattern in Docker community is to use data containers for that, containers which just wrap some data directories. These usually use busybox repo since they do not need to run anything.
After killing the redis process, issue docker run -v /redis_data:/data --name data_container busybox true. This will make sure that /redis_data folder,
where our dump lives will be seen as /data in this container.
Now re-run the samba sharing container with this data container:
docker run --rm -v $(which docker):/docker -v /var/run/docker.sock:/docker.sock -e DOCKER_HOST svendowideit/samba data_container
You should now be able to have access to the dump directory without redis running.
To verify that this is so, on OS X, start the local redis-server with this data directory:
Run redis-server -
Type DIR /Volumes/data followed by Ctrl-d
This will make sure that local redis on OS X will be using the dump we shared from docker.
Running simply redis-cli followed by smembers teams should output 1)Brazil and 2) Germany as expected.
Good god, finally got this working because of this. Thanks!!