robinmonjo/cargo_how_to.md

## cargo_how_to.md

      
    Raw
  

              cargo_how_to.md
            
          
    #cargo, docker hub without docker, how to
18 Jan 2015
###Background
I have been using linux containers for almost 2 years now. It started during an internship at Applidget, a startup in Paris. My job was to integrate linux containers technology in their private PaaS.
At this time, there where no doubt, the technology to use was LXC. One week or so after I started digging into LXC, one of my co-worker talked to me about this new thing called docker. He told me "isn't it what you are supposed to do during your internship ?". And it was kind of it.
At this time I already highlighted a big challenge of containers in PaaS: creation time. Waiting for bundle install to complete is already long enough to have to wait for another 40 secondes to create a container :). Docker landed just in time !
Docker is open source, great ! It's written in Go, ok, I heard about it. So I started reading the source code. At this time, docker was using LXC as default container backend (today they use their own libcontainer, I will talk about it later). I quickly identified the use of AuFS, a union file system, and then understood how docker was able to spawn containers that fast. Obviously using docker wasn't an option for me, early versions wasn't production ready. I ended up writting some ruby and bash scripts to efficiently use LXC with AuFS, and Applidget PaaS is using it everyday since then.
This was old time ! Today docker is "everywhere": on aws,
google cloud,
azure
and in the future, even on Windows.
However, docker is not perfect for everyone and recently faced some criticism from core-os. I'm not here to argue if these critics are justified or not, but something is sure, docker is not the only container engine out there. However they are one big step ahead on root file systems and how they make them available.
###Docker images
Docker images are remotly stored on a registry. Once in a registry, they can be pulled and pushed. Docker Inc. maintains one registry, the docker hub. Go have a look, it's full of pre-configured, official and well maintained images. The thing is, these images can only be pulled and pushed by docker users. cargo is meant to bring the docker hub to other linux container users.
Docker images are not "just" linux root file system. Specification just got merged recently, you can go read it here. To summarize (roughly), images are made of ordered layers. You can read more about it here too, but you have the main idea. This layering approach make it very efficient to share and store images (pushing and pulling just the layer you need).
###Container engine
Linux containers are available in every linux distribution with a recent enough kernel. However, it's hard to setup a container manually. What I call container engines are tools that automates the setup of container. There are several of them (you probably heard about), the ones who come to mind are LXC, systemd-nspawn, docker (obviously), core-os rocket, and google lmctfy. But there is also a really minimal one, used to test libcontainer: nsinit. I mentioned libcontainer already. It's a Go package that replaced LXC as the default container backend in docker since version 0.9. I really like libcontainer, as it's dependency free and it gets a lot of support from  company such as Google and Redhat.
###Cargo
cargo is meant to provide docker hub capabilities to every container engines. In this how to, I will use it with nsinit on a ubuntu 14.04 machine (you will have to install golang since we have to build nsinit).
####1 - Setup
First setup a go workspace in whatever directory you want:
$>mkdir go
$>cd go
$>export GOPATH=`pwd`
Then we will have to get the libcontainer source code, and build nsinit:
$>go get github.com/docker/libcontainer

#heading to the nsinit main package and building the nsinit binary
$>cd $GOPATH/src/github.com/docker/libcontainer/nsinit/

#checkout to a version that I know will wotk with this tutorial
$>git checkout 73ba097bf596249068513559225d6e18c1767b47
$>GOPATH=`pwd`/../vendor:$GOPATH go build

#moving the binary into our path
$>sudo cp nsinit /usr/local/bin/
Now install cargo (check the readme for the latest version available):
$>curl -sL https://github.com/robinmonjo/cargo/releases/download/v1.4.1/cargo-v1.4.1_x86_64.tgz | sudo tar -C /usr/local/bin -zxf -
At this point you should have nsinit and cargo properly installed.
####2 - Pull an image
We are going to download the official debian jessie image into the debian directory:
$>sudo cargo pull debian:jessie -r debian -g

Pulling image library/debian:jessie ...
Image ID: 58052b122b60f9e695b9a1b0b8272bfb40e7249b9ba2d50ac22d12f3a3c9b4dd
Pulling 3 layers:
	511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
	bce696e097dc5286a7b9556a5f7420ff90ca4854b51e28256651d0071f56efac
	58052b122b60f9e695b9a1b0b8272bfb40e7249b9ba2d50ac22d12f3a3c9b4dd
	Done 58052b122b60f9e695b9a1b0b8272bfb40e7249b9ba2d50ac22d12f3a3c9b4dd
	Done 511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
	Done bce696e097dc5286a7b9556a5f7420ff90ca4854b51e28256651d0071f56efac
Downloading layers:
	511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158 ... done
	bce696e097dc5286a7b9556a5f7420ff90ca4854b51e28256651d0071f56efac ... done
	58052b122b60f9e695b9a1b0b8272bfb40e7249b9ba2d50ac22d12f3a3c9b4dd ... done
Done. Rootfs of library/debian:jessie in debian
As we can see, this image is made of 3 layers, let's go check the result:
$>cd debian
$>ls
bin  boot  dev  etc  home  json  layersize  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

$>git branch
  layer_0_511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
  layer_1_bce696e097dc5286a7b9556a5f7420ff90ca4854b51e28256651d0071f56efac
* layer_2_58052b122b60f9e695b9a1b0b8272bfb40e7249b9ba2d50ac22d12f3a3c9b4dd
So first, we can see that we have our debian file system. But why do we have a git repository ? Because we used cargo -g flag. Each branch is a layer of the image (remember, docker images are made of layers). layer_2_* contains the entire image since each layer is downloaded on a branch created from the previous one.
####3 - Run a container
Now that we have our file system, we want to run a container in it. nsinit need to find a container.json file at the root of the filesystem, containing the configuration of the container. There is a simple file that we can use in cargo repository:
$>sudo su
$>curl https://raw.githubusercontent.com/robinmonjo/cargo/master/sample_configs/container.json > container.json
We are now ready to spawn a new container running bash:
$> nsinit exec bash
root@cargo-demo:/#

#great we are in our container, curl is not installed, let's install it:
root@cargo-demo:/# apt-get update -qq
root@cargo-demo:/# apt-get install curl -y

[...]

#ok now we have curl !
root@cargo-demo:/# curl
curl: try 'curl --help' or 'curl --manual' for more information

#exit the container
root@cargo-demo:/# exit
We are back in our git repository. We can use git commands to check what changed (git status for example).
####4 - Push
Now we want to push this image on the docker hub. First, we need to commit it:
$>cargo commit -r debian -m "install curl"

Changes commited in layer_3_36a88d89412c6a7b67e87e1c16be1e21ef64548ce51ee6dab359e8d4026c2c0b
Image ID: c4f9b437f40b18c9603a38c866adcad7ea422aeb38b673ecfb669e9e5e82cbcc
Parent: 58052b122b60f9e695b9a1b0b8272bfb40e7249b9ba2d50ac22d12f3a3c9b4dd
Checksum: tarsum.dev+sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Layer size: 27188224
Done
Ok so what happended ? cargo took all the changes, and commited them into a new properly named branch. Remember, layers in docker images and how cargo save each one of them in a new branch ? That's what happened, we just created a new layer. cargo also wrote some metadata (image id, layer parent, layer checksum and size) that are needed to push and rebuild the image. Let's check the state of our git repository:
$>git branch
  layer_0_511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
  layer_1_bce696e097dc5286a7b9556a5f7420ff90ca4854b51e28256651d0071f56efac
  layer_2_58052b122b60f9e695b9a1b0b8272bfb40e7249b9ba2d50ac22d12f3a3c9b4dd
* layer_3_c4f9b437f40b18c9603a38c866adcad7ea422aeb38b673ecfb669e9e5e82cbcc
Before pushing our image we need a docker hub account.
#push the image. Replace username and password with your own
$>cargo push <username>/debian_curl -r debian_curl -u <username>:<password>

Pushing image robinmonjo/debian_curl:latest ...
Pushing 4 layers:
	511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158 ... done (already pushed)
	d0a18d3b84de6f3a3a9289997757bbfe96c01d5d84317227c9f4f00baf5646f8 ... done (already pushed)
	4d6ce913b130524c79fcf68ea639309bdc27400af16c1a1510c7aadf069c209c ... done (already pushed)
	c4f9b437f40b18c9603a38c866adcad7ea422aeb38b673ecfb669e9e5e82cbcc ... done
Done: https://registry.hub.docker.com/u/robinmonjo/debian_curl
Ok so our image has been pushed. The first 3 layers already existed on the docker hub (we pulled them from the debian jessie image earlier), so their data have not been re-uploaded (that what makes the image layering approach of docker really nice).
We can now go ahead, and delete our debian image, it's safely stored in the cloud. But just for this tutorial we will download it again:
#remove the image we just pushed
$>cd .. && sudo rm -rf debian

#download it again, just to make sure everything worked
#Note: you don't need to specify you credentials unless you made your image private
$>sudo cargo pull <username>/debian_curl -u <username>:<password>

Pulling image robinmonjo/debian_curl:latest ...
Image ID: c4f9b437f40b18c9603a38c866adcad7ea422aeb38b673ecfb669e9e5e82cbcc
Pulling 4 layers:
	511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
	d0a18d3b84de6f3a3a9289997757bbfe96c01d5d84317227c9f4f00baf5646f8
	4d6ce913b130524c79fcf68ea639309bdc27400af16c1a1510c7aadf069c209c
	c4f9b437f40b18c9603a38c866adcad7ea422aeb38b673ecfb669e9e5e82cbcc
	Done 511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
	Done 4d6ce913b130524c79fcf68ea639309bdc27400af16c1a1510c7aadf069c209c
	Done c4f9b437f40b18c9603a38c866adcad7ea422aeb38b673ecfb669e9e5e82cbcc
	Done d0a18d3b84de6f3a3a9289997757bbfe96c01d5d84317227c9f4f00baf5646f8
Downloading layers:
	511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158 ... done
	d0a18d3b84de6f3a3a9289997757bbfe96c01d5d84317227c9f4f00baf5646f8 ... done
	4d6ce913b130524c79fcf68ea639309bdc27400af16c1a1510c7aadf069c209c ... done
	c4f9b437f40b18c9603a38c866adcad7ea422aeb38b673ecfb669e9e5e82cbcc ... done
Done. Rootfs of robinmonjo/debian_curl:latest in rootfs
So we pulled 4 layers, that makes sense. Do we have curl installed in this image ?
$>cd rootfs  #root file system is in rootfs, we didn't set the -r flag

#we can use nsinit directly, our previous configuration container.json was pushed earlier
$>sudo nsinit exec bash

root@cargo-demo:/# curl --version
curl 7.26.0 (x86_64-pc-linux-gnu) libcurl/7.26.0 OpenSSL/1.0.1e zlib/1.2.7 libidn/1.25 libssh2/1.4.2 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtmp rtsp scp sftp smtp smtps telnet tftp 
Features: Debug GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP 
Yes curl is installed ! You will notice that the filesystem wasn't downloaded with the -g flag, so it's not a git repository (downside, you can't push it)
We just used cargo and the docker hub to pull, commit and push a debian jessie file system. I choosed to use nsinit here, but you can do it with whatever container engine you like (with of course some specific configuration).
So far, cargo only supports the docker hub, but during the development process I easily got it working on a private registry I deployed for debugging purpose. This could be added in a near future.
###Conclusion
I think cargo is a cool tool, and hopefully it will be useful fore some people out there. Go check the repo here. If you have any issue using it, open an issue on Github, if you feel like you want to contribute, open a pull request, and if you have any comments, leave a comment below.
Robin.