Skip to content

Instantly share code, notes, and snippets.

@ubergarm
Last active January 11, 2016 16:17
Show Gist options
  • Star 15 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save ubergarm/ed42ebbea293350c30a6 to your computer and use it in GitHub Desktop.
Save ubergarm/ed42ebbea293350c30a6 to your computer and use it in GitHub Desktop.
How to compile nsinit for Docker to access a shell in a running container.

Docker nsinit

nsinit provides a handy way to access a shell inside a running container's namespace. This is useful for learning about how containers work, debugging your system without worrying about sshd daemons, and even hot fixes in production all you sad pandas!

:p

Running the docker daemon with the lxc driver allows you to use lxc-attach to do this. But now that docker deafults to the new native libcontainer driver, nsinit is probably the best way to go. jpetazzo's blog has a great high level summary.

The new issue is that the libcontainer .json format is under heavy development so you need to keep an nsinit binary built from the exact release tag from the the docker github repo. The tricky part for me was understanding that golang is designed with certain assumptions about dependencies vs versioning with regards to when you push to master, push to a topic branch, and fork a project.

Long story short, I just modified crosbymichael's Dockerfile for skydock to build nsinit in a container and then copy it out to my Linux host system. Here is the procedure:

Clone

Get yourself the docker repo and select the tag that matches the daemon version installed on your host system.

$ cd ~/projects
$ git clone https://github.com/dotcloud/docker.git
$ cd docker
$ git checkout -b v1.0.0 tags/v1.0.0 # create a local branch from the tagged version

Create Dockerfile

Put this Dockerfile in the nsinit directory.

$ cd pkg/libcontainer/nsinit
$ vi Dockerfile

Dockerfile:

FROM crosbymichael/golang

# setup build environment
RUN apt-get update && \
    apt-get install -y \
    build-essential  

# install just the deps
RUN go get -d github.com/dotcloud/docker/pkg/libcontainer/nsinit

# stuff in the specifically tagged code from the current directory 
ADD . /go/src/github.com/dotcloud/docker/pkg/libcontainer/nsinit

# build a binary image
RUN cd /go/src/github.com/dotcloud/docker/pkg/libcontainer/nsinit && \
    go install . ./...

Build and Run

$ docker build -t ubergarm/nsinit .
$ docker run --name nsinit -d ubergarm/nsinit
$ docker cp nsinit:/go/bin/nsinit ./outdir/                                 
$ docker rm nsinit
$ docker rmi ubergarm/nsinit

Test

Confirm you can run the binary on your host system by checking with ldd ./outdir/nsinit.new and ./outdir/nsinit.new.

Install

Do this however you'd like. Something like this should be close enough.

$ sudo chmod 755 ./outdir/nsinit
$ sudo chown root:root ./outdir/nsinit
$ sudo mv ./outdir/nsinit /usr/local/bin/nsinit-1.0.0
$ sudo ln -s /usr/local/bin/nsinit-1.0.0 /usr/local/bin/nsinit

Nifty Script

Since it is a pain to manually look up the container hash and find the directory I put together a script:

#!/bin/bash
[[ -n "$1" ]] || { echo "Usage: `basename $0` <partial container ID or name>"; exit 0 ; }
FULL_ID=$(docker inspect --format='{{.Id}}' $1)

# LXC driver method:
# sudo lxc-attach -n $FULL_ID /bin/bash 

# libcontainer driver method:
sudo bash -c "cd /var/lib/docker/execdriver/native/$FULL_ID && nsinit exec bash"

Done

Now hop in your favorite running container and have some fun!

@defender
Copy link

Hi

This the error I got when I run docker build -t ubergarm/nsinit .

Step 10 : RUN cd /go/src/github.com/dotcloud/docker/pkg/libcontainer/nsinit && go install . ./...
---> Running in fe1a70e0dbdd
found packages nsinit (create.go) and main (main.go) in .
can't load package: package github.com/dotcloud/docker/pkg/libcontainer/nsinit: found packages nsinit (create.go) and main (main.go) in /go/src/github.com/dotcloud/docker/pkg/libcontainer/nsinit
package github.com/dotcloud/docker/pkg/libcontainer/nsinit: found packages nsinit (create.go) and main (main.go) in /go/src/github.com/dotcloud/docker/pkg/libcontainer/nsinit

Please assist I need nsinit urgently :)

@defender
Copy link

Hi
I Just overcome the problem but I have another one : nsinit centos version `GLIBC_2.14' not found
I try to run nsinit on Centos that came by default with GLIBC_2.12 , I did not found any trivial way to update GLIBC to required version. Can you please try to provide nsinit compilation with lower version of GLIBC that can match my OS

Thanks a lot !!!

@ubergarm
Copy link
Author

Hello defender, I just made some updates to reflect changes in Docker Engine v1.0.0 (especially the CamelCase in json e.g. ID went to Id).

As I run mostly on Debian/Ubuntu hosts the crosbymichael/golang base image suits my needs.

To fix your problem you need to build nsinit within the context of a CentOS image with all of the go environment installed and configured. I'd say start with "FROM centos" and add in the yum version of the commands for everything from here: https://github.com/crosbymichael/golang-docker/blob/master/Dockerfile

You should be close! Good Luck!

@defender
Copy link

Hi ubergarm
Thank you for your quick response, im trying to do exactly what you suggested, but have another problem
at the compile time a got :

github.com/dotcloud/docker/pkg/libcontainer/namespaces

/tmp/go-build100046430/github.com/dotcloud/docker/pkg/libcontainer/namespaces/_obj/nsenter.cgo2.o: In function nsenter': ../namespaces/nsenter.go:106: undefined reference tosetns'
collect2: ld returned 1 exit status

I'm trying to update kernel-headers-2.6.32-431.17.1.el6.centos.plus.x86_64.rpm because as I read they provide support for setns but with no success.

I will be really appreciate if can look on usage and compilation of nsinit on Centos

Thanks.

@zcox
Copy link

zcox commented Jun 13, 2014

I'm seeing this error trying to build the image using the Dockerfile:

$ sudo docker build -t ubergarm/nsinit .
Uploading context 14.85 kB
Uploading context
Step 0 : FROM crosbymichael/golang
 ---> f3d34753217d
Step 1 : RUN apt-get update &&    apt-get install -y    build-essential
 ---> Using cache
 ---> bfd0b68419c1
Step 2 : RUN go get -d github.com/dotcloud/docker/pkg/libcontainer/nsinit
 ---> Running in 130059ccb1a3
package github.com/dotcloud/docker/pkg/libcontainer/nsinit
    imports github.com/dotcloud/docker/pkg/libcontainer/nsinit
    imports github.com/dotcloud/docker/pkg/libcontainer/nsinit: cannot find package "github.com/dotcloud/docker/pkg/libcontainer/nsinit" in any of:
    /usr/local/go/src/pkg/github.com/dotcloud/docker/pkg/libcontainer/nsinit (from $GOROOT)
    /go/src/github.com/dotcloud/docker/pkg/libcontainer/nsinit (from $GOPATH)
2014/06/13 19:58:32 The command [/bin/sh -c go get -d github.com/dotcloud/docker/pkg/libcontainer/nsinit] returned a non-zero code: 1

@defender
Copy link

Hi if you want to compile nsinit from 1.0.0
Please change your Dockerfile, since the location of nsinit was changed

FROM crosbymichael/golang

# setup build environment
RUN apt-get update && \
    apt-get install -y \
    build-essential

# install just the deps
RUN go get -d github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit

# stuff in the specifically tagged code from the current directory
ADD . /go/src/github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit

# build a binary image
RUN cd /go/src/github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit && \
    go install . ./...

@johnjelinek
Copy link

I get this when trying to build from v1.0.1 tag:

Step 4 : RUN cd /go/src/github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit &&    go install . ./...
 ---> Running in b7c2afe23d12
# github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit
./exec.go:34: cannot use nspid (type int) as type *libcontainer.State in function argument
./exec.go:51: undefined: libcontainer.Container
./exec.go:59: undefined: libcontainer.Container
./spec.go:33: undefined: libcontainer.Container
./stats.go:34: undefined: libcontainer.Container
./utils.go:14: undefined: libcontainer.Container
./utils.go:21: undefined: libcontainer.Container
./utils.go:54: undefined: libcontainer.Container
./utils.go:55: undefined: libcontainer.Container
2014/06/28 01:54:13 The command [/bin/sh -c cd /go/src/github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit &&    go install . ./...] returned a non-zero code: 2

@johnjelinek
Copy link

I built the v1.0.0 one, but that didn't work:

# nsinit exec bash
2014/06/28 02:42:27 failed to exec: bridge is not specified

@johnjelinek
Copy link

Finally got it, I had to clone the docker repo, checkout the tag, and run make shell in the docker repo root directory. Once I was in the container that created, I had to do:

# go get -d github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit
# cd /go/src/github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer/nsinit && \
    go install . ./...

After this I copied the nsinit binary from /go/bin to docker/bundles (since that was a volume created from make shell) and then I proceeded to chmod/chown/mv/ln steps to install nsinit in /usr/local/bin.

@sidkaz
Copy link

sidkaz commented Jul 11, 2014

I am not sure if this is the right way of building nsinit, but it worked for me:

$ export GOPATH=$(pwd)
$
$ go get github.com/dotcloud/docker/vendor/src/github.com/docker/libcontainer
$ go get github.com/codegangsta/cli
$ go get github.com/coreos/go-systemd/activation
$ go get github.com/coreos/go-systemd/dbus
$ go get github.com/godbus/dbus
$ go get github.com/syndtr/gocapability/capability
$
$ go build github.com/docker/libcontainer/nsinit/nsinit
$ ./nsinit --version
nsinit version 0.1

@voidzero
Copy link

For me the build is failing miserably with Docker v1.5.0. Can someone provide an update on how to build this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment