It is possible to find Docker images for Amazon Linux, but because the RPM repositories for Amazon Linux are only available within AWS, they only work if the Docker host is also on AWS.
Instead, we're going to try use Centos, which like Amazon Linux is derived from RedHat Enterprise Linux, cross our fingers, and hope.
First let's see what we've actually got on uc3-dash2-dev.cdlib.org:
-bash-4.2$ uname -a
Linux uc3-dash2-dev 4.1.13-19.30.amzn1.x86_64 #1 SMP Fri Dec 11 03:42:10
UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Now, Centos ships with a 3.1 kernel, but since this is Docker, we don't
care; all Docker containers share the kernel of the Docker host (which in
this case is a VirtualBox VM installed by Docker Toolbox). Let's spin up a
temporary Docker container and run uname on that so we can see what we've
got.
$ docker run ubuntu:14.04 uname -a
Linux 18f2e96c0abc 4.1.13-boot2docker #1 SMP Fri Nov 20 19:05:50 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
Excellent, Docker and Amazon are both keeping their kernels up to date.
Now let's find and install Centos on Docker.
$ docker pull centos:7
Pulling repository docker.io/library/centos
Error while pulling image:
Get https://index.docker.io/v1/repositories/library/centos/images:
dial tcp: lookup index.docker.io on 128.32.136.9:53: server misbehaving
Okay. This Stack Overflow answer implies it may have something to do with the fact that we started the Docker machine yesterday over at BIDS, and now we're back at CDL. Let's try the answer suggested there:
$ docker-machine restart default && eval "$(docker-machine env default)"
(default) Restarting VM...
Restarted machines may have new IP addresses. You may need to re-run the
`docker-machine env` command.
Error checking TLS connection: Error checking and/or regenerating the
certs: There was an error validating certificates for host
"192.168.99.100:2376": dial tcp 192.168.99.100:2376: getsockopt: connection
refused
You can attempt to regenerate them using 'docker-machine regenerate-certs
[name]'.
Be advised that this will trigger a Docker daemon restart which will stop
running containers.
Hmm. Regenerating certs sounds good, but what do I put in for [name]?
This answer suggests default
(cf. eval "$(docker-machine env default)". Presumably if we had other
machines in ~/.docker/machine/machines/ we'd have to worry about which
one, but we don't.
$ docker-machine regenerate-certs default
Regenerate TLS machine certs? Warning: this is irreversible. (y/n): y
Regenerating TLS certificates
Detecting the provisioner...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Okay, let's try that pull again.
$ docker pull centos:7
7: Pulling from library/centos
fa5be2806d4c: Pull complete
2bf4902415e3: Pull complete
86bcb57631bd: Pull complete
c8a648134623: Pull complete
Digest: sha256:db4d91ef365f4716517d5cbdb1be54b6d205912038d67bf687a4c2dad9b85928
Status: Downloaded newer image for centos:7
Much better. And let's check our assumptions about how Docker kernels work:
$ docker run centos:7 uname -a
Linux 96c0898c8ad8 4.1.13-boot2docker #1 SMP Fri Nov 20 19:05:50 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
Ok, good.
We start with a basic Dockerfile, in a directory we'll call stash-docker:
# image: stash/docker
FROM centos:7
MAINTAINER david.moles@ucop.edu
WORKDIR /home
(According to Titus Brown at the BIDS workshop, /home is more or less the
conventional Docker working directory. Note that by default, Docker runs
everything as root. This isn't as big a deal as it might seem since
root's privilges are confined to this single-purpose Docker container. In
a sense the container itself takes the place of the "role account".)
Starting with Scott's "Installing Ruby, Rails and Passenger to Home Directory on Server", let's figure out what we're going to need.
We'll do this interactively first.
$ docker run -it centos:7 /bin/bash
[root@76981257b993 /]#
We're now logged into a Centos 7 Docker container, so we can try following Scott's instructions.
[root@76981257b993 /]# cd home
[root@76981257b993 /home]# wget http://pyyaml.org/download/libyaml/yaml-0.1.5.tar.gz
bash: wget: command not found
Oops. The centos image must be pretty bare. Let's try curl instead.
[root@76981257b993 home]# curl -O http://pyyaml.org/download/libyaml/yaml-0.1.5.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 493k 100 493k 0 0 378k 0 0:00:01 0:00:01 --:--:-- 378k
So far, so good.
[root@76981257b993 home]# tar -zxvf yaml-0.1.5.tar.gz
yaml-0.1.5/
...
(lots of files here)
...
yaml-0.1.5/tests/test-version.c
Let's add this to the Dockerfile:
RUN curl -O http://pyyaml.org/download/libyaml/yaml-0.1.5.tar.gz &&
tar -zxvf yaml-0.1.5.tar.gz
(We do this as a single operation, with &&, because reasons.)
And now on to configure:
[root@76981257b993 home]# cd yaml-0.1.5
[root@76981257b993 yaml-0.1.5]# ./configure --prefix=/dash2/local
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... no
checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/home/yaml-0.1.5':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
[root@76981257b993 yaml-0.1.5]#
OK, the Centos image is really bare. It looks like we need make and
gcc, at minimum.
Let's check uc3-dash2-dev.cdlib.org to see which RPMs we ought to be
installing:
dash2@uc3-dash2-dev:~$ rpm -q --whatprovides `which make`
make-3.82-21.10.amzn1.x86_64
Sensible enough:
[root@76981257b993 yaml-0.1.5]# yum install make
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
* base: mirrors.usc.edu
* extras: mirror.vcu.edu
* updates: mirrors.gigenet.com
Resolving Dependencies
--> Running transaction check
---> Package make.x86_64 1:3.82-21.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================================
Package Arch Version Repository Size
=========================================================================================================
Installing:
make x86_64 1:3.82-21.el7 base 420 k
Transaction Summary
=========================================================================================================
Install 1 Package
Total download size: 420 k
Installed size: 1.1 M
Is this ok [y/d/N]: y
Downloading packages:
make-3.82-21.el7.x86_64.rpm | 420 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : 1:make-3.82-21.el7.x86_64 1/1
Verifying : 1:make-3.82-21.el7.x86_64 1/1
Installed:
make.x86_64 1:3.82-21.el7
Complete!
[root@76981257b993 yaml-0.1.5]#
Good, now let's add it to the Dockerfile. (We'll make this the first RUN statement since we want it already there when we start messing with pyyaml.) Note the -y so we don't get prompted with Is this ok [y/d/N].
RUN install -y make
What about gcc?
dash2@uc3-dash2-dev:~$ rpm -q --whatprovides `which gcc`
file /usr/bin/gcc is not owned by any package
...OK, that's slightly disturbing. But let's assume it's gcc and see
where that gets us.
[root@76981257b993 yaml-0.1.5]# yum install gcc
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
* base: mirrors.usc.edu
* extras: mirror.vcu.edu
* updates: mirrors.gigenet.com
Resolving Dependencies
--> Running transaction check
---> Package gcc.x86_64 0:4.8.5-4.el7 will be installed
--> Processing Dependency: libgomp = 4.8.5-4.el7 for package: gcc-4.8.5-4.el7.x86_64
--> Processing Dependency: cpp = 4.8.5-4.el7 for package: gcc-4.8.5-4.el7.x86_64
--> Processing Dependency: glibc-devel >= 2.2.90-12 for package: gcc-4.8.5-4.el7.x86_64
--> Processing Dependency: libmpfr.so.4()(64bit) for package: gcc-4.8.5-4.el7.x86_64
--> Processing Dependency: libmpc.so.3()(64bit) for package: gcc-4.8.5-4.el7.x86_64
--> Processing Dependency: libgomp.so.1()(64bit) for package: gcc-4.8.5-4.el7.x86_64
--> Running transaction check
---> Package cpp.x86_64 0:4.8.5-4.el7 will be installed
---> Package glibc-devel.x86_64 0:2.17-106.el7_2.1 will be installed
--> Processing Dependency: glibc-headers = 2.17-106.el7_2.1 for package: glibc-devel-2.17-106.el7_2.1.x86_64
--> Processing Dependency: glibc-headers for package: glibc-devel-2.17-106.el7_2.1.x86_64
---> Package libgomp.x86_64 0:4.8.5-4.el7 will be installed
---> Package libmpc.x86_64 0:1.0.1-3.el7 will be installed
---> Package mpfr.x86_64 0:3.1.1-4.el7 will be installed
--> Running transaction check
---> Package glibc-headers.x86_64 0:2.17-106.el7_2.1 will be installed
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers-2.17-106.el7_2.1.x86_64
--> Processing Dependency: kernel-headers for package: glibc-headers-2.17-106.el7_2.1.x86_64
--> Running transaction check
---> Package kernel-headers.x86_64 0:3.10.0-327.4.4.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================================
Package Arch Version Repository Size
=========================================================================================================
Installing:
gcc x86_64 4.8.5-4.el7 base 16 M
Installing for dependencies:
cpp x86_64 4.8.5-4.el7 base 5.9 M
glibc-devel x86_64 2.17-106.el7_2.1 updates 1.0 M
glibc-headers x86_64 2.17-106.el7_2.1 updates 661 k
kernel-headers x86_64 3.10.0-327.4.4.el7 updates 3.2 M
libgomp x86_64 4.8.5-4.el7 base 130 k
libmpc x86_64 1.0.1-3.el7 base 51 k
mpfr x86_64 3.1.1-4.el7 base 203 k
Transaction Summary
=========================================================================================================
Install 1 Package (+7 Dependent packages)
Total download size: 27 M
Installed size: 59 M
Is this ok [y/d/N]: y
Downloading packages:
(1/8): kernel-headers-3.10.0-327.4.4.el7.x86_64.rpm | 3.2 MB 00:00:00
(2/8): glibc-devel-2.17-106.el7_2.1.x86_64.rpm | 1.0 MB 00:00:01
(3/8): glibc-headers-2.17-106.el7_2.1.x86_64.rpm | 661 kB 00:00:01
(4/8): cpp-4.8.5-4.el7.x86_64.rpm | 5.9 MB 00:00:02
(5/8): libgomp-4.8.5-4.el7.x86_64.rpm | 130 kB 00:00:00
(6/8): libmpc-1.0.1-3.el7.x86_64.rpm | 51 kB 00:00:00
(7/8): mpfr-3.1.1-4.el7.x86_64.rpm | 203 kB 00:00:00
(8/8): gcc-4.8.5-4.el7.x86_64.rpm | 16 MB 00:00:05
---------------------------------------------------------------------------------------------------------
Total 5.3 MB/s | 27 MB 00:00:05
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : mpfr-3.1.1-4.el7.x86_64 1/8
Installing : libmpc-1.0.1-3.el7.x86_64 2/8
Installing : cpp-4.8.5-4.el7.x86_64 3/8
Installing : libgomp-4.8.5-4.el7.x86_64 4/8
Installing : kernel-headers-3.10.0-327.4.4.el7.x86_64 5/8
Installing : glibc-headers-2.17-106.el7_2.1.x86_64 6/8
Installing : glibc-devel-2.17-106.el7_2.1.x86_64 7/8
Installing : gcc-4.8.5-4.el7.x86_64 8/8
Verifying : kernel-headers-3.10.0-327.4.4.el7.x86_64 1/8
Verifying : gcc-4.8.5-4.el7.x86_64 2/8
Verifying : cpp-4.8.5-4.el7.x86_64 3/8
Verifying : libgomp-4.8.5-4.el7.x86_64 4/8
Verifying : glibc-devel-2.17-106.el7_2.1.x86_64 5/8
Verifying : mpfr-3.1.1-4.el7.x86_64 6/8
Verifying : glibc-headers-2.17-106.el7_2.1.x86_64 7/8
Verifying : libmpc-1.0.1-3.el7.x86_64 8/8
Installed:
gcc.x86_64 0:4.8.5-4.el7
Dependency Installed:
cpp.x86_64 0:4.8.5-4.el7 glibc-devel.x86_64 0:2.17-106.el7_2.1
glibc-headers.x86_64 0:2.17-106.el7_2.1 kernel-headers.x86_64 0:3.10.0-327.4.4.el7
libgomp.x86_64 0:4.8.5-4.el7 libmpc.x86_64 0:1.0.1-3.el7
mpfr.x86_64 0:3.1.1-4.el7
Complete!
[root@76981257b993 yaml-0.1.5]#
Looks pretty good. Let's go back to uc3-dash2-dev.cdlib.org and check the gcc version there:
dash2@uc3-dash2-dev:~$ gcc --version
gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-9)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
dash2@uc3-dash2-dev:~$
4.8.3 vs. 4.8.5. Close enough. Let's add that to the Dockerfile:
RUN yum install -y make &&
yum install -y gcc
Back to yaml.
We run ./configure --prefix=/dash2/local again. It looks pretty good this time, except,
about halfway through:
./configure: line 6343: /usr/bin/file: No such file or directory
It looks like it completes successfully anyway, but just to be on the safe side, let's
yum install file as well, update the Dockerfile appropriately, and re-configure,
just to be sure.
[root@76981257b993 yaml-0.1.5]# ./configure --prefix=/dash2/local
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking whether ln -s works... yes
checking whether make sets $(MAKE)... (cached) yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... no
checking if : is a manifest tool... no
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for doxygen... false
checking for ANSI C header files... (cached) yes
checking for stdlib.h... (cached) yes
checking for an ANSI C-conforming const... yes
checking for size_t... yes
configure: creating ./config.status
config.status: creating yaml-0.1.pc
config.status: creating include/Makefile
config.status: creating src/Makefile
config.status: creating Makefile
config.status: creating tests/Makefile
config.status: creating win32/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
Much better.
make looks to complete successfully, and so does make install, even helpfully
creating /dash2/local for us:
[root@76981257b993 yaml-0.1.5]# find /dash2/local -type f
/dash2/local/include/yaml.h
/dash2/local/lib/libyaml-0.so.2.0.3
/dash2/local/lib/libyaml.la
/dash2/local/lib/pkgconfig/yaml-0.1.pc
/dash2/local/lib/libyaml.a
So the first part of this is pretty straightforward:
RUN curl -L https://github.com/postmodern/ruby-install/archive/v0.5.0.tar.gz > \
ruby-install-0.5.0.tar.gz &&
tar -zxvf ruby-install-0.5.0.tar.gz &&
cd ruby-install-0.5.0
The only tricky bits are (1) the -L, telling curl to follow redirects
(otherwise you get an HTML file instead of a tarball), and the trailing \
to let us sensibly break that first line. However, Scott then tells us:
# edit PREFIX variable toward the first of the Makefile to have the correct directory such as /dash2/local
Let's look at that line:
PREFIX?=/usr/local
Luckily, the Makefile authors aren't insane, and we should be able to
set the environment variable externally (the ?= should mean "set only
if absent").
[root@76981257b993 ruby-install-0.5.0]# PREFIX=/dash2/local && make install
for dir in `find bin share -type d`; do mkdir -p /usr/local/$dir; done
for file in `find bin share -type f`; do cp $file /usr/local/$file; done
mkdir -p /usr/local/share/doc/ruby-install-0.5.0
cp -r *.md *.txt /usr/local/share/doc/ruby-install-0.5.0/
Nice try, but nope. It's still stuffing things in /usr/local/.