GitLab Container Registry Setup

GitlabContainerRegistrySetup.md

Settting up a Container Registry with docker-gitlab

This should be used for new users to getting started with the container registry feature on docker-gitlab.

Requirements

• Docker Distribution >= 2.4

• Docker GitLab >= 8.8.4 ( #708 must be merged)

• TLS certificates because it should be run with https it's not designed to use it without https

Quickstart

Generate your needed certificates and use this docker-compose.yml

version: '2'

services:
  redis:
    restart: always
    image: sameersbn/redis:latest
    command:
    - --loglevel warning
    volumes:
    - /srv/gitlab/redis:/var/lib/redis:Z
  postgresql:
    restart: always
    image: sameersbn/postgresql:9.4-21
    volumes:
    - /srv/gitlab/postgresql:/var/lib/postgresql:Z
    enviroment:
    - DB_USER=gitlab
    - DB_PASS=password
    - DB_NAME=gitlabhq_production
    - DB_EXTENSION=pg_trgm

  gitlab:
    restart: always
    image: sameersbn/gitlab:latest-container-registry
    depends_on:
    - redis
    - postgresql
    ports:
    - "10080:80"
    - "5005:5005"
    - "1022:22"
    volumes:
    - /srv/gitlab/gitlab:/home/git/data:Z
    - /srv/gitlab/logs:/var/log/gitlab
    - ./certs:/certs
    networks:
    - gitlab_backend
    - nginx
    enviroment:
    - DEBUG=false

    - DB_ADAPTER=postgresql
    - DB_HOST=postgresql
    - DB_PORT=5432
    - DB_USER=gitlab
    - DB_PASS=password
    - DB_NAME=gitlabhq_production

    - REDIS_HOST=redis
    - REDIS_PORT=6379
    - GITLAB_SSH_PORT=1022
    - GITLAB_PORT=10080
    - GITLAB_HOST=localhost

    - GITLAB_SECRETS_DB_KEY_BASE=superrandomsecret
    - GITLAB_REGISTRY_ENABLED=true
    - GITLAB_REGISTRY_HOST=localhost
    - GITLAB_REGISTRY_PORT=5005
    - GITLAB_REGISTRY_API_URL=http://registry:5000
    - GITLAB_REGISTRY_KEY_PATH=/certs/registry-auth.key
    - SSL_REGISTRY_KEY_PATH=/certs/registry.key
    - SSL_REGISTRY_CERT_PATH=/certs/registry.crt

  registry:
    restart: always
    image: registry:2.4.1
    volumes:
     - /srv/gitlab/shared/registry:/registry
     - ./certs:/certs
    enviroment:
    - REGISTRY_LOG_LEVEL=info
    - REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/registry
    - REGISTRY_AUTH_TOKEN_REALM=http://gitlab/jwt/auth
    - REGISTRY_AUTH_TOKEN_SERVICE=container_registry
    - REGISTRY_AUTH_TOKEN_ISSUER=gitlab-issuer
    - REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE=/certs/registry-auth.crt
    - REGISTRY_STORAGE_DELETE_ENABLED=true
    ports:
    - "5000:5000"

Manual

Setup Docker Distrubition / Docker Registry

First of all create a cotainer from registry:2.4.1. here is a registry connfiguration example:

config.yml

version: 0.1
log:
  level: info
  formatter: text
http:
  addr: 0.0.0.0:5000
  secret: <yoursecret>
storage:
  filesystem:
    rootdirectory: /var/lib/registry
  delete:
    enabled: true
auth:
  token:
    realm: https://<yourGitLabDomain>/jwt/auth
    service: container_registry
    issuer: <gitlab-issuer>
    rootcertbundle: </certs/registry-auth.crt>

For the authentication you can use a self-signed certificate, but I would recomend a trusted certificate.

Important! Do not change the service and the realm endpoint. This leads to cruel errors.

Every <placeholder> should be replaced by your own configuration.

• rootcertbundle: This is a certificate for the private key GITLAB_REGISTRY_KEY_PATH. This could be self signed key or a normal key both should work. To generate a pr
• issuer Challenger that asks for the authentication. It must be the same like GITLAB_REGISTRY_ISSUER But this could self defined by yourself.
• realm: FQDN of your GitLab instance
• secret: A random data to sign your repsonse. Link

Generating a self signed key

#/bin/bash
echo "Create Signing Key and CSR"
openssl req -nodes -newkey rsa:2048 -keyout registry-auth.key -out registry-auth.csr -subj "/CN=gitlab-issuer"

echo "Self-Sign Certificate"
openssl x509 -in registry-auth.csr -out registry-auth.crt -req -signkey registry-auth.key -days 3650

docker-compose.yml

version: '2'
services:
  registry:
    restart: always
    image: registry:2.4.1
    ports:
      - '5000:5000'
    volumes:
      - ./data:/var/lib/registry
      - ./certs:/certs
      - ./auth:/auth
      - ./config.yml:/etc/docker/registry/config.yml

Setup Docker GitLab

docker-compose.yml

version: '2'
services:
  redis:
    restart: always
    image: sameersbn/redis:latest
    command:
    - --loglevel warning
    volumes:
    - ./redis:/var/lib/redis
  postgresql:
    restart: always
    image: sameersbn/postgresql:9.4-21
    volumes:
    - ./postgresql:/var/lib/postgresql
    enviroment:
    - DB_USER=gitlab
    - DB_PASS=<yourdatabasepassword>
    - DB_NAME=gitlabhq_production
    - DB_EXTENSION=pg_trgm

  gitlab:
    restart: always
    image: sameersbn/gitlab:latest-container-registry
    volumes:
    - ./gitlab:/home/git/data
    - ./log/:/var/log/gitlab
    - <registryPath>/data:/home/git/data/shared/registry
    - ./certs:/home/git/certs
    ports:
    - "2222:22"
    - "5005:5005"
    depends_on:
    - redis
    - postgresql
    enviroment:
    - DB_HOST=postgresql
    - DB_PORT=5432
    - DB_TYPE=postgres
    - DB_USER=gitlab
    - DB_PASS=<yourdatabasepassword>
    - DB_NAME=gitlabhq_production

    - REDIS_HOST=redis
    - REDIS_PORT=6379

    - GITLAB_SSH_PORT=2222
    - GITLAB_HOST=<yourGitLabDomain>

    - GITLAB_SECRETS_DB_KEY_BASE=<DB SECRET KEYS>

    - GITLAB_REGISTRY_ENABLED=true
    - GITLAB_REGISTRY_HOST=<Your Gitlab Container Registry Domain>
    - GITLAB_REGISTRY_PORT=<5005>
    - GITLAB_REGISTRY_API_URL=<https://<yourRegistryDomain:5000/>
    - GITLAB_REGISTRY_KEY_PATH=</home/git/certs/privkey.pem>
    - GITLAB_REGISTRY_PATH=<shared/registry>
    - GITLAB_REGISTRY_ISSUER=<gitlab-issuer>

    - SSL_REGISTRY_KEY_PATH=</home/git/certs/docker-registry.key>
    - SSL_REGISTRY_CERT_PATH=</home/git/certs/docker-registry.crt>

Every <placeholder> should be replaced by your own configuration. For all GitLab related env's have a look at the README.md here you can find what they are needed for and what they do.

Different Deployment Options:

Option 1

Registry and Gitlab are running as Containers on the same host. Both are connected to a shared docker network. Neither are externally reachable. There is a Reverse Proxy handling all incoming connections to the host (I use HAProxy).

docker-compose.yml: (only the relevant parts, this will not give you a functional setup)

version: '2'

networks:
  default:
    ipam:
      config:
        - subnet: 172.50.0.0/24

services:
  gitlab:
    image: sameersbn/gitlab:8.8.2-registry
    ports:
      - 172.50.0.1:8022:22
      - 172.50.0.1:8080:80
    environment:
      - [...]
      - GITLAB_REGISTRY_ENABLED=true
      - GITLAB_REGISTRY_HOST=registry.example.com
      - GITLAB_REGISTRY_PORT=443
      - GITLAB_REGISTRY_API_URL=http://registry:5000
      - GITLAB_REGISTRY_KEY_PATH=/certs/registry.key

      - SSL_REGISTRY_KEY_PATH=/certs/registry.key
      - SSL_REGISTRY_CERT_PATH=/certs/registry.crt
    volumes:
      - ./certs:/certs
  registry:
    image: registry:2
    ports:
      - 172.50.0.1:5000:5000
    volumes:
      - ./registry/config/config.yml:/etc/docker/registry/config.yml
      - ./certs:/certs

config.yml:

version: 0.1
log:
  fields:
    service: registry
http:
  secret: <some random string>
storage:
    cache:
        blobdescriptor: inmemory
    filesystem:
        rootdirectory: /var/lib/registry
http:
    addr: :5000
    headers:
        X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
auth:
  token:
    realm: https://gitlab.example.com/jwt/auth
    service: container_registry
    issuer: gitlab-issuer
    rootcertbundle: /certs/registry.crt

With this setup, Gitlab and the Registry are reachable only from the host. To actually use them, you need to setup a Reverse Proxy. I use HAProxy haproxy.cfg:

global
	[...]
	crt-base /etc/letsencrypt/live

frontend https
	bind <public-ipv4>:443 ssl crt example.com/haproxy.pem
	bind <public-ipv6>:443 ssl crt example.com/haproxy.pem
	mode http

	http-request add-header X-Forwarded-Proto https
	http-request set-header X-Forwarded-Port 443

	use_backend bk_gitlab if { ssl_fc_sni gitlab.example.com }
	use_backend bk_registry if { ssl_fc_sni registry.example.com }

backend bk_gitlab
	server gitlab 172.50.0.1:8080

backend bk_registry
	server registry 172.50.0.1:5000

listen gitlab
	bind <public-ipv4>:22
	bind <public-ipv6>:22

	mode tcp
	option tcplog
	timeout client 1h
	timeout server 1h
	server gitlab_registry 172.50.0.1:8022

With this setup you don't have to include any ports in any commands:

git clone git@gitlab.example.com/user/fancy-project
docker login registry.example.com
docker push registry.example.com/user/fancy-project:1.0

Option 2

Here you don't use docker networks but expose Gitlab and the Registry directly to the outside world.

gitlab:
  ports:
    - <your_public_ip>:22
    - <your_public_ip>:443
registry:
  ports:
    - <your_public_ip>:5000

In this case, you need to specify the TLS section in config.yml and point the API URL to https://example.com:5000 Again, the Reverse Proxy in the Gitlab Container for the Registry is not used.

Option 3

Let Gitlab handle all connections, Registry is not accessible directly from the outside:

gitlab:
  links:
   - registry:registry
  ports:
    - <your_public_ip>:22
    - <your_public_ip>:443
    - <your_public_ip>:5005
  environment:
    - GITLAB_REGISTRY_API_URL=http://registry:5000
registry:
  [...]

In this scenario, you don't need the TLS section in config.yml

Contributors

• mgansler
• cvle

A few things I have learned during the setup of my Gitlab/Registry System:

• The TLS section int config.yml is optional. Gitlab acts as a Reverse Proxy/SSL Termination for the registry. If you do drop the TLS section, change the GITLAB_REGISTRY_API_URL to `http://.
• rootcertbundle can point to ANY certificate, the only requirement is that GITLAB_REGISTRY_KEY_PATH points to the corrosponding private key of this certificate. I tested this with an expired key-pair for another domain and it worked just fine. I created my key-pair (certificate) like this:

openssl req -new -newkey rsa:4096 > registry.csr
openssl rsa -in privkey.pem -out registry.key
openssl x509 -in registry.csr -out registry.crt -req -signkey registry.key -days 3650

• It doesn't matter what information you put in the csr. I set the expiring time to 10 years, but it doesn't really matter as well.
• The values of issuer and GITLAB_REGISTRY_ISSUER have to match

Deployment Option 1: (I use this one, that doesn't mean you should)
Registry and Gitlab are running as Containers on the same host. Both are connected to a shared docker network. Neither are externally reachable. There is a Reverse Proxy handling all incoming connections to the host (I use HAProxy).

docker-compose.yml: (only the relevant parts, this will not give you a functional setup)

version: '2'

networks:
  default:
    ipam:
      config:
        - subnet: 172.50.0.0/24

services:
  gitlab:
    image: sameersbn/gitlab:8.8.2-registry
    ports:
      - 172.50.0.1:8022:22
      - 172.50.0.1:8080:80
    environment:
      - [...]
      - GITLAB_REGISTRY_ENABLED=true
      - GITLAB_REGISTRY_HOST=registry.example.com
      - GITLAB_REGISTRY_PORT=443
      - GITLAB_REGISTRY_API_URL=http://registry:5000
      - GITLAB_REGISTRY_KEY_PATH=/certs/registry.key

      - SSL_REGISTRY_KEY_PATH=/certs/registry.key
      - SSL_REGISTRY_CERT_PATH=/certs/registry.crt
    volumes:
      - ./certs:/certs
  registry:
    image: registry:2
    ports:
      - 172.50.0.1:5000:5000
    volumes:
      - ./registry/config/config.yml:/etc/docker/registry/config.yml
      - ./certs:/certs

note: GITLAB_REGISTRY_PORT could be empty, but even than the Gitlab UI shows a colon in the docker login/push/pull commands. Also, Gitlab fails to start because the (unused) reverse proxy fails to start. This may change in the future. For the same reason SSL_REGISTRY_KEY_PATH and SSL_REGISTRY_CERT_PATH could be omitted (but currently can't).

config.yml:

version: 0.1
log:
  fields:
    service: registry
http:
  secret: <some random string>
storage:
    cache:
        blobdescriptor: inmemory
    filesystem:
        rootdirectory: /var/lib/registry
http:
    addr: :5000
    headers:
        X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
auth:
  token:
    realm: https://gitlab.example.com/jwt/auth
    service: container_registry
    issuer: gitlab-issuer
    rootcertbundle: /certs/registry.crt

With this setup, Gitlab and the Registry are reachable only from the host. To actually use them, you need to setup a Reverse Proxy.
I use HAProxy
haproxy.cfg:

global
    [...]
    crt-base /etc/letsencrypt/live

frontend https
    bind <public-ipv4>:443 ssl crt example.com/haproxy.pem
    bind <public-ipv6>:443 ssl crt example.com/haproxy.pem
    mode http

    http-request add-header X-Forwarded-Proto https
    http-request set-header X-Forwarded-Port 443

    use_backend bk_gitlab if { ssl_fc_sni gitlab.example.com }
    use_backend bk_registry if { ssl_fc_sni registry.example.com }

backend bk_gitlab
    server gitlab 172.50.0.1:8080

backend bk_registry
    server registry 172.50.0.1:5000

listen gitlab
    bind <public-ipv4>:22
    bind <public-ipv6>:22

    mode tcp
    option tcplog
    timeout client 1h
    timeout server 1h
    server gitlab_registry 172.50.0.1:8022

With this setup you don't have to include any ports in any commands:

git clone git@gitlab.example.com/user/fancy-project
docker login registry.example.com
docker push registry.example.com/user/fancy-project:1.0

Deployment Option 2:
I haven't tried this one so I don't have examples.
Here you don't use docker networks but expose Gitlab and the Registry directly to the outside world.

gitlab:
  ports:
    - <your_public_ip>:22
    - <your_public_ip>:443
registry:
  ports:
    - <your_public_ip>:5000

In this case, you need to specify the TLS section in config.yml and point the API URL to https://example.com:5000
Again, the Reverse Proxy in the Gitlab Container for the Registry is not used.

Deployment Option 3:
Let Gitlab handle all connections, Registry is not accessible directly from the outside:

gitlab:
  links:
   - registry:registry
  ports:
    - <your_public_ip>:22
    - <your_public_ip>:443
    - <your_public_ip>:5005
  environment:
    - GITLAB_REGISTRY_API_URL=http://registry:5000
registry:
  [...]

In this scenario, you don't need the TLS section in config.yml

Deployment Option n:

Maybe Gitlab and Registry on different Hosts? I don't know, but I'm sure there are many more choices ;-)

on dev option3 can i push pull directly to the gitlab which handles those commands?

You can create the signing key pair non-interactively using:

#/bin/bash
echo "Create Signing Key and CSR"
openssl req -nodes -newkey rsa:2048 -keyout registry-auth.key -out registry-auth.csr -subj "/CN=gitlab-issuer"

echo "Self-Sign Certificate"
openssl x509 -in registry-auth.csr -out registry-auth.crt -req -signkey registry-auth.key -days 3650

@boonkerz sorry, I did not get notified, but yes you can

@cvle thanks! I think the /CN=gitlab-issuer is optional though, the certificate itself never get's checked for anything, only the private part must match the public part.

with self signed certs you have to do https://docs.docker.com/registry/insecure/

If its possible to use letsenrypt?

@boonkerz: yes.

Newer versions of Gitlab (>=9.4) now overwrite the key you generated with openssl, and you need to supply it in the registry['internal_key'] variable on startup (in addition to the file). It's horrible, but the workaround described in there works.

https://gitlab.com/gitlab-org/gitlab-ce/issues/35461

There's four hours of my life I won't get back.