GitLab CI local development

gitlab-ci-local.bash

# The central problem right now, is that gitlab-runner exec was deprecated with GitLab 10 - and then undeprecated later,
# because of missing alternatives - see https://gitlab.com/gitlab-org/gitlab-runner/issues/2797
# So this guide is somehow only a work documentation on how I tried to use this feature

### OPTION 1: Install gitlab-runner directly on your system

# See 
# http://bryce.fisher-fleig.org/blog/faster-ci-debugging-with-gitlabci/index.html

# Install GitLab Runner locally
# for Mac see https://docs.gitlab.com/runner/install/osx.html#installing
brew install gitlab-runner 

# Linux & Windows 10 WLS see https://docs.gitlab.com/runner/install/linux-repository.html
# For Debian/Ubuntu/Mint
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner
# For RHEL/CentOS/Fedora
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh | sudo bash
sudo yum install gitlab-runner


# Register GitLab Runner as service and start it
brew services start gitlab-runner

# clone and cd into your project with .gitlab-ci.yml File
git clone https://github.com/jonashackt/restexamples.git
cd restexamples

# run GitLab CI Job locally with Docker and gitlab-runner service in format
gitlab-runner exec docker build-image

# or shell
gitlab-runner exec shell build-image

# error
$ docker login -u '$CI_REGISTRY_USER' -p '$CI_JOB_TOKEN $CI_REGISTRY'
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://registry-1.docker.io/v2/: unauthorized: incorrect username or password

# problem: the predefined GitLab CI variables https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
# aren't predefined in our local environment - see https://stackoverflow.com/a/43874529/4964553
# See GitLabs ideas to improve here: https://gitlab.com/gitlab-org/gitlab-runner/issues/2797#note_51070788

# Step 1: Show values of variables 
before_script:
  - printenv
  
# Step 2: Set unset variables
export CI_REGISTRY=gitlab.jonashackt.io:4567
export CI_REGISTRY_USER=yourUserNameHere
export CI_JOB_TOKEN=yourPasswordHere

# problem: Secret varialbes like CI_JOB_TOKEN are blanked out, even when they are set
# see https://gitlab.com/gitlab-org/gitlab-runner/issues/3102

# We can try to override CI_JOB_TOKEN, but https://gitlab.com/gitlab-org/gitlab-runner/issues/3996
Currently, the CI_JOB_TOKEN inside .gitlab-ci.yml has a higher priority then the Environment varialbe defined outside - so we can't override it


### OPTION 2: Use gitlab-runner itself inside a Docker Container
# See https://docs.gitlab.com/runner/install/docker.html

# clone and cd into your project with .gitlab-ci.yml File
git clone https://github.com/jonashackt/gitlab-ci-dind-example.git
cd gitlab-ci-dind-example

# run GitLab CI Job locally with Docker and gitlab-runner service in format
# with -v $(pwd):/app we make the application code incl. the .gitlab-ci.yml available inside the gitlab/gitlab-runner container
# with -w=/app we set `/app` as the workdir, so that gitlab-runner finds the .gitlab-ci.yml to execute
# (which has no option for other filelocations, see https://docs.gitlab.com/runner/commands/README.html#gitlab-runner-exec)
docker run --rm -t -i -v $(pwd):/app -w=/app gitlab/gitlab-runner exec shell build-image

# or we can even run docker-in-docker
# then we additionally need to mount the Docker Socket as a volume into the gitlab/gitlab-runner container with
# -v /var/run/docker.sock:/var/run/docker.sock
# to prevent `2019-08-22T12:59:20.965373372Z mount: permission denied (are you root?)` errors, we also add
# docker --docker-privileged 
docker run --rm -t -i -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):/app -w=/app gitlab/gitlab-runner exec docker --docker-privileged build-image