gitlab production installation: good bye subversion

gitlab production installation: good bye subversion

# Installation
* See the restore instructions if you need to restore from backups
* Make sure /srv/gitlab/ is completely removed in the host
* Use the below ~/docker-compose.yml file. The gitlab root user takes effect *only* for brand new installations
```
gitlab:
  image: 'gitlab/gitlab-ce:8.7.0-ce.0'
  restart: always
  hostname: 'gitlab.sample.com'
  container_name: 'gitlab'
  environment:
    GITLAB_OMNIBUS_CONFIG: |
      external_url 'http://gitlab.sample.com:9090'
      gitlab_rails['gitlab_shell_ssh_port'] = 2222
      gitlab_rails['gitlab_email_enabled'] = true
      gitlab_rails['gitlab_email_from'] = 'gitlab@sample.com'
      gitlab_rails['gitlab_email_display_name'] = 'sample Gitlab'
      gitlab_rails['gitlab_email_reply_to'] = 'noreply@sample.com'
      gitlab_rails['smtp_enable'] = true
      gitlab_rails['smtp_address'] = "samplembx.krco.com"
      gitlab_rails['smtp_port'] = 25
      gitlab_rails['smtp_domain'] = "sample.com"
      gitlab_rails['smtp_authentication'] = "plain" #plain, login
      gitlab_rails['smtp_enable_starttls_auto'] = false
      gitlab_rails['backup_keep_time'] = 604800
    GITLAB_ROOT_EMAIL: 'ci@sample.com'

  ports:
    - '9090:9090'
    - '2222:22'
  volumes:
    - '/srv/gitlab/config:/etc/gitlab'
    - '/srv/gitlab/logs:/var/log/gitlab'
    - '/srv/gitlab/data:/var/opt/gitlab'
```
Note that we must use a specific version otherwise backup/restore would fail.

# Image version upgrade
Must be tested: Apparently just stoping, removing and pulling the new version will take care of upgrading the database.

# Troubleshooting
* To connect to local gitlab running docker container:
```
sudo docker exec -i -t gitlab /bin/bash
```

# Manual configuration
Additional manual configurations are not possible when using docker so changing /etc/gitlab/gitlab.rb and then running the below will not work as expected:
```
gitlab-ctl reconfigure
```
Instead, use docker-compose.yml to introduce the changes and kill/start the container.

# Project creation:
* Better to be created under a specific group (like for example dev)
* Should be private always and access granted user by user

# git client configuration
* Use Profile Settings | SSH Keys to add your public SSH key.
* Command line instructions to work with git are available in each project landing page

# Administration
* To kill:
```
docker-compose kill gitlab
```
* To start:
```
docker-compose up -d gitlab
```

# Migrating from subversion
First of all consider keeping for some months your old subversion in read only and start fresh with git. If that is a possibility (leaving without full history in git) you will save an outrageous amount of time in your conversion.

Assuming you want a full copy, start by hitting your gitlab instance and:
* Create group dev and your own user
* Assign master role to your user
* Assign your user to the group (members link on the left after selecting the group)
* Create project in the group namespace

On the client side:
* I had to follow https://github.com/nirvdrum/svn2git/issues/50#issuecomment-14707504 to resolve git svn clone issues.
* Other than taht just follow the below steps and make sure you run this from a machine that will not be off for some time or probably use nohup. It will take a while ...
```
git config --global user.name "Nestor Urquiza"
git config --global user.email "nurquiza@sample.com"
sudo apt-get install -y git-svn
cat ~/.ssh/id_rsa.pub
svn log -q http://subversion.sample.com/repos/reporting/projects/fe/ | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > ~/authors.txt
# Warning: Better if you can update ~/authors.txt with proper email for users
git svn clone --stdlayout --no-metadata -A ~/authors.txt http://subversion.sample.com/repos/reporting/projects/fe/
cd fe
git remote add gitlab ssh://git@gitlab.sample.com:2222/dev/sample-fe.git
git push --set-upstream gitlab master
git for-each-ref refs/remotes/tags | cut -d / -f 4- | grep -v @ | while read tagname; do git tag "$tagname" "tags/$tagname"; git branch -r -d "tags/$tagname"; done
git for-each-ref refs/remotes | cut -d / -f 3- | grep -v @ | while read branchname; do git branch "$branchname" "refs/remotes/$branchname"; git branch -r -d "$branchname"; done
git branch -d trunk
git push gitlab --all
git push gitlab --tags
```

# Add .gitignore based on svn:ignore property
From a fresh copy of the subversion repo I used https://gist.github.com/iegik/3450385 to generate .gitignore:
```
svn propget -R svn:ignore | grep -v "^$" | sed "s/\(\(.*\) - \)\(.*\)/\2\/\3/g" | sort >> .gitignore
```
Clone the git repo and add the .gitignore file, commit and push back to the origina master.

# Shrink the git repo
If the repo is big this will take a long time. Once done clone your repo and run the below command. If your repo is big consider looking for a specific culprit like big binary files that were committed before (regardless if they were already deleted). If that is not the apparent cause of the problems then most likely history alone is the culprit. You will need to identify then full directories not in use anymore. Follow the steps from http://naleid.com/blog/2012/01/17/finding-and-purging-big-files-from-git-history to determine what can be eliminated. In our case history and not specific big files were the culprits. The below command for example calculates the total size being used by the webapp directory which was needed in old pure server side MVC application but not longer needed in new SPA based application:
```
cat bigtosmall.txt | grep webapp | awk '{total=total+$2} END{print total}'
```
Per the link eliminating these would require the below command:
```
git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch webapp' --prune-empty --tag-name-filter cat -- --all
```
Note that after running the command you need to deparately clone the git repo to confirm the size has shrunk.

# Backup
Backup is done via environment/production/scripts/tools.sample.com/tool_backup.sh

# Restore
To restore:
* Put the backup file in /srv/gitlab/data/backups/ and run the below. It will import projects, users etc. You must not try to recover from a backup done with a different version of gitlab!:
```
sudo find /srv/gitlab/data/backups/ -name "*gitlab_backup.tar" | xargs sudo chmod 666
sudo docker exec gitlab /bin/bash -c 'export force=yes &&  gitlab-rake gitlab:backup:restore'
```

# Upgrade
* One more time: You must not try to recover from a backup done with a different version of gitlab! This means you need to test the new version in a separated server before. Shouldn't you always do this anyway?

# Maintenance
* To change password inside container
```
root@gitlab:/# gitlab-rails console production
irb(main):001:0> user = User.where(id: 1).first
irb(main):001:0> user.password = '*****'
irb(main):001:0> user.password_confirmation = '******'
irb(main):001:0> user.save
```