smichaelsen/gitlab_typo3_setup.md

## gitlab_typo3_setup.md

      
    Raw
  

              gitlab_typo3_setup.md
            
          
    Setting up a Build Server for TYPO3 projects on Gitlab.com

Background

We're a small team of only two people, but we're tackling more and more rather big TYPO3 projects. Over time we introduced package management for PHP (composer) and JS (bower) and depended on frontend building with Grunt. With rising complexity and importance of our projects we felt the demand for an efficient and safe way to deploy our websites on different servers.
We helped our selves with small self written deploy scripts that performed building on the target server, but you can't run node (for Grunt) or sass in every environment, and if building goes wrong the website was likely to be broken on the live server - so we needed a Build Server.
But solutions we looked at turned out to be quite time consuming (setting up and managing self hosted Gitlab and Gitlab-CI or a Jenkins server and runners) or costly.
Free Hosting on Gitlab.com

Since a few months Gitlab.com offers free hosting for private repositories and also a free "shared" build runner. I really like the Gitlab interface and its features, but I also need to tell you the downside of building everything on Gitlab.com hosted repositories:
Since they started their free service, the git server was always a bit slow and they promised to improve speed. I didn't notice any improvements so far. So any network based git operations (pull, push, fetch, etc) take a few seconds to complete. Something that we accept for the benefits we get.
Set up

To enable the building for a repository you need to provide a .gitlab-ci.yml file on the root level of your project.
We organize our TYPO3 installations by having a main repository for the installation and more repositories for each of our client specific TYPO3 extensions. We load all extensions via composer. Most of our extensions need some frontend building to run properly. What we want our Build Server to do is:

Load the main repository
Load the composer dependencies
Perform frontend building for our custom extensions
Sync the ready-to-run installation on the target server
Perform finishing commands on the target server (like clearing the cache, making a backup, perform database scheme compare etc)
Notify us if something goes wrong

Here's what a .gitlab-ci.yml looks like for a project of ours:
variables:
  SSH_HOST: username@ssh.targethost.org
  SSH_PATH: html

stages:
  - deploy
  - notify

deploy:
  stage: deploy
  before_script:
    - apt-get update -qq && apt-get -qq install -y nodejs npm php5 php5-cli ruby rsync
    - ln -s /usr/bin/nodejs /usr/bin/node
    - gem install sass
    - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
    # we include a .ssh folder in our project with a keypair to authenticate ourselves on the target server. Further down the folder is excluded from the rsync command. That's important - you MUST NOT have your keys be downloadable on the webserver
    - mkdir ~/.ssh && cp .ssh/id_rsa ~/.ssh/ && chmod 600 ~/.ssh/id_rsa && cp .ssh/id_rsa.pub ~/.ssh/ && cp .ssh/config ~/.ssh/
    - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
  script:
    - composer install --no-dev
    - cd typo3conf/ext/client_extension_1
    - npm install && ./node_modules/grunt-cli/bin/grunt build
    - cd - && cd typo3conf/ext/client_extension_2
    - npm install && ./node_modules/grunt-cli/bin/grunt build
    - cd -
    - rsync -az -e "ssh" --delete --exclude=fileadmin --exclude=typo3conf/l10n --exclude=.git --exclude=node_modules --exclude=bower_components --exclude=typo3temp --exclude=.ssh --exclude=.sass-cache --exclude=/uploads ./ $SSH_HOST:$SSH_PATH/
    # Perform some commands on the target server as you need
    - ssh $SSH_HOST 'cp $SSH_PATH/typo3conf/AdditionalConfiguration_Stage.php $SSH_PATH/typo3conf/AdditionalConfiguration.php && cd $SSH_PATH && ./scripts/backup_on_deploy.sh'

notify_build_error:
    stage: notify
    when: on_failure
    script:
        - curl --data "Deployment failed // Build error! $CI_BUILD_REF $CI_BUILD_REPO" $'https://our-slack-team.slack.com/services/hooks/slackbot?token=someslacktoken&channel=%23general'


Running a build with this setup takes something between 4 and 7 minutes. It is triggered on every commit. Next steps would be to distinguish between staging and production server (e.g. by branch).
The yml file has a lot more options: http://doc.gitlab.com/ee/ci/yaml/README.html
Performing tests

Our build process does not execute any tests on the extensions. They don't belong here. Performing tests on the extensions should be done by adding a build job to their repositories so you get aware of problems while comitting extension changes.
What could be added here are functional tests for the website. But at this stage we have none.
Conclusion

This is what we have so far. We're new to the topic and happy about any feedback or input. Also I hope to help to get more people started.
Sebastian Michaelsen

@s_michaelsen

sebastian@app-zap.de

http://app-zap.de