RPM build and hosting workflow using github, travis-ci and copr

rpm-build-workflow.md

RPM Build Flow

This document details a simple RPM build flow pattern used to build and host RPM artifacts for open source projects. The below is a visual summary of this flow.

In order to achieve this multiple tools and services are used. The services and their purpose in the flow is as listed below.

Service	Purpose
GitHub	As is the most common use for GitHub, it holds the build source code. In this case we hold only the spec files and related source files. All other sources, including project binaries/sources are retrieved at build time.
Travis CI	Travis provides the CI environment in order to generate source RPMs required for the final Copr builds.
Docker Hub	At the core of the source rpm builds, we use a containerised RPM build environment. The image for which is hosted on Docker Hub as alectolytic/rpmbuilder and is available for Fedora and Centos distros.
Fedora Copr	The Fedora Copr project provides an automated build system and a hosted package repository for non-mainstream packages. This is build system is where the RPMs following this flow gets built and hosted.

Build Flow Steps

RPM specfile and sources

The RPM specfile and sources should be compatible with the containerised RPM build environment. Examples of this can be found at the following repositories:

• https://github.com/abn/metabase-rpm
• https://github.com/abn/stolon-rpm
• https://github.com/abn/traefik-rpm

Travis CI build

The Travis CI build is triggered on changes to the repository. However, a source RPM is only pushed to the Copr build system only on a tag.

Travis configuration

The following is an example travis configuration used.

sudo: required
language: python

env:
  global:
    - DOCKER_IMAGE=alectolytic/rpmbuilder
    - COPR_REPOSITORY=repository
    - OS_ARCH=x86_64
  matrix:
    - OS_TYPE=fedora OS_DIST=fedora OS_VERSION=24
    - OS_TYPE=centos OS_DIST=epel OS_VERSION=7

services:
  - docker

install: true

script:
  - docker run -v ${PWD}:/sources -v ${PWD}:/output:Z -e "SRPM_ONLY=1" ${DOCKER_IMAGE}:${OS_TYPE}-${OS_VERSION}

after_success:
  - pip install copr-cli simplejson
  - openssl aes-256-cbc -K $<ENCRYPTED KEY VAR> -iv $<ENCRYPTED KEY VAR> -in .copr.enc -out .copr -d
  - if [ ! -z "${TRAVIS_TAG}" ]; then copr-cli --config .copr build -r ${OS_DIST}-${OS_VERSION}-${OS_ARCH} ${COPR_REPOSITORY} *.src.rpm; fi

The above configuration builds the SRPMs using the alectolytic/rpmbuilder image by mounting the source code in the working directory for fedora-24 and centos-7. The operating system distro and versions are controlled via the environment variables OS_TYPE, OS_DIST and OS_VERSION respectively. Note the distinction and requirement of both OS_TYPE and OS_DIST this is required because the version mapping for the rpmbuilder image and the copr repositories are not 1:1.

Note that the OS_ARCH variable is not under the matrix section; this is because this build flow has not been validated on multi-arch examples.

Copr configuration

The COPR_REPOSITORY variable in the .travis.yml file specifies which of your copr repositories you want to build this package for.

Under the travis configuration's after_success section, you'll notices the openssl command. This is there to decrypt Copr API credentials. The confiugration file contents can be retrieved from here once you have logged in.

Once you have this file the you can generate the openssl command by doing the following if you have travis command line client installed.

travis encrypt-file ~/.config/copr .copr.enc

Note that the -out section of the command that is output when executing the above is different to what we have in the .travis.yml file. Additionally, be also careful as to not commit the unencrypted file if you have created it in the working directory. More information around file encryption and travis can be found here.

Copr build

Once decryption is correctly configured, any tag builds on travis will push a source RPM to your copr repository. This will trigger a new build based on your configuration of your repository. Be sure to correctly configure supported environments. A step-by-step guide on creating a project on copr is available here and user documentation is available here.

My common repository containing all packages built using this flow is available at abn/repository.

Using packages in copr repository

Manual methods

Documentation on how to enable repositories is available here.

Ansible role

An ansible role to enable copr repositories is available at abn/role-copr-repository. This supports both Fedora and CentOS distros.

rpm-build-workflow.png

Hi! We've made a tool for that: https://github.com/packpack/packpack

HI!
I created https://github.com/patsevanton/nomad-rpm
Added .travis.yml
But get error:
travis encrypt-file ~/.config/copr .copr.enc
repository not known to https://api.travis-ci.org/: patsevanton/nomad-rpm

How fix it?

Answer: need time

Hi!
If i run
openssl aes-256-cbc -K $encrypted_5a1ae0da8fa2_key -iv $encrypted_5a1ae0da8fa2_iv -in .copr.enc -out .copr -d
get error:
iv undefined

Travis write
Please add the following to your build script (before_install stage in your .travis.yml, for instance):
openssl aes-256-cbc -K $encrypted_5a1ae0da8fa2_key -iv $encrypted_5a1ae0da8fa2_iv -in .copr.enc -out ~/.config/copr -d

travis encrypt-file .copr .copr.enc -p
and 2 keys copy to travis env config