As you may already know, maintaining the mirror is a painful task. Everytime a new package changes release number (not even a version number) we need to update the mirror. There's a lot of work here and we are looking for solutions in different places.
Thinking in this problem I was wondering on how other distributions build their packages and on what are the requirements they request to the developers to have a complete build. I remove from my list the big ones, mostly because to me everything looks so complex that I get lost. So, I started to look for "small" linux distributions, more specific on Fedora and CentOS derivatives. By this way I found the Korora Project.
Looking into their repositories I found that they use mock with the difference that they don't require to download a mirror to get a build. If you see [these files] you'll see mock configuration files for several Fedora versions. So I started to think that we could define our mock environment in a similar way and remove the need of the mirror.
In this report I'll detail an experiment with StarlingX build system where the mock environment was changed to use external repositories.
The build system does a lot of things (that I don't understand yet), but in some point a mock environment is created to build every package. In the mock config file you can see that four reposities are defined:
- local-std: Holds packages resulting from the
stdtarget. Most of the resulting packages goes here. - local-rt: The real time packages, mainly a kernel and the related modules.
- local-installer: Packages from the
installertarget, I think is justanacondaandrpm. - TisCentos7Distro: This points to the mirror that was downloaded.
In other words, mock has four repositories to find packages from the mirror and from the current building. Actually if some package depends on a package that hasn't be built yet, then the build system will reschedule until the dependency is found.
Based on our mock environment, the experiment consists on modify the mock config file in this way:
- Remove the
TisCentos7Distrorepository. - Add the repositories for CentOS 7.4 and 7.5 used by the mirror downloader.
- Enable networking
- Run the build
The mirror still exists in the build system environment as the src.rpm and tarballs are used from there.
In the first attempt the build failed due to not have the GPG keys for some packages. So the gpgcheck variable was set to 0 to avoid this problem. This problem was caused by not having the same GPG keys in the builder container as the mirror downloader has.
The rt target built successfully, but six packages failed:
- libvirt-python
- openstack-murano-ui
- kubernetes
- shim-signed
- lldpd
- python-request
This failure was caused due to the mock environment was installing libvirt-devel version 4.2 (and dependent packages) instead of the required 3.5 (and generated) by StarlingX. I was able to build manually in mock with:
rpm -e libvirt-devel
yum install libvirt-devel --disablerepo=* --enablerepo=local-std
And then rebuild libvirt-python.
Apparently yum wasn't able to find (or decide in) the correct version of python-django-bash-completion, a dependency for openstack-murano-ui. Going into mock and install manually the version generated by StarlingX, 1.8.14-1, solved the problem.
Kubernetes failed due to not having a newer version of Go, mock was able to find go 1.8, but this kubernetes required 1.9 or newer. Putting the correct Go packages in the mock environment solved the problem.
This package only needed to be rebuilt inside mock and it didn't presented any problem. Not sure what happened.
This was one of the weirdest issues. The configure script in lldpd was failing with:
/usr/lib64/libnetsnmpmibs.so: undefined reference to `headerGetEntry'
It turns out that lldpd links against net-snmp and, for some reason, the net-snmp libraries built here tried to link against a different librpm version, one which has headerGetEntry defined. I solved this by rebuild net-snmp in mock and then lldpd. This magically solved the problem.
python-requests required three packages that weren't not found in with yum.
- python2-httbin
- python2-pytest-httbin
- python2-pytest-mock
Actually the mirror downloader also can't find these packages and those are part of the "3rd party" stuff that we need to download manually.
At this point, with some hacks, a complete build was done. The next step is to try to build the ISO. However, I started to have a lot of problems here.
build-iso uses the generated packages by the build and some other from the mirror. Unfortunately this script relies on having a mirror to build the ISO. I wasn't able (yet) to generate an image with these packages.
It is possible to build StarlingX without the need to download a mirror, however there are issues that we might need to review, like having a well defined repository priorities. Also, although it is possible to build, the ISO creation is still unclear. And more important, even if we are able to build this ISO, we don't know how will work. Anyway it was fun to try this and look for a different way to build the project.
The build-iso script expects to have a mirror