yast-libyui-build-system.md

YaST libyui Build System Requirements

Current Situation

Custom CMake with a convoluted, hard to understand set of centralized .cmake include files and a toplevel CMakeList.txt that is symlinked to the central one somewhere in a system directory. In each project repo, information is scattered among a dozen file snippets that are magically included from somewhere.

Nobody understands the control flow of that, so everybody avoids touching that, so even obvious problems are never cleaned up.

Goal for the Near Future

• Use plain CMake, not voodoo magic hidden in scripts out of reach and out of touch

• Conform to CMake standards (i.e. do it by the book - literally)

• Standard CMake documentation should be enough to understand what's going on

• Simple standard CMake control flow, not triggered from some outside script

• Don't overengineer with dozens of artificially introduced variables that only obscur what's going on

• We don't need to have even the simplest things configurable (which nobody ever did anyway in all the history of libyui and YaST)

• Avoid tiny files with important information; only if there is no other reasonable way to achieve the same goal. We might still need a VERSION.cmake, but not all the other tiny files that are so easy to overlook, yet constantly need editing.

• Keep it simple (KISS)

Requirements

(using CMake here as a general term for $BUILDSYSTEM_OF_CHOICE; it might be something different in the future. We don't plan to, but you can never know.)

Required Operations

• make (build, i.e. compile and link)

• sudo make install

• Create a source tarball (doesn't need to be with make tarball)

  Update: Our rake setup already does this (rake package, rake tarball).

Nice to Have

• Build class documentation in HTML (doxygen) (formerly make doc)

  • Building documentation should be optional so it doesn't always delay a build during normal development

  • It should be disabled by default, only enabled with some special build commands in the .spec file

  • Don't overengineer doc stuff like creating LaTeX documentation (nobody will ever print it anyway)

• One source directory, one CMake file (so the control flow is clear and simple)

• In the toplevel project directory, only descend into subdirectories; no fancy voodoo stuff that developers are reluctant to ever touch

• Build examples that need to be compiled; also in a separate directory with a separate CMake file (libyui only, the others are in interpreted languages like Ruby)

• Execute a test suite if there is one (make test); also in a separate directory with a separate CMake file

• Subdirectories like doc/, examples/, test/ should be self-sufficient so we can build / execute each one separately if needed

• Subdirectories refer to their sibling src/ directory, not to a global system directory: We want to test the code we are currently working on even without having to execute sudo make install for every minor edit

• Work well with out existing CI systems (Jenkins) with only minimal changes; we still need it to support automatically building, executing test suites, creating tarballs and submitting source packages to OBS

• Allow setting a prefix so users can build their own version in /usr/local instead of our standard /usr

  We discussed that and found it of questionable value in these days of virtual machines, dockers and similar technologies.

Keeping the libyui Community Happy

There are some projects that are maintained outside the YaST team and SUSE that are using libyui:

• libyui-gtk https://github.com/libyui/libyui-gtk

• manatools (Mageia Linux system management tools):

  • dnfdragora, the DNF-based Mageia software management UI tool: https://github.com/manatools/dnfdragora

  • Additional libyui-based widgets for those manatools:

    • https://github.com/manatools/libyui-mga
    • https://github.com/manatools/libyui-mga-gtk
    • https://github.com/manatools/libyui-mga-qt
    • https://github.com/manatools/libyui-mga-ncurses

We don't want to break their build system, so we will keep the existing one intact; they can continue to use it without needing to change their build infrastructure. However, we will stop using it and migrate to a new one that meets the above requirements.

We will keep the existing buildtools/ directory in the libyui source repository (https://github.com/libyui/libyui/tree/master/buildtools) and still install those files to the target system upon sudo make install in libyui, so they can still use the existing bootstrap.sh script in their toplevel project directories; that script creates a toplevel CMakeLists.txt as a symlink to /usr/share/libyui/buildtools/CMakeLists.common. That is the starting point for the existing libyui cmake buildsystem logic.

After a transitional period (months, if not years) when we found the new build logic stable enough, we will approach them and assist them to migrate their projects to the new one as well, using our existing subprojects as templates how to do that.

We can keep the old buildtools/ working for quite a while, but eventually bit rot will set in as the outside world (including CMake and related tools) changes, so in the long run a migration will be inevitable.

But we want that to be a smooth, well-understood and and well-supported process in close cooperation between our libyui community and us, the YaST team.

For readability, we should follow a consistent style in the CMakefiles: https://community.kde.org/Policies/CMake_Coding_Style

About prefix (/usr/local), I used to like and use this, but found it hard to maintain and felt I was the only user. Nowadays I find it easier to blast over /usr in a container or in a VM.
Who is using it? Speak up.

For readability, we should follow a consistent style in the CMakefiles: https://community.kde.org/Policies/CMake_Coding_Style

👍

Yes; that was part of what I meant with "conform to CMake standards".

@mvidner i don't know if you mean default prefix or configurable one... i'm used to check as much as possible into my home instead of root, i mean user space.
To do that i set cmake macros like:

-DCMAKE_MODULE_PATH=$HOME/local/share/cmake/Modules 
-DYPREFIX=$HOME/local 
-DCMAKE_INSTALL_PREFIX=$HOME/local

and for testing:

INST_DIR=$HOME/local
export LD_LIBRARY_PATH=${INST_DIR}/$lib/:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=${INST_DIR}/$lib/pkgconfig:$PKG_CONFIG_PATH
export PATH=${INST_DIR}/bin:${PATH}

I also install libyui-bindings locally into my home using virtualenv for python for instance.

I think that should still work

OverlayFS Instead of Prefix?

To Do:

Check if OverlayFS would be a good alternative.
https://en.wikipedia.org/wiki/OverlayFS

Creating a Package Tarball

This is provided by rake package which calls rake tarball:

https://github.com/openSUSE/packaging_rake_tasks/blob/master/lib/tasks/tarball.rake from package
ruby2.7-rubygem-packaging_rake_tasks. This is a little more fancy than a plain git archive call, but not by very much.

So we don't need to replicate this in our CMake setup.