usr_etc_checker.md

Problem

SUSE Linux is moving from using single configuration files to using multiple files. Moreover, some typical configuration files placed at /etc are being moved to /usr/etc, see https://github.com/thkukuk/atomic-updates_and_etc/.

All those changes could make YaST fails. For example, when a YaST module tries to reads/writes a file from /etc but that file is not there anymore.

The main problem for the YaST team is that nobody is noticing in advance when a package configured by YaST changes the location of its configuration files. Simply a new bug is reported to YaST when a YaST module fails because that. This approach would not be a big deal if all that kind of errors are detected by openQA, but we cannot ensure that is the case. Therefore, YaST needs an early defense barrier in order to prevent bug reports from final users as much as possible.

To fix this issue we would need:

• to find out which packages are configured by YaST,
• to detect whether any of that packages are now using /usr/etc,
• to alert YaST team when that change is detected and
• to run this check periodically and automatically.

Possible Solutions

From the four points mentioned above, the two first ones are the most difficult to accomplish:

How to discover which packages are configured by YaST

This could be very complicated because we need to do an exhaustive research through the code of all YaST modules. This is specially difficult for the code using SCR agents.

How to detect whether a package is using /usr/etc

For this we could list the files of a rpm (i.e., rpm -ql package-name) and check whether any of that files is under /usr/etc. Maybe this does not catch all cases, for example when a package creates a config file by means of a post install script, but it would be a good first approach.

Proposal A

A possible solution could be to install all yast* packages, and also all possible packages installed by YaST at some point. For example, that packages installed by a kind of check_and_install method when starting a module. For this we need to do a search in the YaST code. All those packages should be taken from the Factory project to ensure we are checking the packages before reaching any release.

Once we have a system with all those packages, then we can check whether any package in the system provides a /usr/etc file. If so, an email would be sent to yast-internal. We also need a kind of white list to filter out the packages we know that YaST does not configure or the packages for which YaST was already adapted. That list would be filled and updated manually by us.

Such a script could be run periodically by Jenkins in a docker container.

For the record (I mentioned this on our daily call):

AppArmor has a "complain" mode where it does not simply block any access to a file or directory, but just records that there was such an access. If we run YaST inside AppArmor in that mode, we could later check what files were accessed.

Then of course we'd need to find out whether or not that access was okay or not.

About finding out what packages make the transition from /etc to /usr/etc:

Use an up-to-date Tumbleweed and monitor new files that appear in /usr/etc. From then on, it's a matter of rpm -qf for that file to find out what package it belongs to. That would also cover files created by post-install scripts, fillup calls and whatever.

Since nobody has all packages installed, that might leave leaks; but then, we are not exactly touching all packages with YaST, either. We'd have to make sure to have the packages installed that we allow to configure with YaST, of course.

For the record (I mentioned this on our daily call):

AppArmor has a "complain" mode where it does not simply block any access to a file or directory, but just records that there was such an access. If we run YaST inside AppArmor in that mode, we could later check what files were accessed.

Then of course we'd need to find out whether or not that access was okay or not.

But IIUC, somehow we would need to run all YaST modules in order to generate that AppArmor access complaintment, right? If so, I would say it is not much doable.

About finding out what packages make the transition from /etc to /usr/etc:

Use an up-to-date Tumbleweed and monitor new files that appear in /usr/etc. From then on, it's a matter of rpm -qf for that file to find out what package it belongs to. That would also cover files created by post-install scripts, fillup calls and whatever.

Yes, maybe it would be better to check for package ownership of /usr/etc files instead of searching for /usr/etc files inside rpms.

Since nobody has all packages installed, that might leave leaks; but then, we are not exactly touching all packages with YaST, either. We'd have to make sure to have the packages installed that we allow to configure with YaST, of course.

Proposal B

• Parse the Factory repository metadata and extract the file names from the *-filelists.xml.gz file.
• This file list not only contains the real files but also the ghost files (with the <file type="ghost"> tag), so if the RPM is created correctly even the files created by %post scripts will be found
• The advantage is that you do not need to install the packages at all and you can run it in any system (not only in the latest Factory)
• Some time ago I created this parsing PoC script https://github.com/lslezak/scripts/blob/master/repo/etc_files.rb which parses an filelists.xml.gz file and prints all /usr/etc/ files
• It needs some improvement, e.g. it needs that *-filelists.xml.gz file already downloaded. Because that file name is changed after each repository update we need to also parse the main repomd.xml index file. We already do something similar to get the *-primary.xml.gz file name here: https://github.com/yast/yast-packager/blob/master/src/lib/y2packager/repomd_downloader.rb#L75, so that should be quite easy.
• Using a white list filtering from Proposal A would also apply here