Skip to content

Instantly share code, notes, and snippets.

@joseivanlopez
Last active October 5, 2023 10:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save joseivanlopez/14671d51715f7d95735ad03c8f79f936 to your computer and use it in GitHub Desktop.
Save joseivanlopez/14671d51715f7d95735ad03c8f79f936 to your computer and use it in GitHub Desktop.

Agama And Product Registation

This document analyses different approaches for registering a product and exposes some possible solutions for implementing registration in Agama.

Installation Vs Registration (Chicken-egg Problem)

An installer like Agama (or YaST) needs to know any repository from which to get the packages of the product to install.

SUSE repositories are only known after registering a product, but SUSEConnect CLI (i.e., the official tool for registering) cannot register a product unless it is already installed. So, how is the system actually installed by YaST?

Summarizing a lot: YaST "workarounds" the registration system. YaST is actually registering the system that is running the installer program. Then, YaST manages to install the product packages on the root file system of the target system (chroot). And finally, YaST copies the repositories and everything needed into the target system.

Options For Registering

There are two options: a) registering before installating a system (YaST way) or b) registering after installing the system.

  • Register a product before installing it.

    • Problems:
      • SUSEConnect expects to work on a system already installed.
      • The installer has to do quite some work in order to properly install the product on the target file system.
      • If the installation is aborted after the registration, then a ghost system is registed in SCC.
    • Advantages:
      • Updates (e.g., security fixes) and add-ons can be directly installed.
      • Add-ons can be selected at installation time.
  • Register a product after installing it.

    • Problems:
      • At least a minimal product repo should be available without registration (by including the repo inside the iso or by providing a public online repo).
      • The minimal repo should be online accesible in order to provide updates (security fixes).
      • Add-ons have to be selected after installing and registering the system (*).
    • Advantages:
      • The system can be directly registered by SUSEConnect, without using any kind of installation magic.
      • SUSEConnect takes cares of adding repos, etc.
      • In case of aborting, no dead system is registered in SCC.
      • Auto-installable images could use the very same approach for registering on first boot.

(*) Note that booting is not required for registering the system. The system could be registered once the installation is done at some path (e.g., /tmp/chroot). SUSEconnect allows to indicate the target root with --root option.

That hypotetical minimal repositories for ALP products do no exist. So, let's assume that Agama is going to implement a similar approach to YaST. That is, the product is registered before installing.

Implementation

We plan to extend the current Software D-Bus service in Agama with a new D-Bus API. That new API would have methods for registering/deregistering a system.

There are three different options to communicate to SCC: a) using SUSEConnect ruby bindings, b) using libsuseconnect.so via FFI to implement rust bindings, c) directly calling to SUSEConnect CLI.

The option b) will not be done yet because the Software service is currently written in ruby. We could opt for that solution if we decide to migrate Software service to rust. Note that creating a separate service for registration probably does not work. Software service holds libzypp lock.

The option c) is also discarded because SUSEConnect CLI cannot register a product unless it is already installed. Note that SUSEConnect always registers the running base product (no way to indicate another product).

So far, SUSEConnect ruby bindings is the best approach that fits our needs. Note that most of the logic we need to implement is already provided by yast2-registration. Nevertheless, yast2-registration supports a lot of use cases that we don't need for Agama, and some parts of its code is coupled to the YaST UI.

Steps for registering

YaST currently does:

# Import certificate (if needed)
# * The certificate is imported into the running system.
# * YaST has to copy it to the target system (e.g., /tmp/chroot) once the system is installed.

SUSE::Connect::YaST.import_certificate(certificate)

# Obtain credentials

login, password = SUSE::Connect::YaST.announce_system(client_params, distro_target)

# Write the global credentials
# * YaST has to copy it to the target system (e.g., /tmp/chroot) once the system is installed.

SUSE::Connect::YaST.create_credentials_file(login, password)

# Register product
# * Repositories are added to the running system by SUSE::Connect (what else is done?).
# * YaST has to copy it to the target system (e.g., /tmp/chroot) once the system is installed.

service = SUSE::Connect::YaST.activate_product(product, client_params, email)

# YaST installs packages on the target file system (e.g., /tmp/chroot)

....

# YaST copies repositories, credentials, certificates (what else?) to the target system.

....

Note that at installation time we don't need to change repositories in the running system. Ideally, we would have a way for calling to #activate_product without doing changes into the system. But somehow it has to return the path of the repositories.

And the same for #deactivate_product. Doing changes in the running system is not needed at installation time.

# Register product (without changes)

service = SUSE::Connect::YaST.activate_product(product, client_params, email)
service.repos #=> ["https://...", ...]

D-Bus API

/org/opensuse/Agama/Software1
    org.opensuse.Agama.Software1
        #AvailableProducts
        #SelectProduct
        #SelectedProduct
        #AvailablePatterns
        #SelectedPatterns
        #AddPattern
        #RemovePattern
        #SetUserPatterns
        #ProvisionsSelected
        #IsPackageInstalled
        #UsedDiskSpace
        #Probe
        #Propose
        #Install
        #Finish

    org.opensuse.Agama1.Registration
        #RegCode (ro)
        #Email (ro)
        #URL (ro)
        #Certificate (ro)
        #State (ro) (0 DISABLED, 1 OPTIONAL, 2 MANDATORY)
        #Register(regcode s, options a{sv}) (options like email, url, certificate)
        #Deregister
@lslezak
Copy link

lslezak commented Oct 2, 2023

That minimal repo should be online accessible in order to install updates (security fixes).

That's not needed. In the current SLE you do not have access to updates unless you register.

Implementation

Alternatively the registration service can be implemented in Rust using libsuseconnect.so via FFI. That's actually exactly what the SUSEConnect Ruby bindings do. Then we just need to extend the Software D-Bus service to allow adding repositories/services and selecting a product to install.

SMT/RMT Support?

The question is whether we should also support registration using an SMT/RMT server. Probably not in the first implementation because it makes things more complicated. It requires UI for entering the server URL, the SMT/RMT instances by default use a self-signed SSL certificate, we need to import it to the installation system and later write to the installed system.

@jreidinger
Copy link

SUSE repositories are only known after registering a product, but SUSEConnect CLI (i.e., the official tool for registering) cannot register a product unless it is already installed. So, how is the system actually installed by YaST?

This statement is not true. It is possible to register it. Lets see how I register my Leap system to SLES ->

jreidinger@localhost:~/prace/yast/skelcd-control-leanos> sudo suseconnect -r <filtered> -p SLES/15.4/x86_64
Registering system to SUSE Customer Center

Announcing system to https://scc.suse.com ...

Activating SLES 15.4 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
command '/usr/bin/zypper --no-refresh --non-interactive install --no-recommends --auto-agree-with-product-licenses -t product SLES' failed
Error: zypper returned 4 with 'Building repository 'SLE-Product-SLES15-SP4-Pool' cache [...done]
Building repository 'SLE-Product-SLES15-SP4-Updates' cache [...done]
Loading repository data...
Reading installed packages...
Resolving package dependencies...
2 Problems:
Problem: the installed product:Leap-15.4-1.x86_64 conflicts with 'namespace:otherproviders(distribution-release)' provided by the to be installed product:SLES-15.4-0.x86_64
Problem: the to be installed product:SLES-15.4-0.x86_64 requires 'product(SLES) = 15.4-0', but this requirement cannot be provided

Problem: the installed product:Leap-15.4-1.x86_64 conflicts with 'namespace:otherproviders(distribution-release)' provided by the to be installed product:SLES-15.4-0.x86_64
 Solution 1: Following actions will be done:
  deinstallation of product:Leap-15.4-1.x86_64
  deinstallation of product:Leap-15.4-1.x86_64
 Solution 2: do not install product:SLES-15.4-0.x86_64

Choose from above solutions by number or skip, retry or cancel [1/2/s/r/c/d/?] (c): c' (exit status 4)

So in fact it is properly registered, it just failed to install SLES product on system as I have already leap there. And I see system correctly at scc.suse.com.

@jreidinger
Copy link

Also note that suseconnect has also --root parameter that can be used to register system on target system...see it on my leap machine ( it failed as I probably do not have proper signing keys ):

jreidinger@localhost:~/prace/yast/skelcd-control-leanos> sudo suseconnect -r INTERNAL-USE-ONLY-28ab-5d40 -p SLES/15.4/x86_64 --root /tmp/chroot
[sudo] password for root: 
Registering system to SUSE Customer Center
Rooted at:/tmp/chroot

Announcing system to https://scc.suse.com ...

Activating SLES 15.4 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
command '/usr/bin/zypper --root /tmp/chroot --no-refresh --non-interactive install --no-recommends --auto-agree-with-product-licenses -t product SLES' failed
Error: zypper returned 104 with 'Error building the cache:
[SUSE_Linux_Enterprise_Server_15_SP4_x86_64:SLE-Product-SLES15-SP4-Pool|https://updates.suse.com/SUSE/Products/SLE-Product-SLES/15-SP4/x86_64/product?K8bZwEiNda7JMoFP4wpYQg8CKAIqvt-UhsAJd36LVaPUF2qHuoXn_4gJ98Ao1vdLynrMC2znJpVAc_BeROXYMsSEEF5XaI1RPq4n5YEcZCiS6RESbD6nvPJzcQwPy7co0DoZY5rOGnZehckhiVbZdXOWeg] Valid metadata not found at specified URL
History:
 - Signature verification failed for repomd.xml
 - Can't provide /repodata/repomd.xml

Error building the cache:
[SUSE_Linux_Enterprise_Server_15_SP4_x86_64:SLE-Product-SLES15-SP4-Updates|https://updates.suse.com/SUSE/Updates/SLE-Product-SLES/15-SP4/x86_64/update?GR8DJ09Rhv2CWRW6SRYqrhX__GJ9-HGdrNQT40lUnQiiP8o2gi72M0ncGVIEG3cjG4SxLHUn4Z8Y2TjgdRMZQp6qII9aGiKLDAPxSWJCk6QF6ze93Uzv35tdt4ZKMWBa_ahy5GfC-LmO1BPdg8fR-hA] Valid metadata not found at specified URL
History:
 - Signature verification failed for repomd.xml
 - Can't provide /repodata/repomd.xml

Some of the repositories have not been refreshed because of an error.
No provider of 'SLES' found.' (exit status 104)

@jreidinger
Copy link

Notes about implementation:

  1. software holds zypp lock, but it does not mean that we cannot have separate registration service. We just need to delete all that software handling to software service instead of doing it in registration itself.
  2. registration does not need to be implemented everywhere like if we have TW only iso, then no registration is there.
  3. as already commented suseconnect CLI supports different CLI. Just not sure if it also supports separation of zypper calls and registration calls.
  4. yast2-registration handles also bunch of different failures that can happen during registration. We need to have way to handle it according to dbus ( so no popups like in https://github.com/yast/yast-registration/blob/master/src/lib/registration/sw_mgmt.rb#L118 )
  5. in fact B and ruby does not make sense as connect ruby bindings is FFI to suseconnect - https://github.com/SUSE/connect-ng/blob/main/yast/lib/suse/connect/yast.rb#L7
  6. big question for implementation is if there will be extensions/modules in ALP same as in SLE15 as it is a lot more API for it.

@joseivanlopez
Copy link
Author

SUSE repositories are only known after registering a product, but SUSEConnect CLI (i.e., the official tool for registering) cannot register a product unless it is already installed. So, how is the system actually installed by YaST?

This statement is not true. It is possible to register it. Lets see how I register my Leap system to SLES ->

Sorry, this parapgraph was still a WIP. I explain below how I see it.

...

So in fact it is properly registered, it just failed to install SLES product on system as I have already leap there. And I see system correctly at scc.suse.com.

If I am not wrong, suseconnect is actually registering your running system (e.g., the live ISO system) as the product you indicated. SUSEconnect performs all the actions (e.g., adding repos, install product package, etc) over that running system. And using --root /tmp/chroot also requires to have a system already installed in the given path.

SUSEconnect documentation states: "NOTE: It will not be possible to register a product unless it is already installed."

IMHO, SUSEconnect is intended to work on an installed system. Doing differently is not the expected way of using it. Of course, we can workaround it, as YaST does.

@joseivanlopez
Copy link
Author

Notes about implementation:

  1. software holds zypp lock, but it does not mean that we cannot have separate registration service. We just need to delete all that software handling to software service instead of doing it in registration itself.
  2. registration does not need to be implemented everywhere like if we have TW only iso, then no registration is there.

This is a good point. Maybe the D-Bus interface for registration should be associated to some kind of product object. So, the product would implement the interface only if registration is required.

  1. as already commented suseconnect CLI supports different CLI. Just not sure if it also supports separation of zypper calls and registration calls.

That is the key. If suseconnect implies zypper calls, then we have a lock problem when calling suseconnect command from a different service. It could even be a problem when calling to suseconnect command from the Software service (I have to try it).

  1. yast2-registration handles also bunch of different failures that can happen during registration. We need to have way to handle it according to dbus ( so no popups like in https://github.com/yast/yast-registration/blob/master/src/lib/registration/sw_mgmt.rb#L118 )

Yes, I fear we need to reimplement something similar. That YaST code is coupled to the UI.

  1. in fact B and ruby does not make sense as connect ruby bindings is FFI to suseconnect - https://github.com/SUSE/connect-ng/blob/main/yast/lib/suse/connect/yast.rb#L7

Ruby? Option b) mentions rust :)

  1. big question for implementation is if there will be extensions/modules in ALP same as in SLE15 as it is a lot more API for it.

Not now, but surely somebody will request it :)

@jreidinger
Copy link

Regarding API:

  1. I think registration interface should not be under Software but standalone as it is not just about software and e.g. in future installer self update API can be added
  2. I thinkg that patterns should not be under Product interface as patterns are not product specific. E.g. in Leap it is shared between Leap, Leap Micro and other possible products.

@joseivanlopez
Copy link
Author

Regarding API:

  1. I think registration interface should not be under Software but standalone as it is not just about software and e.g. in future installer self update API can be added
  2. I thinkg that patterns should not be under Product interface as patterns are not product specific. E.g. in Leap it is shared between Leap, Leap Micro and other possible products.

How are patterns shared? Do all those products use a common patterns repo?

@jreidinger
Copy link

Question for open hands:

  1. is planned to have fully offline installation even without RMT?
  2. is planned to allow customer to modify iso for offline installation?
  3. is planned to allow "full" medium that contain repositories, but allow registration for getting updates?
  4. is installer self-update still wanted feature?
  5. is quaterly update with respin of media planned for ALP or it will be more regular or not at all?

@lslezak
Copy link

lslezak commented Oct 5, 2023

Implementation

To the b) implementation option: that could be possible if we extend the Software service API with methods like AddService, Disable/EnableRepository and similar.

Steps for registering

There is a missing part with extensions/modules:

  1. After registering the base product YaST asks for the available modules/extensions SUSE::Connect::YaST.show_product(product, params).extensions (code)
  2. User can select which ones to register
  3. YaST displays the license (only if the selected extension has one and was not confirmed before)
  4. YaST asks for the registration key (only if the selected extension needs one)
  5. YaST Registers the extension

I'm not sure if it is still relevant for ALP or not. But unless someone makes that decision I'd suppose there will be some extensions, maybe just for some products.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment