d-installer-cli.md

D-Installer CLI

D-Installer already shipped an initial CLI prototype for managing and driving the installation process. Note that such a CLI was created as a proof of concept, and its current API needs some refactoring. This document is intended to discuss how the new CLI should look like, what patterns to follow, etc.

CLI Guidelines

There already are guidelines for creating modern CLI applications. For example clig.dev defines a guide that is agnostic about programming languages and tooling in general, and it can be perfectly used as reference for D-Installer CLI.

Command name

Some naming recommendations from the guidelines:

• Make it a simple, memorable word
• Use only lowercase letters, and dashes if you really need to
• Keep it short
• Make it easy to type

Currently we have two executable scripts: d-installer for managing the D-Bus services and dinstallerctl for configuring and performing the installation. We would need to re-consider which one should have the ctl suffix. Moreover, dashes are not recommended (dinstaller vs d-installer).

Subcommands

Let's list the recommendations from the guidelines:

• Be consistent across subcommands. Use the same flag names for the same things, have similar output formatting, etc.
• Use consistent names for multiple levels of subcommand. If a complex piece of software has lots of objects and operations that can be performed on those objects, it is a common pattern to use two levels of subcommand for this, where one is a noun and one is a verb. For example, docker container create. Be consistent with the verbs you use across different types of objects.
• Don’t have ambiguous or similarly-named commands. For example, having two subcommands called “update” and “upgrade” is quite confusing.

New CLI

The API of the current CLI is not consistent. It sometimes uses verbs for the subcommand action (e.g., dinstallerctl user clear), and for other subcommands adjectives or noums are used (e.g., dinstallerctl language selected <id>). Moreover, there is a subcommand per each area, for example dinstallerctl language, dinstallerctl software, dinstallerctl storage, etc. Having a subcommand for each area is not bad per se, but for some areas like storage the subcommand could grow with too many actions and options.

The new CLI could be designed with more generic subcommands and verbs, allowing to configure any installation setting in a standart way. Note that the installation process can be already configured by means of a YAML config file with dinstallerctl config load <file>. And the options currently supported by the config file are:

---
product: "Tumbleweed"

languages:
  - "es_ES"
  - "en_US"

disks:
  - /dev/vda
  - /dev/vdb

user:
  name: "test"
  fullname: "User Test"
  password: "12345"
  autologin: true

root:
  ssh_key: "1234abcd"
  password: "12345"

We could extend the config subcommand for editing such a config without the need of a subcommand per area. In general, the config subcommand should have verbs for the following actions:

• To load a YAML config file with the values for the installation.
• To edit any value of the config without loading a new complete file again.
• To show the current config for the installation.
• To validate the current config.

Moreover, the CLI should also offer subcommands for these actions:

• To ask for the possible values that can be used for some setting (e.g., list of available products).
• To start and abort the installation.
• To see the installation status.

Let's assume we will use dinstallerctl for managing D-Bus services and dinstaller for driving the installation (the opposite as it is now). The CLI for D-Installer could look like something similar to this:

$ dinstaller install
Starts the installation.

$ dinstaller abort
Aborts the installation.

$ dinstaller status
Prints the current status of the installation process and informs about pending actions (e.g., if there are questions waiting to be answered, if a product is not selected yet, etc).

$ dinstaller watch
Prints messages from the installation process (e.g., progress, questions, etc).

$ dinstaller config load <file>
Loads installation config from a YAML file, keeping the rest of the config as it is.

$ dinstaller config show [<key>]
Prints the current installation config in YAML format. If a <key> is given, then it only prints the content for the given key.

$ dinstaller config set <key>=<value> ...
Sets a config value for the given key.

$ dinstaller config unset <key>
Removes the current value for the given key.

$ dinstaller config reset [<key>]
Sets the default value for the given <key>. If no key is given, then the whole config is reset.

$ dinstaller config add <list-key> [<key>=]<value> ...
Adds a new entry with all the given key-value pairs to a config list. The key is omitted for a list of scalar values (e.g., languages).

$ dinstaller config delete <list-key> [<key>=]<value> ...
Deletes any entry matching all the given key-value pairs from a config list. The key is omitted for a list of scalar values.

$ dinstaller config check
Validates the config and prints errors

$ dinstaller info <key> [<value>]
Prints info about the given key. If no value is given, then it prints what values are admitted by the given key. If a value is given, then it shows extra info about such a value.

$ dinstaller summary [<section>]
Prints a summary with the actions to perform in the system. If a section is given (e.g., storage, software, ...), then it only shows the section summary.

$ dinstaller questions
Prints questions and allows to answer them.

In those commands <key> represents a YAML key from the config file (e.g., root.ssh_key) and <value> is the value associated to the given key. Note that dots are used for nested keys. Let's see some examples:

# Set a product
$ dinstaller config set product=Tumbleweed

# Set user values
$ dinstaller config set user.name=linux
$ dinstaller config set user.fullname=linux
$ dinstaller config set user.password=linux
$ dinstaller config set user.name=linux user.fullname=linux user.password=12345

# Unset user
$ dinstaller config unset user

# Add and delete languages
$ dinstaller config add languages en_US
$ dinstaller config delete languages en_US

# Set storage settings
$ dinstaller config set storage.lvm=false
$ dinstaller config set storage.encryption_password=12345

# Add and delete candidate devices
$ dinstaller config add storage.candidate_devices /dev/sda
$ dinstaller config delete storage.candidate_devices /dev/sdb

# Add and delete storage volumes
$ dinstaller config add storage.volumes mountpoint=/ minsize=10GiB
$ dinstaller config delete storage.volumes mountpoint=/home

# Reset storage config
$ dinstaller config reset storage

# Show some config values
$ dinstaller config show storage.candidate_devices
$ dinstaller config show user

# Dump config into a file
$ dinstaller config show > ~/config.yaml

# Show info of a key
$ dinstaller info storage.candidate_devices
$ dinstaller info storage.candidate_devices /dev/sda
$ dinstaller info languages

Config file

The current YAML config file needs to be extended in order to support the new storage proposal settings offered by the D-Bus API:

...

storage:
  candidate_devices:
    - /dev/sda
  lvm: true
  encryption_password: 12345
  volumes:
    - mountpoint: /
      fstype: btrfs
    - mountpoint: /home
      fstype: ext4
      minsize: 10GiB

Product Selection

D-Installer can automatically infers all the config values, but at least one product must be selected. Selecting a product implies some actions in the D-Bus services (e.g., storage devices are probed). And the D-Bus services might emit some questions if needed (e.g., asking to provide a LUKS password). Because of that, the command for selecting a product could ask questions to the user:

$ dinstaller config set product=ALP
> The device /dev/sda is encrypted. Provide an encryption password if you want to open it (enter to skip):

Another option would be to avoid asking questions directly, and to request the answer when another command is used (see D-Bus Questions section).

If a product is not selected yet, then many commands cannot work. In that case, commands should inform about it:

$ dinstaller config show
A product is not selected yet. Please, select a product first: dinstaller config set product=<product>.

D-Bus Questions

The CLI should offer a way of answering pending questions. For example, for single product live images the storage proposal is automatically done because the target product is already known. If some questions were emitted during the process, then they have to be answered before continuing using the CLI. Therefore, most of the commands would show a warning to inform about the situation and how to proceed:

$ dinstaller config show
There are pending questions. Please, answer questions first: dinstaller questions.

Non Interactive Mode

Commands should offer a --non-interactive option to make scripting possible. The non interactive mode should offer a way to answer questions automatically. Non interactive mode will be defined later in a following interation of the CLI definition.

Current CLI

As reference, this is the current CLI:

dinstallerctl install # Perform the installation

dinstallerctl config dump            # Dump the current installation config to stdout
dinstallerctl config load <config>   # Load a config file and apply the configuration

dinstallerctl language available           # List available languages for the installation
dinstallerctl language selected [<id>...]  # Select the languages to install in the target system

dinstallerctl rootuser clear                        # Clear root configuration
dinstallerctl rootuser password [<plain password>]  # Set the root password
dinstallerctl rootuser ssh_key [<key>]              # Set the SSH key for root

dinstallerctl software available_products       # List available products for the installation
dinstallerctl software selected_product [<id>]  # Select the product to install in the target system

dinstallerctl storage actions                         # List the storage actions to perform
dinstallerctl storage available_devices               # List available devices for the installation
dinstallerctl storage selected_devices [<device>...]  # Select devices for the installation

dinstallerctl user clear           # Clear the user configuration
dinstallerctl user set <name>      # Configure the user that will be created during the installation
dinstallerctl user show            # Show the user configuration`

How would we unset a value from the config (that is, deleting the whole key/entry)? I see several options:

• using set but having some way to specify nil
• using delete (which is currently documented to only act on the elements of a list)
• adding unset (which would be my preferred way, I miss it)

How would we unset a value from the config (that is, deleting the whole key/entry)? I see several options:

• using set but having some way to specify nil

From examples: dinstaller config set user:name "". But I am not against doing it more explicit with a verb like unset.

I generally like the approach. But I would put some emphasis in the consistency of the config options and the corresponding subcommands to check the possible values.

Examples:

Not exactly the same name:

dinstaller storage-devices list
dinstaller config add storage:candidate_devices /dev/vda

Singular vs plural:

dinstaller products list
dinstaller config set product Tumbleweed

I would even consider to group those queries into a subcommand with the same structure than config

From examples: dinstaller config set user:name "". But I am not against doing it more explicit with a verb like unset.

Deleting a key may not be the same than setting its value to an empty array or to the empty string. Typically an empty value means "none" and an absent one means "default".

dinstaller products list
dinstaller config set product Tumbleweed

I would even consider to group those queries into a subcommand with the same structure than config

Something like this?

$ dinstaller list products
$ dinstaller list storage:candidate_devices
$ dinstaller show storage:candidate_devices <device>

Something like this?

Yes, something like that would make it.

My notes:

1. for dbus I would not use public binary and instead use some non public one ( so not something in path. It does not need to be in path, it just needs full path in dbus config ). So I would remove d-installer from public CLI and just keep dinstaller as CLI name as you suggest.
2. in section Moreover, the CLI should also offer subcommands for these actions: I miss questions subcommands
3. I would consider having --non-interactive global option. E.g. for install or abort it makes sense. As with interactive we can ask for confirmation.
4. is possible to set list in single command? and if so, what is syntax? comma separated or another argument, something like dinstaller config set storage:candidate_devices /dev/sda /dev/sdb
5. If we expect user that they will store with dinstaller show > ~/profile.yaml we should explicitly mention it
6. if you mention that encryption password do you get plain text or encrypted one? I expect that some users will definitively want encrypted one in yaml

2. in section Moreover, the CLI should also offer subcommands for these actions: I miss questions subcommands

Questions are tricky. Maybe all commands should ask for pending questions. Otherwise there is no guarantee that the requested action can be performed. For example:

$ dinstaller config show # this runs a system probing if the system is not probed yet, so questions can be raised.
> The device /dev/sda1 is encrypted. Enter its encryption password (enter to skip): 

4. is possible to set list in single command? and if so, what is syntax? comma separated or another argument, something like dinstaller config set storage:candidate_devices /dev/sda /dev/sdb

Good point. Maybe set can also be used for lists of scalar values. But I am not sure how it would look for complex lists (e.g., storage volumes).

6. if you mention that encryption password do you get plain text or encrypted one? I expect that some users will definitively want encrypted one in yaml

To be honest, I don't know. Maybe it does make sense to use encrypted values in yaml. But, as far as I remember, we don't have support for encrypted password in YaST yet (only for users). We have to define use cases and options.

In general, the proposal looks good. The only command I do not like is config. My main objection is that, at first sight, it is not easy to distinguish between keys and values. I am not a big fan of "positional" arguments for this case.

$ dinstaller config set user:name linux user:fullname linux user:password linux

It takes a little bit of time to "parse" the line for me :-) And if I make a mistake, it is even worse:

dinstaller config set user:name storage:lvm false

Is storage:lvm the value for user:name? OK, we will parse and report a proper error, but the line looks confusing.

After discussing that with @ancorgs, he proposed using an = character. In the beginning, I was a little bit skeptical (I did not remember seeing this syntax in the Linux world so often), but I found out that npm does pretty much the same. And pip does something similar.

npm config set prefix=$HOME/.local/share/npm

Another thing that looks strange to me is using a : to separate the parts of the key. I would go with a . instead.

npm config set init.author.email=user@example.net

Thinking about complex values like the storage volumes I realized it would be useful to have a "partial load". Let me elaborate.

I guess the idea is that the command below will load the configuration from volumes.yaml resetting any value that is omitted there to "null" (that is, to its default value).

dinstallerctl config load volumes.yaml

If volumes.yaml only contains the array defining storage:volumes that will result in resetting the configuration to the system defaults with the only exception of the volumes.

But what if I want to inject my list of volumes (that I keep in a yaml file because is absolutely inconvenient to enter them via CLI arguments) into the existing configuration without affecting the other values that are not part of volumes.yaml?

About using a point (.) as a separator, it is something I have seen in other tools like git, npm or pip:

$ git config user.name
Foo Bar

$ npm config get init.author.email
user@example.net

But what if I want to inject my list of volumes (that I keep in a yaml file because is absolutely inconvenient to enter them via CLI arguments) into the existing configuration without affecting the other values that are not part of volumes.yaml?

I think we can use a flag for that kind of changes in the default command behavior, for example: dinstaller config load --without-defaults volumes.yaml.

About using a point (.) as a separator, it is something I have seen in other tools like git, npm or pip:

$ git config user.name
Foo Bar

$ npm config get init.author.email
user@example.net

I also prefer dinstaller config set storage.lvm=yes user.name=linux vs dinstaller config set storage.lvm yes user.name linux. It is easier to read and less error prone. But I guess we still need a separate key for adding elements to a list:

$ dinstaller config add storage.volumes mountpoint=/home fstype=ext4 minsize=10GiB

But what if I want to inject my list of volumes (that I keep in a yaml file because is absolutely inconvenient to enter them via CLI arguments) into the existing configuration without affecting the other values that are not part of volumes.yaml?

I think we can use a flag for that kind of changes in the default command behavior, for example: dinstaller config load --without-defaults volumes.yaml.

Thinking twice, I have proposed a separated config reset subcommand, which sets the default values.

For loading only the values from a config file:

$ dinstaller config load ~/config.yaml 

For loading values and using defaults for missing keys in the config file:

$ dinstaller config reset
$ dinstaller config load ~/config.yaml