agama-auto-storage.md

Agama Auto-installation (Storage Settings)

Format specification

Storage
  drives <Drive[]>
  volume_groups <VolumeGroup[]>
  software_raids <SoftwareRaid[]>
  bcache_devices <BCache[]>
  nfs_devices <NFS[]>
  btrfs_raids <BtrfsRaid[]>
  guided <Guided>

Drive
  search [<Search>]
  alias [<string>]
  encrypt [<EncryptAction>]
  format [<FormatAction>]
  mount [<MountAction>]
  ptable_type [<string>]
  partitions [<Partition[]>]

EncryptAction
  method <string>
  key [<string>]
  pdkdf [<string>]
  label [<string>]
  cipher [<string>]
  key_size [<number>]

FormatAction
  filesystem <string|Btrfs>
  label [<string>]
  mkfs_options [<string[]>]

Btrfs
  subvolume_prefix [<string>]
  subvolumes [<Subvolume[]>]
  snapshots [<boolean=false>]
  quotas [<boolean=false>]

MountAction
  path <string>
  mount_options [<string[]>]
  mount_by [<string>]

Partition
  search [Search]
  alias [<string>]
  id [<string>]
  type [<string>]
  size [<Size>]
  encrypt [EncryptAction]
  format [<FormatAction>]
  mount [<MountAction>]
  delete [<boolean=false>]

VolumeGroup
  search [<Search>]
  alias [<string>]
  name [<string>]
  pesize [<number>]
  physical_volumes [<string[]>]
  logical_volumes [<LogicalVolume[]>]
  delete [<boolean=false>]

LogicalVolume
  search [<Search>]
  alias [<string>]
  name [<string>]
  size [<Size>]
  pool [<boolean>]
  used_pool [<string>]
  stripes [<number>]
  strip_size [<number>]
  encrypt [<EncryptAction>]
  format [<FormatAction>]
  mount [<MountAction>]
  delete [<boolean=false>]

SoftwareRaid
  search [<Search>]
  alias [<string>]
  name [<string>]
  level [<string>]
  chunk_size [<number>]
  devices [<string[]>]
  encrypt [<EncryptAction>]
  format [<FormatAction>]
  mount [<MountAction>]
  ptable_type [<string>]
  partitions [<Partition[]>]
  delete [<boolean=false>]

BtrfsRaid
  search [<Search>]
  data_raid_level <string>
  metadata_raid_level <string>
  devices <string[]>
  label [<string>]
  mkfs_options [<string[]>]
  [Btrfs]
  delete [<boolean=false>]

Size <'default'|string|SizeRange>

SizeRange
  min <string>
  max <string>

Search
  condition [<Condition>]
  sort [<Sort>]
  min [<number>]
  max [<number>]
  if_not_found [<NotFoundAction='skip'>]

Condition <Rule|OperatorAnd|OperatorOr>

OperatorAnd
  and: <Condition[]>

OperatorOr
  or: <Condition[]>

Rule
  property <string>
  value <any>
  operator [<Operator='equal'>]

Operator <'equal'|'not_equal'|'less'|'greater'|'less_or_equal'|'greater_or_equal'>

Sort
  property <string>
  order <'asc'|'desc'>

NotFoundAction <'continue'|'skip'|'error'>

Examples

Format and mount a disk

storage: {
  drives: [
    {
      format: {
        filesystem: 'xfs',
        label: 'data'
      },
      mount: {
        path: '/home'
      }
    }
  ]
}

Create partitions

storage: {
  drives: [
    {
      ptable_type: 'gpt',
      partitions: [
        {
          size: '10GiB',
          format: { filesystem: 'btrfs' },
          mount: { path: '/' }
        },
        {
          size: '50GiB',
          format: { filesystem: 'xfs' },
          mount: { path: '/home' }
        }
      ]
    }
  ]
}

Create a volume group

storage: {
  drives: [
    {
      ptable_type: 'gpt',
      partitions: [
        {
          alias: 'pv1',
          size: '10GiB'
        },
        {
          alias: 'pv2',
          size: '50GiB'
        }
      ]
    }
  ],
  volume_groups: [
    {
      name: 'vg0',
      physical_volumes: ['pv1', 'pv2'],
      logical_volumes: [
        {
          name: 'home',
          size: '10GiB',
          format: { filesystem: 'btrfs' },
          mount: { path: '/' }
        }
      ]
    }
  ]
}

Format and configure a Btrfs

storage: {
  drives: [
    {
      ptable_type: 'gpt',
      partitions: [
        {
          size: '10GiB',
          format: {
            btrfs: {
              subvolumes: [
                { path: '/home', nocow: true }
              ],
              snapshots: true
            }
          },
          mount: { path: '/' }
        }
      ]
    }
  ]
}

Search a device

storage: {
  drives: [
    {
      // Search for a device with size >= 5GiB and it is not '/dev/vda'.
      search: {
        condition: {
          and: [
            { property: 'size', value: '5GiB', operator: 'greater_than' },
            { property: 'name', value: '/dev/vda', operator: 'not_equal' }
          ]
        }
      },
      format: { filesystem: 'xfs' }
    }
  ]
}

Delete all partitions

storage: {
  drives: [
    {
      partitions: [
        // Search without condition, so everything is selected.
        { search: {}, delete: true }
      ]
    }
  ]
}

Mount an existing partition

storage: {
  drives: [
    {
      partitions: [
        {
          // First partition with XFS file system
          search: { property: 'filesystem', value: 'xfs', max: 1 },
          mount: { path: '/home' }
        },
        {
          // And also create a new partition
          size: '10GiB'
        }
      ]
    }
  ]
}

Delete the 3 first volume groups

storage: {
  volume_groups: [
    {
      { search: { max: 3 }, delete: true }
    }
  ]
}

Find a disk or use any

storage: {
  drives: [
    {
      search: {
        condition: { property: 'size', value: '10GiB', max: 1 },
        if_not_found: 'continue'
      },
      format: { filesystem: 'xfs' }
    }
  ]
}

Reuse or create a partition

storage: {
  drives: [
    {
      partitions: [
        {
          // Try to reuse, but continue if not found.
          search: {
            condition: { property: 'filesystem', value: 'xfs' },
            if_not_found: 'continue'
          },
          size: '10GiB',
          format: { filesystem: 'xfs' }
        }
      ]
    }
  ]
}

Condititional search

Create a script:

#!/bin/sh
# check_intel.sh

if grep -i intel /proc/cpuinfo > /dev/null; then
  echo -n "intel"
else
  echo -n "non_intel"
fi;

Pass script output as a jsonnet external variable:

$ jsonnet --ext-str cpu="$(check_intel.sh)" profile.jsonnet

Add search section if needed:

storage: {
  drives: [
    {
      [if std.extVar('cpu') == 'intel' then 'search']: {
        condition: { property: 'size', value: '10GiB', max: 1 }
      },
      format: { filesystem: 'xfs' }
    }
  ]
}

Note 1: Additionally, it would be also great to have the ability to select the "fastest" disk, but as lshw does not provide that info, something like hdparm -t /dev/device would have to be used, and each such call takes ~6.2 sec on my system.

Note 2: Some of our customers/partners have written very sophisticated AY scripts, but they seem to be mainly about partitioning - something that AY was not able to do or they did not know how. It would be great to compare if the suggested way is actually simpler than using a script.

Note 1: Additionally, it would be also great to have the ability to select the "fastest" disk, but as lshw does not provide that info, something like hdparm -t /dev/device would have to be used, and each such call takes ~6.2 sec on my system.

Jsonnet could be used, for example:

{
  conditions: { fact: 'name', value: find_fastest_disk() },
  partitions: [...]
}

I don't know if jsonnet allows to "embed" script code, but at least we can inject the output of some script as parameter when compiling the jsonnet file.

It seems we could do something like the following:

• Use std.extVar to get values from command line:

{
  conditions: { fact: 'name', value: std.extVar('fastest_disk') },
  partitions: [...]
}

• Pass the variable when compiling, fidding it with a script output:

jsonnet --ext-str fastest_disk="$(find_fastest_disk.sh)" profile.jsonnet

I'm still thinking about conditions->fact. For me, the name "fact" is somehow questionable. It IMO rather something like "object" or "subject" or even "quality" (not really).

I'm still thinking about conditions->fact. For me, the name "fact" is somehow questionable. It IMO rather something like "object" or "subject" or even "quality" (not really).

Yes. The syntax for searching devices is still under discussion. In fact, the lastest proposal is to use the "search" keyword:

{
  "search": {
    "condition": { ... },
    "short": { ... }
  }
}

And there could be different ways to express a rule (we have to decide one):

"condition": { "name": "/dev/vda" } 

"condition": { "property": "name", "value": "/dev/vda" } 

BTW, probably this document will become obsolete in some parts. @ancorgs already started writing a document describing the format and agreements, see https://github.com/openSUSE/agama/pull/1256/files.