misextended_salt_pillar.md

This note demonstrates need for alternate approach to extend pillar data.

Example

• Salt Master config /etc/salt/master:

  pillar_roots:
      base:
          - /srv/derived_pillars
          - /srv/base_pillars

• Pillar top file /srv/base_pillars/top.sls:

  base:
      '*':
          - pillar_data

• "Base" (read-only) pillar /srv/base_pillars/pillar_data.sls:

  alpha: "base"
  beta: "base"

• "Derived" (extended) pillar /srv/derived_pillars/pillar_data.sls:

  beta: "derived"
  gamma: "derived"

• Result - extended pillar:

  salt-call pillar.items
  local:
      ----------
      beta:
          derived
      gamma:
          derived

Observations

• Configuration key pillar_roots extends filesystem content instead of dictionary keys.

  Filesystem overlay hides entire content of "base" pillar_data.sls file.

  Isn't more fine-grained key/value extension expected for data?

  alpha: "base"
  beta: "derived"
  gamma: "derived"

• Pillar data is accessed through pillar object and does not map naturally into files.

  Unlike Salt states which are referred to as SLS-"modules", Jinja-templates, etc., values in Salt pillars are only reffered as (sub) keys into pillar (nested) dictionary.

  # Salt state location.
  /srv/salt/database/postgresql.sls
  # The only reference to Salt state.
  database.postgresql
  
  # One of the plausible pillar file paths
  # with configuration input for `postgresql` service.
  /srv/pillar/path/to/services.sls
  
  # Reference to compiled pillar data
  # which configure `postgresql` service.
  pillar['services']['postgresql']['enabled']

• Pillar data is merged regardless of its location within filesystem.

  Unlike Salt states which is not comined anyhow by default, pillars are merged by default.

• File-based extension model does not fit for pillars.

  While there are many common mechanics (top file, *.sls templates, various renderrers, etc.) between states and pillars, extention model should rather be based on binding model:

  • States references (effectively) bind to file location:

    # Extending using `file_roots` is natural.
    database.postgresql # /srv/salt/database/postgresql.sls

  • Pillars references (effectively) bind to (sub) keys with compiled data in (nested) dictionaries:

    pillar['services']['postgresql']['enabled']