gistfile1.txt

TL;DR: how can I make a pillar file, containing a jinja template, seeded from
another pillar file that isn't visible to my minion?  So far I haven't been
able to make one that outputs any data at all.

Grand mal:

I have two minions, "minion-a" and "minion-b".

Two pillars (under ext_pillar): "admins.yml" and "customers.yml".

Both pillar files are in the form:

  users:
    mo:
      [user config]
    larry:
      [user config]
    curly:
      [user config]
    [...]

The "users" formula processes both pillars to write accounts.[1]

On minion-a, we house a service, so both sets of users need to be treated as users.

On minion-b, we keep records of the service.  So accounts in "admins.yml" need to 
be treated as users, but accounts in "customers.yml" need to be visible in a state
that's assigned to that minion (does record keeping), but they can't be treated as
users (that is, they can't be processed by the "users" formula).

As far as I've been able to figure out, if that pillar file is visible to that minion,
then the accounts listed there will show up as accounts.  I haven't been able to 
make states treat data differently based on the file it came from.

My latest swing at solving this problem:

First, write a new file, call it "customer_accounting.yml", and put it in ext_pillars.
I put "customers.yml" out of view of minion-b.

In that file I don't put any actual data, but instead I have a jinja template that
reads "customers.yml", and changes the first line, "users:", to some other string
like "accounts:".  Then, when "customer_accounting.yml" is mapped to "minions-b"
(but "customers.yml" is not), its data will be visible there but under "accounts:"
instead of "users:".

Groovy?  I've been trying to solve this a long time, am I finally on the right
track at least?

Is it even correct to put a jinja template in this context?  All the docs I find talk 
about web pages and state files; do ext_pillar files even work like this?

That said: it's clear that I don't know what I'm doing when I write my jinja.
A few examples of what happens when I run:
  
  salt -E '(minion-b).*' state.sls service.setup test=True

  {% import 'account-base.yml' as user_records | replace(users,accounts,1) %}
...draws:
      Failed to load ext_pillar stack: expected token 'end of statement block', got '|'

  {% import 'account-base.yml' replace(users,accounts,1) %}
...draws:
      Failed to load ext_pillar stack: expected token 'as', got 'replace'

  {% import 'account-base.yml' as accounts | replace(users,accounts,1) %}
...draws:
      Failed to load ext_pillar stack: expected token 'end of statement block', got 'as'

The docs about the difference between "import" and "include" just make my head
swim, so maybe that's my mistake?  Well maybe, but that can't be my whole problem; the
output changes but it still ain't right:

  {% include 'account-base.yml' replace(users,accounts,1) %}
...draws:
      Failed to load ext_pillar stack: expected token 'end of statement block', got 'replace'

  {% include 'account-base.yml' | replace(users,accounts,1) %}
...draws: no error, but when I call highstate, the module that calls this pillar file
doesn't show up at all.

You get my drift; I've tried lots of other variants on the above but I'm just making
random changes at this point and I don't even know if I'm barkingi up the right tree.

UPDATE: over at IRC, ipmb suggested something like:

  {% import 'accounts.yml' as accounts %}
  {% set users = accounts %}
...or maybe:
  {% set users = accounts.accounts %}

I don't follow the meaning, but I tried both ways (along with a few other cargo hacks),
and I couldn't get any "accounts" pillar to show up:

 % sudo salt -E 'minion-b.*' pillar.items
   minion-b:
    ----------

(That's the sum of the output, once I set aside the pieces of pillar that I don't care
about right now.  If I bring back e.g. "users:", my setup does work, so it's not that
I've busted things entirely.)



[1]: A fair question: why use the formula for this, why not just write states that set
up the accounts directly, and tweak the state as needed?  Answer: because I've been
using the formula for a long time (since I started learning Salt), it's running fine
in a lot of places, and this is an exception case.  It would be a lot of work, and
might pose potential for errors, if I rewrite my whole state tree for the sake of this
one issue.