Manageable salt state independence

salt.markdown

I'm trying to determine how I might be able to keep a group of states (modules) from having direct knowledge about each other, while still allowing them to put and get configuration information on a local-to-the-minion level.

For example, my nginx related states and config should have no knowledge about logstash and where it might put a config file. Instead I want the logstash to make it known (how is the question) that it has a log file available for consumption.

Say I have a logstash process running on a machine and it's configuration looks something like the below. Right now it's going to read in just one log file, but in just a bit I want it to read in any log file that another service, managed via a role/state, might add.

/etc/logstash/agent.conf

input {
  file {
    path => "/var/log/system.log"
  }
}
output {
  elasticsearch { host => el1.example.com }
}

Now I'm going to start up a Nginx server via a state where the config it writes looks something this.

/etc/nginx/server.conf

error_log /var/log/nginx/error.log;

http {
    server {
        listen  80;
        return  503;
    }
}

Is there a way that I could modify my logstash/agent.conf template to be something like this?

salt://module/logstash/template/agent.conf.jinja2

input {
  file {
    path => "/var/log/system.log"
  }
  {% for log_file in salt['magic.function']('local:log_files') %}
  file {
    path => "{{ log_file }}"
  }
  {% endfor %}
}
output {
  elasticsearch { host => el1.example.com }
}

My understanding currently is:

• Grains: I could make a list of all the files I care about as grain info, but now I would have to keep that list up to date in both the original state and the grains.
• Pillar: Same goes for trying to store it in pillar.
• Salt Mine: Doesn't seem like it would work either as I need multiple states to add to my 'local:log_files' list and it looks like it just does a replace. This is something I'm still looking into, despite it seeming to emphasize sharing state with other minions.
• Returners: The sqlite3 looks interesting since I don't need to share this particular data with other minions, but I'm not sure if I could use the data it saves in a jinja template without some other process involved. Also, how would I 'return' the log file locations.
• Event: I don't think this could be used for this, but I could be wrong.
• Overstate: I imagine this is what I will use to get all the services up and running, and then have it run the state that manages logstash and will be able to read in all the 'local:log_files' data for that host.

Other ideas I've had, and somewhat tried, were Jinja templates/macros. They would look something like the following.

salt://module/logstash/macro.sls

{% macro log_file(salt, short_name, log_file_location) %}
logstash-{{ log_file_location }}:
    file.managed:
        name: /etc/logstash/10-{{ short_name }}.conf
        source: salt://module/logstash/template/input.conf
        context:
            file_path: {{ log_file_location }}
{% endmacro %}

salt://module/logstash/template/input.conf

input {
  file {
    path => "{{ log_file_location }}"
  }
}

And so then in whatever state I'm creating my nginx server in, I'd also be importing and then calling {% logstash.log_file(salt, '503-as-a-service', '/var/log/system.log') %}.

In addition to not even solving the issue states having direct knowledge of other states, I really dislike the jinja template/macro route because the macros become far too complex once things like grains and conditionals start getting mixed in.

Yet another idea I'm considering is a custom execution module/state that would have a something like a present(key, value, append=False) module that would act like a mix of mine and the sqlite3 returner. However that is a much more daunting task.

So that leaves me here… unsure of how to reach my goal as every solution feels unnatural and/or complicated.

Thanks for reading!

jasonrm