Provide an include mecanism

[arthurzenika]

It would be nice if pillarstack had a similar include mecanism that base pillars have.

Our use case is that we have role based pillar that "infer" another role. (For example nginx role and apache role both should include "certificates" role) In a traditional pillar setting we would just have an include in both included roles. With pillarstack we are having trouble finding and elegant way of doing this (import_as_yaml is not very nice, and loops and ifs in the config.cfg aren't much nicer).

[daks]

Not sure it answers your question, but we use Jinja to do includes like this

{% include "path/file.yml" %}

[ze42]

With {% include ... %}, it would be included inline. Too many limitations:

• If you include 2 files with the same top-level keys, only one would properly be used.
• For a single key, you can't have one file delete a few items, and an other add some other extra items.

To have include really work, it would need to dynamicly add files like to be included like they would have been present in the top-level file.

As stack is already having special-key named __, maybe having something named __include on top of the file could help list files to include.

An alternative I'm thinking about right now, would be that if top-level is a list (and not a dict), have it merge each element one after an other. That would also allow to add elements, while deleting a top-level key from a single file (currently, I have to do it with different files) -- but probably too complicated, and that could lead to too many errors.

[mrichar1]

I'd like to add my support to this request.

We have lots of places where we use include in pillars to make collections/groupings of config, which can then be included in a DRY way across multiple nodes, e.g. things like:

roles/lamp.sls

include:
  - apache
  - mysql
  - php

The only way I've found of replicating this in pillarstack is by using symlinks and globs, i.e.:

ln -s roles/apache.sls roles/mysql.sls roles/php.sls lamp/ and then doing:

stack.cfg

{%- for role in pillar.get('roles', []) %}
roles/{{ role }}/*.sls
{%- endfor %}

Which is a bit of a hacky way of doing it.

[jgraichen]

You can give salt-tower a try. It works similar to pillarstack and supports an include mechanism (along with a few other things). We are using it for quite some time now in a few production setups.

Disclaimer: I'm the author. Feel free to ask for guidance or take a look at the examples.

[Sjd-Risca]

ln -s roles/apache.sls roles/mysql.sls roles/php.sls lamp/ and then doing:

stack.cfg

{%- for role in pillar.get('roles', []) %}
roles/{{ role }}/*.sls
{%- endfor %}

Which is a bit of a hacky way of doing it.

I do usually do the opposite, see following example.

stack.cfg

minions/{{ minion_id }}-base.yml
roles/*.yml

minions/myminion.yml

roles:
  - apache
  - mysql
  - php

roles/apache.yml

{%- if 'apache' in __stack__['traverse'](stack, 'roles', []) %}
{# custom pillars #}
{%- endif %}

and so on...

---

It is possible to have more complex infers like:

stack.cfg

minions/{{ minion_id }}-base.yml
roles/infer.yml
roles/*.yml

minions/myminion.yml

roles:
  - lamp

roles/infer.yml

{% 'lamp' in __stack__['traverse'](stack, 'roles', []) %}
roles:
  - apache
  - mysql
  - php
{% enfid %}

roles/apache.yml

{%- if 'apache' in __stack__['traverse'](stack, 'roles', []) %}
{# custom pillars #}
{%- endif %}

Given such flexibility I don't know if an include statement is required at all.