How to find faulty file causing parsing error

[sylvainfaivre]

Sorry, this is more of a discussion topic than a real bug.

We use PillarStack with a fairly complicated config, with ext_pillar:stack loading 3 main files, each of these referencing many other files.

If I make a mistake in one of these files, calls to the pillar module give the line & column numbers where the error lies within a file, but the file name is not printed, which makes debugging very hard.

Is there a way to find out which file caused the error ?

Here is an example :

user@salt $ salt minion1 saltutil.refresh_pillar
minion1:
    True


-------------------------------------------
Summary
-------------------------------------------
# of minions targeted: 1
# of minions returned: 1
# of minions that did not return: 0
# of minions with errors: 0
-------------------------------------------

user@salt  $ salt minion1 pillar.items
minion1:
    ----------
    _errors:
        - Failed to load ext_pillar stack: while parsing a block node
          did not find expected node content
            in "<unicode string>", line 31, column 17


-------------------------------------------
Summary
-------------------------------------------
# of minions targeted: 1
# of minions returned: 1
# of minions that did not return: 0
# of minions with errors: 0
-------------------------------------------

No help from the salt master log file either :

2023-07-27 14:03:50,962 [salt.pillar      :1166][ERROR   ][173946] Exception caught loading ext_pillar 'stack':
  File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 1158, in ext_pillar
    ext = self._external_pillar_data(pillar, val, key)
  File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 1088, in _external_pillar_data
    ext = self.ext_pillars[key](self.minion_id, pillar, *val)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/pillar/stack.py", line 414, in ext_pillar
    stack = _process_stack_cfg(cfg, stack, minion_id, pillar)
  File "/usr/lib/python3/dist-packages/salt/pillar/stack.py", line 457, in _process_stack_cfg
    obj = salt.utils.yaml.safe_load(
  File "/usr/lib/python3/dist-packages/salt/utils/yamlloader.py", line 178, in safe_load
    return yaml.load(stream, Loader=Loader)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "ext/_yaml.pyx", line 707, in _yaml.CParser.get_single_node
  File "ext/_yaml.pyx", line 725, in _yaml.CParser._compose_document
  File "ext/_yaml.pyx", line 776, in _yaml.CParser._compose_node
  File "ext/_yaml.pyx", line 890, in _yaml.CParser._compose_mapping_node
  File "ext/_yaml.pyx", line 776, in _yaml.CParser._compose_node
  File "ext/_yaml.pyx", line 890, in _yaml.CParser._compose_mapping_node
  File "ext/_yaml.pyx", line 776, in _yaml.CParser._compose_node
  File "ext/_yaml.pyx", line 890, in _yaml.CParser._compose_mapping_node
  File "ext/_yaml.pyx", line 776, in _yaml.CParser._compose_node
  File "ext/_yaml.pyx", line 890, in _yaml.CParser._compose_mapping_node
  File "ext/_yaml.pyx", line 776, in _yaml.CParser._compose_node
  File "ext/_yaml.pyx", line 890, in _yaml.CParser._compose_mapping_node
  File "ext/_yaml.pyx", line 732, in _yaml.CParser._compose_node
  File "ext/_yaml.pyx", line 905, in _yaml.CParser._parse_next_event

2023-07-27 14:03:50,963 [salt.pillar      :1224][CRITICAL][173946] Pillar render error: Failed to load ext_pillar stack: while parsing a block node
did not find expected node content
  in "<unicode string>", line 31, column 17