Skip to content

Instantly share code, notes, and snippets.

@waylan
Last active October 8, 2021 13:17
Show Gist options
  • Save waylan/703743f0d5caa6651e1036e310fe1dac to your computer and use it in GitHub Desktop.
Save waylan/703743f0d5caa6651e1036e310fe1dac to your computer and use it in GitHub Desktop.
Proof of concept for yaml inherit tag. Unfortunately, defining the tag is ugly. This was based on the code in [this comment](https://gist.github.com/joshbode/569627ced3076931b02f#gistcomment-2309157).
# Unforunately, the tag can't be a standalone item by itelf.
# After all, the following document is invalid:
# foo
# a: bar
# Therefore the tag needs to fit in with the structure of the document.
# I have used a null key and then discard it in the loader
# so that it is not in the final output.
null: !inherit parent.yml
b: baz
c: bar
import yaml
import os.path
def merge(a, b):
""" Merge two documents. TODO: implement deep merge"""
a.update(b)
return a
class InheritLoaderMeta(type):
def __new__(metacls, __name__, __bases__, __dict__):
"""Add include constructer to class."""
# register the inherit constructor on the class
cls = super().__new__(metacls, __name__, __bases__, __dict__)
cls.add_constructor('!inherit', cls.construct_inherit)
return cls
class InheritLoader(yaml.Loader, metaclass=InheritLoaderMeta):
"""YAML Loader with `!inherit` constructor."""
def __init__(self, stream):
"""Initialise Loader."""
try:
self._root = os.path.split(stream.name)[0]
except AttributeError:
self._root = os.path.curdir
self._parent_data = None
super().__init__(stream)
def construct_inherit(self, node):
"""load parent file referenced at node."""
filename = os.path.abspath(os.path.join(
self._root, self.construct_scalar(node)
))
extension = os.path.splitext(filename)[1].lstrip('.')
with open(filename, 'r') as f:
if extension in ('yaml', 'yml'):
self._parent_data = yaml.load(f, InheritLoader)
else:
self._parent_data = ''.join(f.readlines())
return ''
def construct_document(self, node):
""" Merge parent with document."""
doc = super().construct_document(node)
parent = self._parent_data
if parent is not None:
# Reset parent data for a multi-document file
self._parent_data = None
if None in doc:
# remove empty None key
doc.pop(None)
return merge(parent, doc)
return doc
if __name__ == '__main__':
with open('example.yml', 'r') as f:
data = yaml.load(f, Loader=InheritLoader)
print(data)
a: foo
b: bar
@J3ronimo
Copy link

J3ronimo commented Oct 8, 2021

Hey waylan,

for a project I also ended up writing a yaml parser like yours that supports inheritance. For the declaration of the base file I chose this:

!extend : base.yml

So the value after the constructor is simply None, and then the parser knows it has to evaluate what's been assigned to this key.
Although one tends to forget the ":" in between, which leads to confusing error messages, it's quite clean and readable. And the wording "extend" is taken directly from jinja2, which I hoped would be intuitive for users that know it.

Nice to see I'm not the only one missing a de-facto tool for DRY and extendable config files. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment