Inherited properties in TOML

explicit.toml

[thing]
name = "demo"

[thing.version_a]
items = ['base1', 'base2', 'first1', 'first2']

[thing.version_b]
items = ['base1', 'base2', 'second1', 'second2']

implicit.toml

[thing]
name = "demo"
items = ['base1', 'base2']

[thing.version_a]
items = ['first1', 'first2']

[thing.version_b]
items = ['second1', 'second2']

inherit.toml

[thing]
name = "demo"
items = ['base1', 'base2']

[thing.version_a]
items = ['+', 'first1', 'first2']

[thing.version_b]
items = ['+', 'second1', 'second2']

[thing.version_c]
items = ['third1', 'third2']

separate.toml

[thing]
name = "demo"
items = ['base1', 'base2']

[thing.version_a]
sub_items = ['first1', 'first2']

[thing.version_b]
sub_items = ['second1', 'second2']

The problem

TOML allows the specification of lists (arrays). It also allows for the definition of hierarchical sections in the TOML document (as Tables).

I have a use case where I need to specify a list that is composed hierarchically. Elements in a table will share common elements; a collection of elements with a common base definition, extended as part of the table structure. The final parsed data will look something like:

• thing.version_a.items = ['base1', 'base2', 'first1', 'first2']
• thing.version_b.items = ['base1', 'base2', 'second1', 'second2']

Since 'base1' and 'base2' are common elements, I'm looking at a conventions I can use in my configuration file to allow users to avoid repetition.

I can see four possible alternatives, demonstrated above, and explained below. Which do you prefer?

Explicit

Don't do anything fancy. Require the repeated elements to be repeated. This is long-winded, but explicit.

Implicit

Internally, handle "items" as a key that is known to be hierarchical. When processing thing.version_a.items, look for thing.items as well, and concatenate the two values.

This allows for factoring out the common elements, but it requires whatever handles the configuration file to know that "items" is a special value with a hierarchical interpretation. This means any "special" keys will need to be pre-coded in the configuration parser.

Inheritance

Define a special value (+) that is interpreted as "look up one level for the same key, and extend this list with the value of the parent level".

This allows for factoring out the common elements, but it requires no special handling of the key. The configuration file would need to understand how to handle + as a value, but any key could use this syntax without pre-coding in the configuration parser.

The use of + could be restricted to the first element of the list; this would allow + to be a literal in the list itself. Alternatively + could be a placeholder that is interpreted as "insert the parent list" - so, items = ['first1', '+', 'first2'] would be interpreted as ['first1', 'base1', 'base2', 'first2']

This use case also allows you to not include the parent element, so thing.version_c.items = ['third'] becomes possible.

Separate

Use separate keys for the lists at each level. This is similar to the implicit approach, in that whatever uses the configuration file would need to be aware of the multiple keys that need to be processed, but it has the benefit that the extension behavior is more clear. There is also an additional cognitive load on the the user, who needs to know the names of multiple keys.

Idea:

What about doing optional inheritance?

[thing]
name = "demo"
items = ['base1', 'base2']

[thing.version_a]
inherit = ['items']
items = ['first1', 'first2']

[thing.version_b]
inherit = false
items = ['second1', 'second2']

(Omitting inherit is equivalent to disabling inheritance)

@drewbrew Interesting idea... it feels a little bit "action at a distance" (i.e., working out how items is being interpreted depends on a key other than items). However, I do like the fact that it doesn't depend on a "magic value" like +.

Another suggestion that has been made to me - don't use Lists - use inline Tables:

[thing]
name = "demo"
items = {
    'base1' = True,
    'base2' = True
}

[thing.version_a]
items = {
    'first1' = True,
    'first2' = True,
}

[thing.version_b]
items = {
    'second1' = True,
    'second2' = True, 
}

[thing.version_c]
items = {
    'base1' = False,
    'third1' = True,
    'third2' = True, 
}

All inline tables would then be "union of keys".

This requires the dummy value True in the syntax, which isn't ideal. That said, In the specific use case I have for this, there might be some useful interpretations for the "value".

It also allows for partial inheritance - version_c can explicitly exclude some of it's parent values.

I like the inherit and separate solutions – it depends a bit on your target audience, though. The separate one is easy to explain and to read, and the inherit one (maybe to a slightly lesser degree), too.

I generally like the idea of explicit inherit = ["items"] configuration, but when I imagine writing a config file, I think this would actually be confusing. I wouldn't expect the inherit clause to lead to merged lists, because "inherit" sounds so all-or-nothing. "+" is more intuitive that way. Maybe this is just a naming thing though – if you don't intend to use it for primitives, "merge_with_parent" or something like that could work.

Your last proposal of inline tables strikes me as hard to use for anybody who is not an experienced developer, and hard to read when you just want to figure out the actual value of something. (Is that a use case? I think during debugging issues, it will be, regardless of the problem domain.)

+1 to inline tables being extremely hard to use, even for experienced devs.

I wouldn't expect the inherit clause to lead to merged lists, because "inherit" sounds so all-or-nothing. "+" is more intuitive that way. Maybe this is just a naming thing though – if you don't intend to use it for primitives, "merge_with_parent" or something like that could work.

What about separate extend vs inherit directives?

What about separate extend vs inherit directives?

“Extend” would do the job!