Skip to content

Instantly share code, notes, and snippets.

@freakboy3742
Last active October 15, 2019 13:19
Show Gist options
  • Save freakboy3742/1f801db3d441859e91f6304bad8abe09 to your computer and use it in GitHub Desktop.
Save freakboy3742/1f801db3d441859e91f6304bad8abe09 to your computer and use it in GitHub Desktop.
Inherited properties in TOML
[thing]
name = "demo"
[thing.version_a]
items = ['base1', 'base2', 'first1', 'first2']
[thing.version_b]
items = ['base1', 'base2', 'second1', 'second2']
[thing]
name = "demo"
items = ['base1', 'base2']
[thing.version_a]
items = ['first1', 'first2']
[thing.version_b]
items = ['second1', 'second2']
[thing]
name = "demo"
items = ['base1', 'base2']
[thing.version_a]
items = ['+', 'first1', 'first2']
[thing.version_b]
items = ['+', 'second1', 'second2']
[thing.version_c]
items = ['third1', 'third2']
[thing]
name = "demo"
items = ['base1', 'base2']
[thing.version_a]
sub_items = ['first1', 'first2']
[thing.version_b]
sub_items = ['second1', 'second2']
@freakboy3742
Copy link
Author

freakboy3742 commented Oct 14, 2019

The problem

TOML allows the specification of lists (arrays). It also allows for the definition of hierarchical sections in the TOML document (as Tables).

I have a use case where I need to specify a list that is composed hierarchically. Elements in a table will share common elements; a collection of elements with a common base definition, extended as part of the table structure. The final parsed data will look something like:

  • thing.version_a.items = ['base1', 'base2', 'first1', 'first2']
  • thing.version_b.items = ['base1', 'base2', 'second1', 'second2']

Since 'base1' and 'base2' are common elements, I'm looking at a conventions I can use in my configuration file to allow users to avoid repetition.

I can see four possible alternatives, demonstrated above, and explained below. Which do you prefer?

Explicit

Don't do anything fancy. Require the repeated elements to be repeated. This is long-winded, but explicit.

Implicit

Internally, handle "items" as a key that is known to be hierarchical. When processing thing.version_a.items, look for thing.items as well, and concatenate the two values.

This allows for factoring out the common elements, but it requires whatever handles the configuration file to know that "items" is a special value with a hierarchical interpretation. This means any "special" keys will need to be pre-coded in the configuration parser.

Inheritance

Define a special value (+) that is interpreted as "look up one level for the same key, and extend this list with the value of the parent level".

This allows for factoring out the common elements, but it requires no special handling of the key. The configuration file would need to understand how to handle + as a value, but any key could use this syntax without pre-coding in the configuration parser.

The use of + could be restricted to the first element of the list; this would allow + to be a literal in the list itself. Alternatively + could be a placeholder that is interpreted as "insert the parent list" - so, items = ['first1', '+', 'first2'] would be interpreted as ['first1', 'base1', 'base2', 'first2']

This use case also allows you to not include the parent element, so thing.version_c.items = ['third'] becomes possible.

Separate

Use separate keys for the lists at each level. This is similar to the implicit approach, in that whatever uses the configuration file would need to be aware of the multiple keys that need to be processed, but it has the benefit that the extension behavior is more clear. There is also an additional cognitive load on the the user, who needs to know the names of multiple keys.

@drewbrew
Copy link

drewbrew commented Oct 14, 2019

Idea:

What about doing optional inheritance?

[thing]
name = "demo"
items = ['base1', 'base2']

[thing.version_a]
inherit = ['items']
items = ['first1', 'first2']

[thing.version_b]
inherit = false
items = ['second1', 'second2']

(Omitting inherit is equivalent to disabling inheritance)

@freakboy3742
Copy link
Author

@drewbrew Interesting idea... it feels a little bit "action at a distance" (i.e., working out how items is being interpreted depends on a key other than items). However, I do like the fact that it doesn't depend on a "magic value" like +.

@freakboy3742
Copy link
Author

freakboy3742 commented Oct 14, 2019

Another suggestion that has been made to me - don't use Lists - use inline Tables:

[thing]
name = "demo"
items = {
    'base1' = True,
    'base2' = True
}

[thing.version_a]
items = {
    'first1' = True,
    'first2' = True,
}

[thing.version_b]
items = {
    'second1' = True,
    'second2' = True, 
}

[thing.version_c]
items = {
    'base1' = False,
    'third1' = True,
    'third2' = True, 
}

All inline tables would then be "union of keys".

This requires the dummy value True in the syntax, which isn't ideal. That said, In the specific use case I have for this, there might be some useful interpretations for the "value".

It also allows for partial inheritance - version_c can explicitly exclude some of it's parent values.

@rixx
Copy link

rixx commented Oct 15, 2019

I like the inherit and separate solutions – it depends a bit on your target audience, though. The separate one is easy to explain and to read, and the inherit one (maybe to a slightly lesser degree), too.

I generally like the idea of explicit inherit = ["items"] configuration, but when I imagine writing a config file, I think this would actually be confusing. I wouldn't expect the inherit clause to lead to merged lists, because "inherit" sounds so all-or-nothing. "+" is more intuitive that way. Maybe this is just a naming thing though – if you don't intend to use it for primitives, "merge_with_parent" or something like that could work.

Your last proposal of inline tables strikes me as hard to use for anybody who is not an experienced developer, and hard to read when you just want to figure out the actual value of something. (Is that a use case? I think during debugging issues, it will be, regardless of the problem domain.)

@drewbrew
Copy link

+1 to inline tables being extremely hard to use, even for experienced devs.

@drewbrew
Copy link

I wouldn't expect the inherit clause to lead to merged lists, because "inherit" sounds so all-or-nothing. "+" is more intuitive that way. Maybe this is just a naming thing though – if you don't intend to use it for primitives, "merge_with_parent" or something like that could work.

What about separate extend vs inherit directives?

@rixx
Copy link

rixx commented Oct 15, 2019

What about separate extend vs inherit directives?

“Extend” would do the job!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment