yaml-roundtripping-is-hard

yaml-roundtripping-is-hard.md

I wanted to be able to automatically format yaml as follows:

• preserves comments
• modifies sequence order (sort by key or val)
• modifies key order (alpha sort? manual key order? decl key order?)

Here's what I've tried

Use python

• Most of our tooling is written in go, so this isn't exactly DRY

ruamel.yaml

• This appears to be the gold standard of round-tripping
• Supports round-tripping of comments
• Output key order can be explicitly/manually specified

Use golang

• No existing golang lib supports round-tripping of comments (kubernetes/kubernetes#15157)

gopkg.in/yaml.v2 (aka go-yaml/yaml)

• Does not preserve comments
• Does not preserve folded style strings
• Output key order matches struct declaration order
• This may just be bad YAML, but time: 10:00 becomes time: "10:00"
• Can annotate only those fields that need it, eg: yaml:"snake_case", yaml:",omitempty"

sigs.k8s.io/yaml

• does not preserve comments
• Output key order is alphabetic
• Must explicitly annotate everything, eg: json:"field"

gopkg.in/yaml.v3

• just released https://blog.ubuntu.com/2019/04/05/api-v3-of-the-yaml-package-for-go-is-available
• default indent is 4, can use an Encoder to explicitly set indent back to 2
• must unmarshal into a a yaml.Node type to get roundtrip benefit, unclear how to use that alongside a struct
• use of Decoder.KnownFields is hidden from yaml.Node.Decode
• tried a walk-based approach to sort sequences
• I can immediately marshal the yaml.Node back, but no idea how to deal with modifying the thing
  • I tried a walk-based approach to sort sequences
  • trailing whitespace before a comment is stripped
  • folded strings seem to have an extra newline appended