Random OpenEmbedded thoughts

Dislikes.md

• BitBake is too monolithic. It's a beast. Not very in line with UNIX philosophy.
• Metadata is scattered around, difficult to map what bitbake is doing to what came from where
• Recipes are both declarative and imperative. Order matters, yet it doesn't. Some things are immediate, some are lazy. This leads to confusion.
• BitBake behavioral logic is interspersed with real knowledge of the upstream package. E.g. flags in recipes
• BitBake is still rather slow
• The system is unable to do MACHINE=native builds very well. It expects cross-compilation
• Massive overhead even to build something small. The number of tasks needed to build busybox is ridiculous. Admittedly, there are valid reasons for this, but we'd be better off shipping a known sane host environment rather than rebuilding the universe to satisfy our assumptions
• Fetching is a nightmare, particularly SCM handling. The fact that it's impossible to build with BB_NO_NETWORK in some cases is terribly sad
• BitBake's internals are a twisty maze with no exit. Modules are badly intertwined, causing maintenance headaches
• Sleep learning curve in general, high complexity

Likes.md

• Layered metadata facilitates independent maintainers, as well as making it easy to break up changes in a logical way, and avoid having to modify upstream when not necessary.
• Easy to collaborate, due to our built in support for our orthogonal axes: distro, machine, image, and conceptual layers to handle variable specificity (global, distro, arch, machine, ..)
• "Class" mechanism lets us abstract out common functionality from recipes to pare them down to the bare essential description of the upstream project (ideally)

WouldLike.md

• More inline with UNIX philosophy

  • Split out a standalone fetch tool, ideally using documented, well designed URL schemes
  • Split out a tool which produces an image from a package feed (yocto has started in this direction already)
  • Possibly split out a tool which takes a recipe and builds it, or takes a reformatted recipe (e.g. in json) on stdin and builds it, with the result that binary packages are produced. We could consider use of parallel, ninja, tup, or redo to do the actual execution of this against all recipes, obeying dependencies

• Purely declarative recipes

• Some form of plugin mechanism to control what builder implementation is used, for the non-declarative components, with the ability for a recipe or layer to express what builder is required

• Improved method for making local changes to source trees, without the overhead imposed by the fetch/unpack/patch mechanism

• Improved filesystem structure, such that non-temporary files (output) don't go in tmp, to reduce confusion

• Drop the DEPENDS vs RDEPENDS dichotomy, which is also a source of confusion

• Some form of isolated sysroots to reduce potential cross-recipe contamination

• In addition to handling input checksums, we should also respond to changes in the output of an operation, so a change to the input which doesn't affect the emitted binary packages won't result in rebuilding the dependencies of said package

• Support for automatic interaction with a backing store for build results

• Support for capturing useful information about a build and use it for subsequent builds. Not build output, but information which could be used for e.g. more intelligent scheduling

• Some form of configuration UI, ala menuconfig. I don't believe this should be optional or postponed as it was today, as it's a clear advantage (just look at every comparison between buildroot and OE/Yocto)

• Out of the box support for native, on target builds of the entire embedded linux system

• Improved discoverability and inspection mechanisms. It should be easy to ask the system things like:

  • What configuration variables are available?
  • What recipes are available?
  • What recipes depend upon FOO?
  • What recipes does FOO depend upon?
  • What are the configuration variable semantics?
  • What events exist, and when are they fired?
  • What was built? (toaster is handling some of this now, for post-build examination)

• Lower priority, but would be nice

  • Job dispatch across multiple machines