Skip to content

Instantly share code, notes, and snippets.

@kergoth

kergoth/MEL Flex 13 Thoughts.md

Last active February 17, 2022 18:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kergoth/4c6495569ef116610d154f69ff846636 to your computer and use it in GitHub Desktop.
Save kergoth/4c6495569ef116610d154f69ff846636 to your computer and use it in GitHub Desktop.
Download ZIP
MEL Flex 13 Thoughts
Raw
MEL Flex 13 Thoughts.md

MEL Flex 13 Thoughts

Current Plans

  • Integrate vendor SDK in a way that allows for the use of vendor machines directly, not requiring the -mel suffix machines to be able to build. The intention here is a different level of support for those (if any) vs the well supported -mel machines. See also discussion in #flex-future on Slack. We should be able to semi-automate the setting of VENDOR, either through xlayers.conf based on the BSP installed, or based on specific logic around which layer provides the MACHINE config file ("if the machine was in meta-ti, set VENDOR=ti").

  • Usual process for a new major version of flex

    • File-by-file review of our layer content vs upstream
    • Line-by-line review of the mel distro
    • Upstream submissions wherever appropriate
    • Updates for newer Yocto version

      • Update our local.conf.sample from upstream local.conf.sample and extended sample.

      • dunfell -> gatesgarth

        • packagegroup-core-device-devel was removed, drop QEMUDEPS from mel
        • MLPREFIX now required for multilib when runtime dependencies conditionally added (inline python in PACKAGES, RDEPENDS, RRECOMMENDS now must include ${MLPREFIX} appropriately)
        • dhcp-client should be replaced by dhcpcd and dhcp-server should be replaced by kea
        • Revisit our WARN_QA/ERROR_QA, new checks were added
        • Globbing no longer supported in file:// entries in SRC_URI
        • Custom SDK / SDK-style recipes need to include nativesdk-sdk-provides-dummy
        • Image artifact name variables now centralised in image-artifact-names class. References to IMAGE_ vars globally will no longer work as previous.
        • In the Upstream-Status header convention for patches, Accepted has been replaced with Backport

      • gatesgarth -> hardknott

        • Single version common license file naming. GPL-3.0 -> GPL-3.0-only, etc. "It is not required that you change LICENSE values as there are mappings from the original names in place; however, in rare cases where you have a recipe which sets LIC_FILES_CHKSUM to point to file(s) in meta/files/common-licenses (which in any case is not recommended) you will need to update those."
        • setup.py path for python modules
        • "The default poky DISTRO_VERSION value now uses the core metadata’s git hash (i.e. METADATA_REVISION) rather than the date (i.e. DATE) to reduce one small source of non-reproducibility."
        • "ffmpeg is now configured to disable GPL-licensed portions by default to make it harder to accidentally violate the GPL. To explicitly enable GPL licensed portions, add gpl to PACKAGECONFIG for ffmpeg using a bbappend"

      • hardknott -> honister

        • Override syntax changes
        • The lz4c, pzstd and zstd commands are now required to be installed on the build host to support LZ4 and Zstandard compression functionality. We need to update our host setup scripts.
        • Prelinking disabled by default (and removed in the future)
        • Tune files moved to architecture-specific directories
        • a new TOOLCHAIN_HOST_TASK_ESDK variable has been created

      • honister -> kirkstone

        • Because of the uncertainty in future default branch names in git repositories, it is now required to add a branch name to all URLs described by git:// and gitsm:// SRC_URI entries. A convert-srcuri script to convert your recipes is available
        • Because of GitHub dropping support for the git: protocol, recipes now need to use ;protocol=https at the end of GitHub URLs. The same script as above can be used to convert the recipes.
        • Network access from tasks is now disabled by default on kernels which support this feature
        • The append, prepend and remove operators can now only be combined with = and := operators.
        • Move to inclusive language, both in our sources and layers and documentation.

Open Questions

  • I'd like to see a list of exactly which of MEL flex's features are considered our main competitive value. Integration of swupdate via firmware-update feature? IOT? ADE? Precisely which pieces of functionality should stay private, if any?
  • I'd like to see actual documented user stories / use cases for the MEL Flex product. I know Brad Dixon had done some work on this before he left, but I don't think we have anything current here. We do have our questionnaires, but there's a question of how much useful information we're getting from them.

Investigation / Research

Here we should investigate how others are doing things to look for ideas and insights we can apply to improve our product and tooling. yoe, for example, provides shell functions to ease common operations, such as flashing an image to the sd card. Angstrom does interesting things with tuning and other configurations to properly support binary package feeds. Wind river may have interesting structure or tooling.

  • Review the yoe distro
  • Review the angstrom distro
  • Review what wind river is doing

There is also useful tooling independent of the aforementioned review which should be considered.

  • Consider use of the kas tool
    • Consider use of kas's menu plugin to ease usability
  • Consider use of the whisk tool
  • Consider use/inclusion/wrapping of extra tooling surrounding the oe .dot files, such as oe-depends-dot and others

Specific Ideas to Consider

Currently unused or unemphasized Yocto features

  • eSDK
  • create-spdx.bbclass
  • devtool
  • recipetool
  • bitbake-layers
  • oe-pkgdata-util
  • oe-depends-dot

Suggestions

  • It's been suggested that we move to internal toolchain builds, which would require meta-codebench-toolchain to split out the toolchain appends for the non-licensing items.
  • It's been suggested that customers would like to be able to access our content via git. This is easy if everything is open, otherwise less so, since we'd have to deal with access control via licensing, and not all our components are hosted externally. We do, of course, already provide the content in the form of git repositories, just installed via our installer first.

General / Setup

  • Consider supporting kas in some fashion. Using it requires explicit yaml configuration rather than using our setup scripts, but this can be automated or semi-automated by generating them, and it clearly provides value.
  • Consider a UI for configuration of only a subset of the metadata, at the least, what's already in local.conf.
  • Consider a UI or scripts to ease creating a new customer layer, creating a new image based on an existing image, creating a new distro from scratch or based on another distro, and creating a new bsp layer from scratch or based on an existing bsp layer or machine file.
  • Do less in local.conf.append/local.conf.sample, it's unwieldy and confusing, and we're overloading its purpose.
    • Create VENDOR variable inclusion of vendor-specific configuration in a consistent way independent of our custom wrapper MACHINE files.
    • Make mel include a config file to apply distro specific overrides to a MACHINE. This violates typical Yocto/OE orthgonality, but would give us a way to tweak things without having to wrap machines when creating new BSPs.
    • Is it still appropriate to disable the 3g and nfc distro features by default in the local.conf.sample? That doesn't seem appropriate any longer, making assumptions about the market and product the user is building seems problematic.
  • Bring in default required and optional layers via LAYERDEPENDS/LAYERRECOMMENDS in meta-mel, perhaps? This would only work if we never have to have a way to override them from a distro that builds on mel. One option here would be to split out conf/distro/include/mel.conf to a different layer than conf/distro/mel.conf to allow for easier leveraging of mel defaults for other products without pulling in flex-specifics.
  • Consider leveraging xlayers.conf or similar shipped with the flex product to avoid the need for the customer to specify -l to setup-environment to pull in expected default product features. This will be helpful when supporting a vendor sdk with vendor machines.
  • Inclusive language, align with techpubs

More upstream submissions

  • Simplify our setup scripts if possible and submit what's possible.
  • Should meta-mentor-private be a public layer? The content is specific to how we do things with the ADE, but it's not really useful to a competitor without the CodeBench side, which is where most of the real added value lies. I doubt we'd lose much by opening it up.

MEL Distro (and distro layer)

  • Minimize mel.conf as much as is possible.
  • Consider breaking up mel.conf into various included configs to make it easier to reuse its components. Alternatively, add more distro features to exert control over mel behavior.

Yocto layer updates

  • Review local.conf.sample/bblayers.conf.sample vs upstream. We often forget to check for changes in this when updating to a newer yocto version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment