Skip to content

Instantly share code, notes, and snippets.

@georgfaust

georgfaust/readme.md

Created December 3, 2021 16:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save georgfaust/aa759bbacaaec3b804b9e3d88d57eb30 to your computer and use it in GitHub Desktop.
Save georgfaust/aa759bbacaaec3b804b9e3d88d57eb30 to your computer and use it in GitHub Desktop.
Download ZIP
WIP - nerves bootstrap walkthrough
Raw
readme.md

>>> WARNING THIS IS WIP <<<

What?

This is a walkthrough of whats going on when you do

mix nerves.new dummy
dummy$ mix deps.get
dummy$ mix firmware

prereq

  • nerves_bootstrap provides mix Tasks, see [3]
  • you should know the structure of a %NervesPackage see end of file.

setup

To see what's going on we need to use local versions of nerves_bootstrap and nerves and ...

# uninstall nerves_bootstrap (if already installed)
mix archive.uninstall nerves_bootstrap

git clone git@github.com:nerves-project/nerves_bootstrap.git

# install the local version using the alias nerves_bootstrap provides
nerves_bootstrap$ mix install

check if local bootstrap can be used, also enable NERVES_DEBUG.

export NERVES_DEBUG=1
mix nerves.new test
test$ mix deps.get

deps.get should print some debug-messages, but new will not, so add some debug output to new

diff --git a/lib/mix/tasks/nerves/new.ex b/lib/mix/tasks/nerves/new.ex
   import Mix.Generator
+  import Mix.Nerves.IO
 
   defp run(app, mod, path, opts) do
+    debug_info("new")

After the change: reinstall

nerves_bootstrap$ mix install

mix nerves.new

When mix nerves.new dummy is run, mix seaches fo this tasks

  • in the current project
  • and its dependecies
  • installed archives

here it is found in the installed archive nerves_bootstrap - Mix.Tasks.Nerves.New

the happy path is defp run(app, mod, path, opts) which gathers some information into binding:

binding: [
  app_name: "dummy",
  app_module: "Dummy",
  bootstrap_vsn: "1.10",
  shoehorn_vsn: "0.7.0",
  runtime_vsn: "0.11.3",
  ring_logger_vsn: "0.8.1",
  elixir_req: "~> 1.9",
  nerves_dep: "{:nerves, \"~> 1.7.4\", runtime: false}",
  in_umbrella: false,
  nerves_pack?: true,
  nerves_pack_vsn: "0.6.0",
  toolshed_vsn: "0.2.13",
  targets: [
    rpi: "1.17",
    ...
  ],
  ...
]

And then copies templates to the new project with copy_from/3

Then it asks if deps.get should be run (1).

mix nerves.deps.get (aliased to deps.get)

invocation

when mix deps.get is run inside the dummy-project directory the following happens.

  1. mix exectues config/config.exs (no matter what the mix-arguments are: mix.exs and configs are always executed)
  2. Application.start(:nerves_bootstrap) is executed (implementation is in lib/nerves_bootstrap.ex) (2)
  3. all it does is executing Nerves.Bootstrap.Aliases.init which
  4. checks if "We are at the top of the stack" (3)
  5. loads the mix project config and injects aliases:
aliases: [
  "deps.get": ["deps.get", "nerves.deps.get"],
  "deps.update": [&Nerves.Bootstrap.Aliases.deps_update/1]
]
  1. when mix now executes deps.get it finds the alias [nerves.deps.get, deps.get] and therefore first runs nerves.deps.get

nerves.deps.get (Mix.Tasks.Nerves.Deps.Get.run/1)

  1. nerves_env_info is displayed (about Mix targets see [1] and [2])
  2. Nerves.Env.run(["--disable"]) is called
  3. which compiles nerves but not the system and toochain.
    • calls Mix.dep.load_and_cache for all deps
      • Mix.Tasks.Deps.Loadpaths.run(["--no-compile"])
      • Mix.Tasks.Deps.Compile.run(["nerves", "--include-children"])
      • Mix.Dep.load_and_cache() - cache deps in mix state
      • Mix.Task.run("deps.precompile") - load all dependency paths may be aliased to nerves.precompile (4)
      • compile (by mix.exs, rebar or Makefile) - here: nerves and all of its deps
  4. and then runs Nerves.Env.start()
    • checks system requirements
    • source_date_epoch (I do not look into this)
    • starts an agent that holds a list of all "nerves_packages" (system and toolchain)

mix firmware

Note: this is a mix task in nerves not in nerves_bootstrap

  1. Preflight.check!() - check if all tools (fwup ...) are available
  2. check if system and toolchain are set
  3. run nerves.precompile (in nerves_bootstrap)
    • run Nerves.Env (starts an agent that holds a list of all "nerves_packages")
    • get the list of NervesPackage structs from Nerves.Env
    • compile stale nerves_packages (5)
      • stale packages are those where:
        • NervesPackage.app is not the application that is being built (here: :dummy)
        • and :nerves_package is in NervesPackage.compilers
        • and Nerves.Artifact.expand_sites does not return an empty list for the package. (for example there is a site where a prebuilt artifact can be loaded from) (6)
        • and Artifact.stale? (artifact path not set by env OR not in cache)
        • and the package can be compiled anyway
Nerves.Artifact.build(package, toolchain)
# {Nerves.Artifact.BuildRunners.Local, [] = opts} <- package.build_runner
Nerves.Artifact.BuildRunners.Local.build(package, toolchain, opts)
# Nerves.Toolchain.CTNG <- package.platform
Nerves.Toolchain.CTNG.build(package, toolchain, opts)
- script <- <nerves_toolchain_ctng>/build.sh
- defconfig <- <package>/nerves_defconfig
# !!!WIP!!!

Open Questions

(1) Why? this does not do a nerves.deps.get because bootstrap Application is not yet loaded (see below). So we have to mix deps.get in the project anyways.

(2) How is the nerves_bootstrap Application found? It's obviously taken from the nerves_bootstrap archive. But how does that work? nerves_bootstrap is listed in mix.exs/project/archives but it also works if its not set there. Where can I find information about archives? Does mix automagically search for the application in all archive.installed archives?

(3) Aliases.init claims to check if its "at the top of the stack", which is done like this:

with %{} <- Mix.ProjectStack.peek(),
      %{name: name, config: config, file: file} <- Mix.ProjectStack.pop(),
      nil <- Mix.ProjectStack.peek() do
  • As I understand it this does not check if sth is on the top of the stack but if there is only one element on the stack...?
  • where can I find information about Mix.ProjectStack

(4) the docs for Mix.Tasks.Deps.Precompile explicitly mention nerves.precompile:

This is a task that can be aliased by projects that
need to execute certain tasks before compiling dependencies:
  aliases: ["deps.precompile": ["nerves.precompile", "deps.precompile"]]

but nerves.new does not set such an alias.

(5) why are they split in apps and deps but never used individually?

(6) this means I always need a artifact site, otherwise the package could never be stale!? Seems odd.

Bibliography

%Nerves.Package{
  app: :nerves_toolchain_armv7_nerves_linux_gnueabihf,
  build_runner: {Nerves.Artifact.BuildRunners.Local, []},
  compilers: [:nerves_package, :yecc, :leex, :erlang, :elixir, :app],
  config: [
    platform_config: [defconfig: "defconfig"],
    target_tuple: :armv7_nerves_linux_gnueabihf,
    artifact_sites: [github_releases: "nerves-project/toolchains"],
    checksum: ["mingw32_x86_64_defconfig", "defconfig", "README.md", "LICENSE",
     "mix.exs", "VERSION"]
  ],
  dep: :hex,
  dep_opts: [],
  env: %{},
  path: "/home/sf/ws/cobox2/understand_nerves/nerves_system_rpi3/deps/nerves_toolchain_armv7_nerves_linux_gnueabihf",
  platform: Nerves.Toolchain.CTNG,
  type: :toolchain,
  version: "1.4.3"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment