Skip to content

Instantly share code, notes, and snippets.

@nilium
Last active June 1, 2019 20:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nilium/6e63bace65fc0c4929d75cc4da1dd2a0 to your computer and use it in GitHub Desktop.
Save nilium/6e63bace65fc0c4929d75cc4da1dd2a0 to your computer and use it in GitHub Desktop.

Build server notes

Any use of "we" in this document is the authorial we because using "I" sounds weird. This is intended to cover building packages under spiff-packages and not necessarily supporting the entirety of void-packages (due to size, complexity, and not having a good idea of every weird quirk of every package in it).

All jobs are intended to run using parametrized nomad tasks. This is not necessarily a requirement but is how it's planned out for now due to the homelab setup.

When building a package, we first need to request a list of all dependencies that package has and fetch all of them

If any package is not already in either the main or build repo, we need to build it, so we have to submit additional jobs to build those packages.

Once those packages are built, we can then schedule the package to be built.

If a new version is submitted before the package is built, cancel the previous build and schedule the new one the same way.

If a cycle is detected, stop with an error.

A database of some kind should be used to schedule builds. Likely just SQLite to start, since a complete history of builds isn't necessary and we should be able to wipe a DB and be fine. A persistent store should only be necessary for an HA configuration, in which case something like MySQL or Postgres (using Postgres implicitly supports Cockroach) or what have you is necessary so we can kick the can on HA to the DBMS.

All scheduling should go through a primary server that sets up the jobs and schedules them so that we only run what we can support at a time. This way we don't over-allocate resources for jobs that wait on other jobs.

The primary server should be responsible for locking the repodata, updating it, and serving package files (as in paths to .xbps files should be server under the same path as the repodata because this is a requirement).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment