Basic intro to NixOS project's Hydra

nixos-hydra.org

Hydra On-boarding

Hydra: What is it?

Benefits: Hydra vs other CI systems

• supports distributed builds
• can be deployed in a more faul tolerant way with lower operational costs
• supports multiple architecture and different platform builds well
• supports matrix builds (showing support for multiple versions of a language, runtime, database, library, etc.)

Drawbacks: Hydra vs other CI systems

• To allow us to evaluate impure jobs we need to remove sandboxing
• We can turn on sandboxing on our artifact builds again but only after some extra one-time operational work
• Not as GUI point-n-click friendly at the beginning without initiation

Hydra Components

• Queue runner: regularly checks what has changed and what should be built next
• Evaluator: actually builds jobs
• Server: the web application that shows status and details about the projects, jobsets, jobs, evaluations, and build products.

Hydra Dependencies

• Build and run-time: Hydra relies on the Nix package manager heavily.
• Definition: As a result it’s declarative definition of projects, jobsets, jobs, etc are entirely in the Nix expression language.
• Run-time: PostgreSQL (we run a small RDS instance)

Hydra Project Spec

{
  "enabled": 1,
  "hidden": false,
  "description": "",
  "nixexprinput": "src",
  "nixexprpath": "path/to/jobset.nix",
  "checkinterval": 60,
  "schedulingshares": 100,
  "enableemail": false,
  "emailoverride": "",
  "keepnr": 10,
  "inputs": {
    "src" : {
     "type": "git",
     "value": "ssh://git@github.com:org/repo.git release",
      "emailresponsible": false
    },
    "nixpkgs": {
      "type": "git",
      "value": "ssh://git@github.com:NixOS/nixpkgs-channels.git nixos-18.03",
      "emailresponsible": false
    },
    "repoPrs": {
      "type": "githubpulls",
      "value": "org repo",
      "emailresponsible": false
    }
  }
}

Hydra Jobset Expression

• A Hydra expression is a function returning an attrset (think: Ruby Hash)
• Function arguments define variability points, e.g. nixpkgs sources to use, target system architectures, etc.
• Each attribute corresponds to a job definition.
• Each value corresponds to a function that will perform the build or test or deploy.

Example: org/repo

• declarative project spec: z/ci/spec.json (JSON)
• declarative jobsets definition: z/ci/prs.nix (Nix)
• declarative jobs definition: z/ci/jobs.nix (Nix)
• spec.json points to z/ci/prs.nix
• jobset expression points to z/ci/jobs.nix

Jobset Expression Inputs

• All inputs (except where jobset expression source) will be provided to the jobset function as an input.
• Evaluations correspond to the job definition in the jobs.nix expression pointed to by the jobset expression.