general ideas of a stack inside of a startup with remote machines

startup-tech.md

General idea

• gerrit(k8s) - linked to github repo - partial clone activated
• bazel remote cache(k8s)
• bazel Remote Build API on VM (no containers until it is not an issue for bazel runner)
• drone master(k8s)
• drone worker(physical)
• developers working on remote workstation only (except for mobile apps)

Build agent container/machine(agent)

This container/machine is used as a proxy to invoke Bazel to run jobs using Remote Build API, the advantage over using this instead of stateless job is that it can reuse bazel dependencies without download it on every build (this is an issue even using a Remote Build API / Remote Cache)

• Requirements
  • Bazel
  • Persistent storage

Infra

• k8s
  • Drone(just the master)
  • gerrit
  • argocd
  • sailed secrets
  • prometheus
  • graphana
  • jaeger
  • build-barn(bazel remote builder)

Developers remote workstation

This machine is being used by developers as remote workstation, this means that any developer should have code in their machines, except for mobile app development

On day to day basis, a developer should connect via ssh/mosh to the remote workstation to start programming

Requirements:

• NixOS
• all unix users are available(no chroot)
• no root access
• visual studio code remote server [globally]
• k3s [globally]
• kubectl [globally]
• docker [globally]
• docker-compose [globally]
• baselisk [globally]
• Go [globally]
• Rust [globally]
• Mosh/ssh [globally]
• Bazel Requirements
  • Access to Remote Build Cache
  • Access to Remote Build API

CI

gerrit -> drone -> bazel

• gerrit
  • receives the changes
  • manage merges
  • on every commit, and pre merge sends hook to drone
• drone
  • send the job to the remote worker(in pyshical machine to ensure fast execution)
• bazel
  • is executed trough drone
  • Drone should call a worker on a machine that invokes bazel (this is needed to avoid re download dependencies for bazel an for super fast execution)
  • Bazel calls Remote Execution API backend
  • Remote Execution API runs the desired bazel jobs
    • options
      • build-barn[https://github.com/buildbarn]
      • bazel-buildfarm[https://github.com/bazelbuild/bazel-buildfarm]

CD

bazel -> gitops -> argocd

• bazel
  • test
  • build artifacts
  • upload artifacts
  • generates k8s yaml files
• gitops
  • commit k8s yaml files
• drone
  • watch for changes

Gerrit

• plugins
  • Verify status
  • github
  • webhooks