gitlab-ci-job-dispatcher-spec.md

gitlab-ci-dispatcher

Launch the gitlab runner, through a cli command.

Todo: Build a job dispatcher service that reads in a .gitlab-ci.yml definition file and creates a queue of jobs

• parse the stages
• parse the dependencies and artifacts
• parse "when"
• have queues of jobs that can run in parallel
• have system where artifacts are passed along between stage
• mechanism to send files to the pipeline (git repo, tar, etc) Docker stack
  • gitlab-runner
  • gitlab-ci-dispatcher

gitlab-ci-multi-runner

gitlab-ci-dispatcher (or gitlab-ci-job-dispatcher) must be able to satisfy this interface

package common

type Network interface {
	RegisterRunner(config RunnerCredentials, description, tags string, runUntagged, locked bool) *RegisterRunnerResponse
	VerifyRunner(config RunnerCredentials) bool
	UnregisterRunner(config RunnerCredentials) bool
	RequestJob(config RunnerConfig) (*JobResponse, bool)
	UpdateJob(config RunnerConfig, jobCredentials *JobCredentials, id int, state JobState, trace *string) UpdateState
	PatchTrace(config RunnerConfig, jobCredentials *JobCredentials, tracePart JobTracePatch) UpdateState
	DownloadArtifacts(config JobCredentials, artifactsFile string) DownloadState
	UploadRawArtifacts(config JobCredentials, reader io.Reader, baseName string, expireIn string) UploadState
	UploadArtifacts(config JobCredentials, artifactsFile string) UploadState
	ProcessJob(config RunnerConfig, buildCredentials *JobCredentials) JobTrace
}

And then it can be used as a network in commands/multi.go, probably as a switch.

func init() {
	common.RegisterCommand2("run", "run multi runner service", &RunCommand{
		ServiceName:       defaultServiceName,
		network:           network.NewGitLabClient(),
		prometheusLogHook: prometheus_helper.NewLogHook(),
	})
}

To see see how the GitlabClient Network works search for this in the repo:

func (n *GitLabClient) RequestJob(config common.RunnerConfig) (*common.JobResponse, bool) {

Reverse engineering

gitlab CE Check out file: spec/requests/api/runner_spec.rb

Seeing how gitlab-ce fulfills the common.Network from the ruby on rails perspective will inform how it should be done in go for the gitlab-ci-dispatcher. Particularly the job dependencies and stage construction.