In the beginning of time, the Nomad server makes a JobEndpoint using the NewJobEndpoints function.
A job is submitted to the API (either directly or via the CLI).
The API sends a JobRegisterRequest to Register(). The Register() call is forwarded to the leader where execution continues.
At line 97, it calls the admissionControllers function on the Job.
At line 45, The admissionControllers function calls the admissionMutators function.
The mutators specified on the JobEndpoint created at Server start are:
mutators: []jobMutator{
jobCanonicalizer{},
jobConnectHook{},
jobExposeCheckHook{},
jobImpliedConstraints{},
},Of specific note is the jobConnectHook here. The admissionMutators function call the jobConnectHook's Mutate function which in turn calls its groupConnectHook function.
The groupConnectHook function iterates over all of the services in the group and checks to see if they Connect-enabled services.
For each enabled type it applies a Kind which is used in later validation steps.
-
Connect Sidecars— Check to see if a SidecarTask exists. If not, call
newConnectSidecarTask. newConnectSidecarTask sets Kind toconnect-proxy:«service.Name»as part of the returned Task. -
Connect Native — Sets Kind to
connect-native:«service.Name»and attaches it to the *Tasktprovided from thegetNamedTaskForNativeServicefunction call above it. -
Connect Gateways —
newConnectGatewayTaskcallsnewConnectGatewayTaskwhich returns a Task that has Kind set toconnect-ingress:«service.Name»orconnect-terminating:«service.Name»depending on the result of the call toPrefix()at line 296 of job_endpoint_hook_connect.go.
Once all of the mutation hooks have run, control is returned to the admissionController function which then calls the admissionValidators funtion to range over the admission validators.
validators: []jobValidator{
jobConnectHook{},
jobExposeCheckHook{},
jobValidate{},
&memoryOversubscriptionValidate{srv: s},Again, jobConnectHook will be of note. The admissionValidators function calls Validate. This in turn calls the groupConnectValidate function.
groupConnectValidate runs a series of validations specific to Connect. Upon completion it returns back to admissionControllers. If it errored, it returns a nil Job, nil warnings, and the error encountered, otherwise, it returns a validated Job, an []error with any warnings, and a nil for the error value.
admissionControllers returns to Register function. Any errors cause Register to return an error. Otherwise, the mutated/validated function replaces args.Job.
In line 279, Register calls checkConsulToken with the result of args.Job.ConsulUsages() as its argument.
ConsulUsages returns a map from a Consul namespace to job components that require Consul,
including ConsulConnect, Task.Kinds, Services from groups and tasks, and
a bool indicating if Consul KV is in use (determined by the presence of template stanzas).
In the case of OSS, the only map key will be the empty string "".
This key will refer to a new ConsulUsages struct that contains a map of services built from the group and task levels
While scanning the tasks, if a Template is found, then KV is set to true. (code)
// Enforce the job-submitter has a Consul token with necessary ACL permissions.
if err := checkConsulToken(args.Job.ConsulUsages()); err != nil {
return err
}The checkConsulToken takes the returned map[string]*ConsulUsage and begins to validate it against the Consul token supplied in the Job (j).
If the servers are not configured to require authentication for Consul—that is to say they are not configured with allow_unauthenticated=false, then Nomad will consider the request to be authorized. (code)
Next, Nomad ranges the keys in the ConsulUsages generated earlier—these correlate to the Consul Namespaces the bearer token must be able to access to create all the objects specified in the Job. Again, for OSS, this will always be a single key of the empty string. (code for iteration)
For each usage in the struct, Nomad calls the consulACLs.CheckPermissions function.
This calls CheckPermissions on each usage and against the Job ConsulToken.
(link to code for CheckPermissions function)
func (c *consulACLsAPI) CheckPermissions(ctx context.Context, namespace string, usage *structs.ConsulUsage, secretID string) error {
// consul not used, nothing to check
if !usage.Used() {
return nil
}
9a8c68f6cdad6fa091cef68a6f4020953da6c3e5
// If namespace is not declared on nomad jobs, assume default consul namespace
// when comparing with the consul ACL token. This maintains backwards compatibility
// with existing connect jobs, which may already be authorized with Consul tokens.
if namespace == "" {
namespace = "default"
}
// lookup the token from consul
token, readErr := c.readToken(ctx, secretID)
if readErr != nil {
return readErr
}
// if the token is a global-management token, it has unrestricted privileges
if c.isManagementToken(token) {
return nil
}
// if the token cannot possibly be used to act on objects in the desired
// namespace, reject it immediately
if err := namespaceCheck(namespace, token); err != nil {
return err
}
// verify token has keystore read permission, if using template
if usage.KV {
allowable, err := c.canReadKeystore(namespace, token)
if err != nil {
return err
} else if !allowable {
return errors.New("insufficient Consul ACL permissions to use template")
}
}
// verify token has service write permission for group+task services
for _, service := range usage.Services {
allowable, err := c.canWriteService(namespace, service, token)
if err != nil {
return err
} else if !allowable {
return errors.Errorf("insufficient Consul ACL permissions to write service %q", service)
}
}
// verify token has service identity permission for connect services
for _, kind := range usage.Kinds {
service := kind.Value()
allowable, err := c.canWriteService(namespace, service, token)
if err != nil {
return err
} else if !allowable {
return errors.Errorf("insufficient Consul ACL permissions to write Connect service %q", service)
}
}
return nil
}At this point, if everything is cool, the JobEndpoint is okay with the Consul information in the job and the provided token.
Line 324, The Register function clears out the ConsulToken from the Job, since Nomad will use its token to register services, access KV, and derive SI tokens for Consul Connect sidecars and gateways.