Skip to content

Instantly share code, notes, and snippets.

@jcooklin
Last active December 20, 2015 19:43
Show Gist options
  • Save jcooklin/92b9ea4692a2e9f6f1b8 to your computer and use it in GitHub Desktop.
Save jcooklin/92b9ea4692a2e9f6f1b8 to your computer and use it in GitHub Desktop.
rfc: tribe redux

#RFC: Tribe - subtribes, policy, message encryption and calling remote plugins

This spec proposes the following features and enhancements to tribe

  • Group members of the global tribe into named subtribes replacing what today is called agreements
  • Tribe policies to dynamically add nodes to subtribes
  • Enable calling remote plugins through sharing plugin meta via tribe
  • Encryption of tribe messages

Subtribes

To improve the operational experience what is currently known as an ‘agreement’ will be replace with the term ‘tribe’ and actions that affect a tribe will be made explicit. Let’s start with an example.

Instead of creating a named agreement we will create a named tribe and and join all the members to it.

i1> snapctl tribe create core
i1> snapctl tribe join core i1
i1> snapctl tribe join core i2
i1> snapctl tribe join core i3
i1> snapctl tribe join core i4

s1

Let’s imagine that i1 and i2 are somehow special and should have additional plugins loaded and tasks running beyond what is defined by the core tribe.

i1> snapctl tribe create storage
i1> snapctl tribe join storage i1
i1> snapctl tribe join storage i2

On our core tribe we will load an influxdb and psutil plugin and start a task capturing basic OS utilization details. On our storage tribe we will load the smart plugin and start a task capturing disk IO.

i4> snapctl plugin load influxdb —tribe core
i4> snapctl plugin load psutil —tribe core
i4> snapctl task create -t psutil-influx.json —tribe core
i4> snapctl plugin load smart —tribe storage
i4> snapctl plugin task create -t disk-io.json —tribe storage

Explicitly referring to the tribe when loading plugins or tasks reduces the risk that a user accidentally affects the entire tribe when they perform actions that are intended for an individual node. It also more effectively supports multiple potentially overlapping tribes.

s2

Dynamically adding nodes to tribes through policies

When started in tribe mode snap will establish a list of facts collected from the node it is running on as well as arbitrary key/value pairs provided on startup. These facts will then be used to evaluate tribe policies. When a policy is evaluated positively it will result in the node being added to a tribe.

Example facts: architecture, default_ipv4, default_ipv6, devices, os_dist, os_dist_release, os_dist_version, processor_type, processor_features, memtotal,…

Adding a policy:

i1> snapctl tribe policy create ubuntu_policy “{{os_dist}} == Ubuntu” —tribe core
i1> snapctl tribe policy create storage_policy “{{os_dist}} == Ubuntu && {{storage_tier}} == True” —tribe storage

When snap is started in tribe mode on an Ubuntu host with the policy above configured it will automatically join the core tribe. If it has the fact storage_tier=True it be added to the storage tribe.

Other affected components

  • Global config will need to support arbitrary facts.

Process and publish through remote nodes (calling remote plugins)

Tribe enables the ability to reference a named tribe in the process and/or publish portion of a task definition. When a plugin is loaded on a node that is associated to a tribe it will share the connection details to the global tribe as part of its meta data. This enables each snap node in the tribe to call remote plugins.

s3

Other affected components

  • The scheduler will need to be extended to accept a task manifest with named tribe details
  • Control will need to deal with remote plugin subscriptions

Encrypt tribe messages

Protect tribe communication by supporting symmetric key encryption.

  • Tribe encryption will require the encryption key when starting
  • A helper for generating the encryption key will be provided (example: snapd keygen)
  • The global config will support the tribe encryption key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment