Choria supports Federating multiple collectives of nodes into a large Federated collective for the purpose of orchestrating.
Federation Requests have:
- A orchestration request as normal
- Special headers listing which nodes its for
The client chops a large request into bundles of 200 nodes and publish them to the network for federation. Multiple federation brokers can thus take care of distributing it. This effectively offloads a huge amount of work from the Ruby client on your CLI to the network. Compared to unfederated mcollective a 200 fold decrease in work - now being offloaded to a big pool of fast Go based workers.
The Federation Broker:
- Receives a message with Federation headers that asks it to deliver the message on its behalf (Input)
- Transforms it into a unfederated mcollective request (Transformers)
- Receives it for delivery and delivers it to each client (Outputs)
Thus the input of 0.62 / second yields 14 / second as the client offload work onto the network.
A large federated network might look like this:
This is the component called the Federation Broker and what happens internally to it
I can also trace the path to a machine and back from the CLI:
