Created
June 2, 2011 17:27
-
-
Save jfryman/1004862 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A Service is composed of States and Nodes. A change in state fires *..* Protocol Messages to Nodes. | |
Create the ability to have a defined proxy object in code. We'll call this object a service. The service's job is to route messages between nodes subscribed to a specific topic within a middle tier (RabbitMQ or ActiveMQ or ... - would make sense to leverage mCollective for this action). This service should be able to exist on a separate node within a puppet infrastructure. | |
Each service will maintain a state table. This state table should have the ability to be managed internally or hook into external monitoring system state table (a la: nagios/zabbix/ca/openview/bleg) | |
Each service will the ability to define different node types that subscribe/respond to different protocol messages. | |
To subscribe to a service, a node must identify the type of node it is (service node definition). | |
In order for a node of any type to participate in a service, it must be able to respond to all required methods defined in a controller, and optionally respond to any optional request methods in a controller. A node should provide an acknowledgement of each protocol it can respond to at handshake. | |
A user should be able to force the passing of a message in real-time via a CLI utility (mCollective). | |
Each service must have defined a set of states that are user defined. | |
Each state will have a set of bounds, or triggers, that when reached, will trigger a protocol message. | |
A given state can call or trigger N protocol messages. | |
Services <---> States <---> Protocols <---> Nodes <---> Facts | |
### TODO | |
TODO: Provide a graphical representation of flow. | |
### | |
## Prototype code call from PuppetMaster to Controller | |
## Use Case: Load Balancer auto balancing nodes based on load. | |
### | |
# Controller declaration automatically connects to MQ and creates a topic with the $name | |
service loadbalancer( | |
queue = 'lb-aws', # can be overloaded as appropriate | |
username = 'queue1234', # optional username/password credentials for queue subscription | |
password = 'queue1234', | |
) { | |
# defines the types of nodes that connect to a given controller. Consider not overloading node in the | |
# definition of a controller object | |
node { 'load-balancer': | |
type => 'loadbalancer', # two options here - we create a RAL for a node type, or we can allow providers to be explicitly defined (big5, asa). Maybe a migration from latter to former as this matures | |
minimum => 1, # defines the minimum number of nodes that must exist in order for the controller to be active. | |
requires => [ Protocol['add-node-to-lb'], # Creates the relationship of what each node type | |
Protocol['remove-node-from-lb'], | |
Protocol['report-connections'], | |
], | |
} | |
node { 'ec2-controller': | |
type => 'mcollective', | |
minimum => 2, | |
requires => [ Protocol['create-node'], | |
Protocol['destroy-node'], | |
], | |
} | |
node { 'web-server': | |
type => 'mcollective', | |
minimum => 3, | |
require => [ Protocol['report-status] ], | |
} | |
# Define states for a given service. TODO: Maybe this information can be gathered from | |
# existing monitoring systems. This does not set logic on how a state is defined, but rather | |
# the message relationships between each other. | |
State { poll => '3600', } # Code compression techniques, poll frequency represented in seconds | |
state { 'ok': | |
type => default, # indicates this is the default state or desired state for a service to be in. | |
message => Protocol['report-status'], | |
} | |
state { 'under-capacity': | |
message => Protocol['create-node'], | |
message => Protocol['report-connections'], | |
} | |
state { 'over-capacity': | |
message => Protocol['destroy-node'], | |
} | |
# Define protocols for a given service. | |
protocol { 'create-node': # automatically creates a message type based on $title | |
name => 'create-aws-node', # $title can be overloaded | |
type => {required,optional} # all nodes connecting to a queue must be able to respond if required | |
return => {UUID}, # retrieving key/value pairs from return message in message. | |
return => ipaddress, | |
} | |
protocol { 'report-status': | |
return => state[up/down], # Can/should this be data types? | |
return => total_connections[int], | |
return => current_connections[int], | |
} | |
.... | |
## Relationships between protocols | |
Protocol['create-node'] ~> Protocol['add-node-to-lb'] | |
Protocol['remove-node-from-lb'] ~> Protocol['destroy-node'] | |
... | |
# Logic here to define how to adjust states for a given service. | |
# Built in logic to test that the required number of node types exist and can respond to messages AND | |
# also need to provide method or methods to access the State table maintained. | |
# Example checks for two conditions. The first condition checks to see if the total capacity of the web farm is at 70% capacity. If yes, then spin up additional nodes to handle the change. | |
# The second example has a complex error condition check. I'm sure there is an easier way to check complex logic. This also assumes that we have methods to access declared variables within the state | |
# table as well as the service definition itself. (probably needs refactoring). | |
# The difficulty here is modeling the complex scenarios in which we would want a bot or action executed to create a new node. This can and should be an advanced topic. | |
# Breakout to make the example readable | |
$total_webservers = State['web-server'].count | |
$total_allowed_connections = State['web-server].total_connections.inject(0) { |s,v| s += v } | |
$total_current_connections = State['web-server].current_connections.inject(0) { |s,v| s += v } | |
if (($total_current_connections / $total_allowed_connections) > 0.7) { | |
notify => State['under-capacity'] | |
} elsif (($total_current_connections / $total_allowed_connections) < 0.2) && | |
($total_webservers > Node['web-balancer'].minimum) && | |
(self.state == 'ok') && | |
(self.last_state_change > 84000)) { | |
notify => State['over-capacity'] | |
} | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment