gilmour-proxy-spec.md

Gilmour proxy spec

The control TCP port

Gilmour proxy listens for http requests on inet socket on a port it is configured to start with. The following routes are available on the control port.

:POST /nodes - Create a new upstream gilmour node

Request

{
	listen_sock: string <the unix domain socket this node will listen on>,
	health_check: string <http path at listen_sock which responds to a health ping. default: /health>,
	slots: [
		{
			topic: string <topic to listen on. can be a wildcard>,
			group: string <optional exclusion group>,
			path: string <http path at listen_sock corresponding to the handler for this slot>,
			timeout: int <time after which the proxy times out this call>
		}, ...
	],
	services: [
		{
			topic: string <topic to listen on. cannot be a wildcard>,
			group: string <mandatory exclusion group>,
			path: string <http path at listen_sock corresponding to the handler for this service>,
			timeout: int <time after which the proxy times out this call>
		}, ...
	],
}

Before returning a response, the proxy performs the following steps:

1. Open a connection to listen_sock on the unix domain
2. Send a health check ping on the health_check path. Return error if not successful
3. Create subscriptions for the signals and slots provided

Response

It then returns the following in the response

{
	id: string <a uuid identifying this node. this needs to be reused for further requests>,
	publish_sock: string <unix domain socket on which the proxy accepts request, signal and composition execution requests>,
	status: string <status of the node - "ok", "unavailable" or "dirty">
}

The id should be saved by node for all future requests it makes.

Notes

• Health checks: The proxy continues to repeatedly verify the health_check pings. If the node fails to respond to a health_check ping within a timeout of heath_check_timeout the node is marked as "unavailable". All subscriptions corresponding to this node will be removed. The subscriptions will be setup again once the node starts responding to the health checks. If the socket for the node itself cannot be opened, the node is marked as "dirty" and all activity related to the node is stopped.

:GET /nodes?admin_pass=pass - Get a list of existing nodes on the system

admin_pass must be pre-configured. This request fails with an unauthorised error if the admin_pass does not match

Response

{
	nodes: [
		{
			id: string <a uuid identifying this node>,
			publish_sock: string <unix domain socket on which the proxy accepts request, signal and composition execution requests>,
			status: string <status of the node - "ok", "unavailable". "dirty">
		}, ...
	]
}

:GET /nodes/:id - Get details of an existing node

Response

{
	id: string <node uuid>,
	listen_sock: string <the unix domain socket this node listens on>,
	health_check: string <http path at listen_sock which responds to a health ping>,
	slots: [
		{
			topic: string <topic>,
			group: string <exclusion group>,
			path: string <handler http path for this slot>,
			timeout: int <time after which the proxy times out this call>
		}, ...
	],
	services: [
		{
			topic: string <topic>,
			group: string <exclusion group>,
			path: string <handler http pathfor this service>,
			timeout: int <time after which the proxy times out this call>
		}, ...
	],
	status: string <status of the node - "ok", "unavailable". "dirty">
}

Notes

• The above information can be used to restart an abnormally terminated node with the PUT /nodes/:id call as mentioned above.

:DELETE /nodes/:id - Gracefully remove a node

Response

{
	status: string <"ok" or error message>
}

Notes

• This will not affect any running executions (unless terminated by the node)
• The node will no longer appear in the list of nodes. All resources related to that node will be freed.

:POST /nodes/:id/slots - Add a slot subscription

Request

{
	topic: string <topic>,
	group: string <exclusion group>,
	path: string <handler http path for this slot>,
	timeout: int <time after which the proxy times out this call>
}

Response

{
	status: string <"ok" or error message>
}

:GET /nodes/:id/slots - Get list of slot subscriptions

Response

{
	slots: [
		{
			topic: string <topic>,
			group: string <exclusion group>,
			path: string <handler http path for this service>,
			timeout: int <time after which the proxy times out this call>
		}, ...
	]
}

This endpoint is primarily intended for use by monitoring systems

:DELETE /nodes/:id/slots?topic=<topic>&path=<path> - Remove slot subscription(s)

Response

{
	status: string <"ok" or error message>
}

Notes

• The path is optional. If it is not provided, all subscriptions corresponding to the topic will be removed.
• This will not affect any running executions (unless terminated by the node)

:POST /nodes/:id/services - Add a service subscription

Request

{
	topic: string <topic>,
	group: string <exclusion group>,
	path: string <handler http path for this service>,
	timeout: int <time after which the proxy times out this call>
}

Response

{
	status: string <"ok" or error message>
}

:GET /nodes/:id/services - Get list of service subscriptions

Response

{
	services: [
		{
			topic: string <topic>,
			group: string <exclusion group>,
			path: string <handler http path for this service>,
			timeout: int <time after which the proxy times out this call>
		}, ...
	]
}

This endpoint is primarily intended for use by monitoring systems

:DELETE /nodes/:id/services?topic=<topic>&path=<path> - Remove service subscription(s)

Response

{
	status: string <"ok" or error message>
}

Notes

• The path is optional. If it is not provided, all subscriptions corresponding to the topic will be removed.
• This will not affect any running executions (unless terminated by the node)

The publish unix domain socket

When a node registers with the proxy (or is re-activated), the proxy returns a unix domain socket on which it accepts publish requests from the node. This section documents the spec for this socket

:POST /request/:id - Send a request

The :id is the uuid of the node which makes this request

Request

{
	topic: string <topic to send the request to>,
	composition: compostiion_spec,
	message: any <request data>,
	timeout: int <timeout in seconds>
}

Either one of topic or composition_spec can be specified. This call will block till the request returns a response or times out.

Response

{
	messages: [
		{ data: any <response data>, code: int <response code> },
		...
	],
	code: int <max response code of all the mesages>,
	length: int <number of messages in the response>
}

Notes

• The response is composed of one or more (see batch and parallel compositions) messages. Each message has its own data and code.
• The code in the top level response body is the maximum of all the codes in the response body. This will also be the http response code.

:POST /signal/:id - Send a signal

Request

{
	topic: string <topic to send the request to>,
	message: any <request data>,
}

This call returns as soon as the signal is sent.

Response

{ status: string <"ok" or error message> }

: Composition spec (TBD)

The listen_socket protocol

During registration, every node provides a unix domain listen_sock on which it listens for requests and signals, for its subscriptions. The health_check endpoint has already been documented above. This section describes the protocol for calling the http endpoints when a corresponding request or signal is received.

Service endpoints

These are called for corresponding incoming requests. A POST request is made to the endpoint. The request body is

{
	topic: string <topic on which this request was made>,
	sender: string <a unique uuid for this request>,
	data: any <request data>,
	timeout: int <timeout that this service was setup with>
}

Notes

• If the request handler has not finished executing before the timeout, the proxy will send a timeout error code back to the client. Before it does this, it will also close the connection, so that a response write will fail.
• The response body of the handler will be sent unmodified to the client in the response data

Slot endpoints

These are called for corresponding incoming requests. A POST request is made to the endpoint. The request body is

{
	topic: string <topic on which this request was made>,
	sender: string <a unique uuid for this request>,
	data: any <request data>,
	timeout: int <timeout that this slot was setup with>
}

@adityagodbole Where and How is admin_pass configured?