Composition proxy replacement for ESI, uses node-trumpet.
git clone git@github.com:TSLEducation/service-page-composer.git
cd service-page-composer
npm install
node cluster.js stubVisit http://localhost:5000/
Page composer is a composition proxy - you put it in front of a back end service that acts as a template into which content from microservices is composed.
The back end service returns HTML (with or without the declarative markup explained below), Page composer then parses the HTML on the way through and makes any requests to other services that are then inserted into the HTML on the way through. Typically the backend service will be a CMS or a static HTML page containing specific markup.
This allows the following:
CMS = Return base HTML page layout, with special declarations when it wants to include something more complex from another app, e.g.
<div cx-url='{{server:local}}/application/widget' cx-cache-ttl='10s' cx-cache-key='widget:user:{{cookie:userId}}' cx-timeout='1s'>
This content will be replaced on the way through
</div> Alternatively you can use the configuration based approach, in the configuration file adding a 'transformations' section like below:
<div id='ApplicationWidget'>
This content will be replaced on the way through
</div> Configuration based composition then looks as follows:
"transformations":{
"ApplicationWidget": {
"type": "replacement",
"query": "#ApplicationWidget",
"url": "{{server:local}}//application/widget",
"cacheKey":"delayed1",
"cacheTTL":"10s",
"timeout": "1s"
}
}There are pros and cons to both approaches, with the declarative approach being simplest to manage if you have full control over the back end CMS and it is easy to change, or the configuration based approach if you want to control everything at the proxy layer (e.g. you are replacing chunks of an existing page that is difficult to change).
The full configuration options can be found in /config/default.json, with a file per TSL_ENV that can be used to over-write or add any additional configuration per environment. I will explain the main ones here:
{
"backend": {
"target":"http://localhost:5001",
"host":"localhost",
"ttl":"10s",
"quietFailure":false,
"replaceOuter":false
},
"parameters": {
"urls": [
{"pattern": "/teaching-resource/.*-(\\d+)", "names": ["resourceId"]}
],
"servers": {
"local": "http://localhost:5001"
}
},
"logging": {
"transports": {
"console": {
"transport": "Console",
"level": "debug",
"timestamp": true,
"colorize": true
}
}
},
"cache": {
"engine": "redis"
},
"transformations":{
"HeaderReplacement": {
"type": "replacement",
"query": "#ReplaceHeader",
"url": "{{server:local}}/delayed",
"cacheKey":"delayed1",
"cacheTTL":"10s",
"timeout": "1s"
},
"DeclarativeReplacement": {
"type":"replacement",
"query": "[cx-url]"
}
}
}These properties configure the backend server that the initial request goes to grab the HTML that is then processed.
| Property | Description |
|---|---|
| target | The base URL to the backend service that will serve HTML. URLs are passed directly through, so /blah will be passed through to the backend as http://backend/blah |
| host | The name to be passed through in the request (given the hostname of compoxure is likely to be different to that of the backend server |
| ttl | The amount of time to cache the backend response (TODO : make it honor cache headers) |
| timeout | Time to wait for backend to respond - should set low |
| quietFailure | Used to determine if page composer will serve some error text in response to a microservice failure or fail silently (e.g. serve nothing into the space). |
| replaceOuter | Used to configure if page composer will replace the outer HTML element or not (default is NOT). If you replace the outer element then the response from the micro service will completely replace the matching element |
The parameters section provides configuration that allows compoxure to use data from the initial request (and config) to pass through to microservices (e.g. url parameters, values from the url path via regex, static server names to avoid duplication).
| Property | Description |
|---|---|
| urls | This is an array of objects containing pattern / names values. Pattern is a regex used to parse the request hitting page composer, names are the names to assign to the matches from the regex. All of the patterns are executed against the incoming URL, and any matches added to the parameters that can then be used in the microservice requests. The full list of these are found below. |
| servers | A list of server name to server URL configurations that can be used to avoid repetition of server names in your fragments |
Page composer allows caching of both the back end response and page fragments. This is currently done using Redis, but other cache engines could be put in place.
Note that the cache implementation in Redis is not done using TTLs, as this means that once Redis expires a key if the backend service is down that page composer will serve a broken page. It is instead done slightly differently to allow for the situation that serving stale content is better than serving no content.
To disable caching simply delete the entire config section.
| Property | Description |
|---|---|
| engine | The engine to use (currently only 'redis' is valid). |
The heart of Page Composer is the transformation area. This is where you can add your own selector based transformations, or simply leave the default declarative transformation:
"DeclarativeReplacement": {
"type":"replacement",
"query": "[cx-url]"
}
}Do not delete this transformation if you want the declarative code to work! The properties of new transformations are as follows:
| Property | Description |
|---|---|
| type | The type of transformation to apply to the matching selector (currently only 'replacement' is valid). |
| query | The selector (see trumpet config) that the replacement will be applied to |
| url | The url to call to get the content to put into the section matching the selector (specific to replacement). |
| cacheKey | The key to use to cache the response |
| cacheTTL | The time to cache the response |
| timeout | The timeout to wait for the service to respond |
Note that page composer will always try to fail gracefully and serve as much of the page as possible.
To use page composer in a declarative fashion, simply add the following tags to any HTML element. The replacement is within the element, so the element containing the declarations will remain.
| Property | Description |
|---|---|
| cx-url | The url to call to get the content to put into the section matching the selector (specific to replacement). |
| cx-cache-key | The key to use to cache the response (if blank it will use the cx-url to create a cache key) |
| cx-cache-ttl | The time to cache the response (set to zero for no cache - defaults to 60s). |
| cx-timeout | The timeout to wait for the service to respond |
Page composer allows you to use mustache like templates for a number of strings, specifically URL and Cache Key fields tend to allow the use of variables. The possible variables are:
| Prefix | Description | Example |
|---|---|---|
| param | Parameters matched from the parameters configuration (regex + name) pairs in the configuration | /resource/{{param:resourceId}} |
| query | Parameters matched from any query string key values in the incoming URL | /user/{{query:userId}} |
| cookie | Any cookie value | /user/{{cookie:TSL_UserID}} |
| header | Any incoming header value | /user/feature/{{header:x-feature-enabled}} |
| server | A server from the configuration in the parameters section of config | {{server:live}}/feature |
Page composer uses a number of time intervals for timeouts and TTLS. To make this simpler, there is a simple library that can convert basic string based intervals into ms.
e.g. 1s = 1000, 1m = 60*1000 etc. The valid values are 1s, 1m, 1h, 1d. If you do not provide a suffix it assumes ms.
To assist with local development, there is a very simple stub server that can be invoked by simply adding 'stub' as the last parameter when running, this will spin up a simple HTTP server that will respond to the URL configured as the backend.