The Insights API is the interactive interface for time based metrics as part of the Layer Data Services. It is intended to allow users to query metrics that have been pre-aggregated by us, with or without a limited amount of post-aggregation. Although the Insights API is modelled on common analytics APIs, this API is not intended for the Layer Data Services to be used as a replacement for an analytical database. Rather, as the name implies, it is intended to give our users insights into how they and their users are using Layer. Separately, the Layer Data Services will provide the ability to download relevant data into common analytical databases like Amazon Redshift and Google BigQuery, where it can be combined with other data and used for data analysis.
The Insights API operates on time-series event data that is collected from our backend services, and allows users to submit a query that returns aggregations of this data.
Since the underlying data is time based, every query must specify a start time and an end time.
Queries must also specify the interval into which the results will be aggregated, which must be one of hour, day, week or month.
Also, results are aggregated at the level of an application, the id of which is specified in the API endpoint.
Results returned by the Insights API are made up of metrics and dimensions.
Metrics are quantitative measurements, such as the number of messages sent, or the number of users who logged in.
Dimensions are characteristics that the metrics pertain to, and by which they may be further aggregated.
Thus the metric messages_sent may pertain to conversation layer:///conversations/f3cc7b32-3c92-11e4-baad-164230d1df67 or to user joe_smith, in which case the value of message_sent would correspond to the specified dimension.
Metrics may be aggregated by more than one dimension, e.g. messages_sent in conversation layer:///conversations/f3cc7b32-3c92-11e4-baad-164230d1df67 by user joe_smith, in which case the metric would be aggregated based on all dimensions.
For queries that specify more than one dimension, the order in which they are specified is significant. Results will be aggregated and sub-aggregated in this order.
The external Insights API is scoped by application, so the application dimension is implicit. However, internally the application id is just another dimension. This allows us (Layer) to aggregate metrics over all or a number of applications.
Not all combinations of metrics and dimensions are possible, and when more than one dimension is supported, not all orderings of dimensions are possible.
The table below lists the permitted combinations.
The query request contains a JSON object containing the start time, end time, interval, metrics and dimensions, as shown below. The start end end times are expressed in ISO 8601 format. An optional UTC offset may be specified, which will be applied to the start and end times, as well as to the results. (If the offsets for the start and end times are different (not recommended), the offset for the start time will be used for the results.) This allows results to be aggregated, for example, by days, weeks or months whose boundaries do not correspond to UTC.
(TBD: We should define convenience values for time, e.g. today, yesterday, n hours ago.)
A query may optionally specify a set of filters, which may be used to restrict the set of results. A filter is simply a list of values for a dimension. Only results that match values in the list will be included in the results.
Experimental feature: Needs further investigation, and does not need to be implemented at this time.
Queries also support an optional sub_totals attribute that contains a list of the dimensions and/or the value time.
The results will include sub-totals for these dimensions.
An optional flag totals can be set to include the overall metric totals in the result.
An alternative form of request is supported where the query information is specified as query parameters on the URL instead of as a JSON object. This is mainly for convenience when used from browsers.
POST /insights
{
"start_time": "2015-03-15T04:00:00-08:00",
"end_time": "2015-04-30T20:00:00-08:00",
"interval": "hour",
"metrics": [
"messages_sent",
"messages_delivered"
],
"dimensions": [
"app_id",
"conversation_id",
"user_id"
]
}The external endpoint for the Insights API is scoped based on app id, and authenticated based on app token as follows.
POST https://api.layer.com/apps/{app_id}/insights
The app token is included in the authentication header.
This translates to the more general query above, with app_id included in the list of dimensions and in the filters.
| Metric | Primary Dimension | Secondary Dimension | Tertiary Dimension |
|---|---|---|---|
| user_authentications | app_id | ||
| unique_user_authentications | hardware | ||
| unique_user_authentications | os_version | ||
| unique_user_authentications | sdk_version | ||
| unique_user_authentications | location | ||
| unique_user_authentications | app_id | ||
| messages_sent | hardware | ||
| messages_sent | os_version | ||
| messages_sent | sdk_version | ||
| messages_sent | location | ||
| messages_sent | mime_type | ||
| messages_sent | app_id | hardware | |
| messages_sent | app_id | os_version | |
| messages_sent | app_id | sdk_version | |
| messages_sent | app_id | location | |
| messages_sent | app_id | mime_type | |
| messages_sent | app_id | conversation_id | user_id |
| messages_sent | app_id | user_id | conversation_id |
| messages_sent | app_id | conversation_id | mime_type |
| messages_sent | app_id | user_id | mime_type |
| bytes_sent | hardware | ||
| bytes_sent | os_version | ||
| bytes_sent | sdk_version | ||
| bytes_sent | location | ||
| bytes_sent | mime_type | ||
| bytes_sent | app_id | hardware | |
| bytes_sent | app_id | os_version | |
| bytes_sent | app_id | sdk_version | |
| bytes_sent | app_id | mime_type | |
| bytes_sent | app_id | location | |
| bytes_sent | app_id | conversation_id | user_id |
| bytes_sent | app_id | user_id | conversation_id |
| bytes_sent | app_id | conversation_id | mime_type |
| bytes_sent | app_id | user_id | mime_type |
The normal response to a query will be a 200 (OK) response containing a description of the query (with any convenience values for time replaced by the actual values), followed by the results.
If the service determines that the query will take a long time to process, it may instead return a 202 (Accepted) response, with a link that can be queried to obtain the results.
The results are returned in flattened tabular form, i.e. the results will contain a list of objects, where each object corrsponds to a row in a table where the columns are the timestamp of the interval, the values of dimensions, followed by the values of the metrics.
This flattened form makes the results easier to use in display and other applications, but it increases the size of the payload because elements may be repeated.
However, most compression algorithms (e.g. gzip) will compress repeated items, so this may not be an issue.
> 200 (OK)
{
"query": {
"start_time": "2015-03-15T04:00:00-08:00",
"end_time": "2015-04-30T20:00:00-08:00",
"interval": "hour",
"metrics": [
"messages_sent",
"messages_delivered"
],
"dimensions": [
"app_id",
"conversation_id",
"user_id"
]
},
"results": [
{
"time": "2015-03-15T04:00:00-08:00",
"app_id": "layer:///apps/de456a87-11e4-b09d-164230d1df67",
"conversation_id": "layer:///conversations/f3cc7b32-3c92-11e4-baad-164230d1df67",
"user_id": "joe_smith",
"messages_sent": 32,
"messages_delivered": 30
},
{
"time": "2015-03-15T04:00:00-08:00",
"app_id": "layer:///apps/de456a87-11e4-b09d-164230d1df67",
"conversation_id": "layer:///conversations/f3cc7b32-3c92-11e4-baad-164230d1df67",
"user_id": "jane_doe",
"messages_sent": 21,
"messages_delivered": 19
},
{
"time": "2015-03-15T04:00:00-08:00",
"app_id": "layer:///apps/de456a87-11e4-b09d-164230d1df67",
"conversation_id": "layer:///conversations/940de862-3c96-11e4-baad-164230d1df67",
"user_id": "joe_smith",
"messages_sent": 14,
"messages_delivered": 10
},
{
"time": "2015-03-15T04:00:00-08:00",
"app_id": "layer:///apps/de456a87-11e4-b09d-164230d1df67",
"conversation_id": "layer:///conversations/940de862-3c96-11e4-baad-164230d1df67",
"user_id": "jane_doe",
"messages_sent": 29,
"messages_delivered": 23
},
...
]
}We may wish to have an alternative form of the response for the external endpoint that does not include the app id in every result item.
Errors responses will have an appropriate HTTP status code and the response body will contain a JSON representation of the error. The JSON object will contain a human readable message, a URL to the relevant documentation and a data object that returns data pertaining to the error.
| Name | Type | Description | Example |
|---|---|---|---|
id |
string | Unique string error identifier. | missing_property |
code |
integer | Unique numeric error code. | 12345 |
message |
string | Details of the error. | The metric name cannot be omitted |
url |
string | A URL to a reference with more info about the error. | https://developer.layer.com/insights.md#metrics |
data |
dictionary | A dictionary of supplemental data specific to the error. | { "property": "metrics" } |
code |
id |
Context | HTTP Status | Description |
|---|---|---|---|---|
| 1 | service_unavailable |
Client | 503 (Service Unavailable) |
The operation could not be completed because a backend service could not be accessed. |
| 2 | invalid_app_id |
Client | 403 (Forbidden) |
The client provided an invalid Layer App ID. |
| 3 | invalid_request_id |
Client | 400 (Bad Request) |
The client has supplied a request ID that is not a valid UUID. |
| 4 | authentication_required |
Client | 401 (Unauthorized) |
The action could not be completed because the client is unauthenticated. The response will include a nonce for satisfying an authentication challenge. |
| 5 | app_suspended |
Client | 403 (Forbidden) |
The app has been suspended. |
| 6 | user_suspended |
Client | 403 (Forbidden) |
The authenticated user has been suspended. |
| 7 | rate_limit_exceeded |
Client | 429 (Too Many Requests) |
The client has sent too many requests in a given amount of time. |
| 8 | request_timeout |
Client | 408 (Request Timeout) or None |
The server or the client timed out waiting for a request to complete. |
| 9 | invalid_operation |
Client | 422 (Unprocessable Entity) or None |
The server or client has declined to perform an invalid operation (i.e. deleting an unsent message). |
| 10 | invalid_request |
Client | 400 (Bad Request) |
The request is structurally invalid. |
| 101 | access_denied |
Resource | 403 (Forbidden) |
The authenticated user does not have access to the resource requested. |
| 102 | not_found |
Resource | 404 (Not Found) |
The resource requested could not be found. |
| 103 | object_deleted |
Resource | 410 (Gone) |
The client requested a resource that has been deleted. |
| 104 | missing_property |
Resource | 422 (Unprocessable Entity) |
A property with a required value was not supplied. |
| 105 | invalid_property |
Resource | 422 (Unprocessable Entity) |
A property was supplied with an invalid value. |
| 106 | invalid_endpoint |
Client | 404 (Not Found) |
The endpoint 'GET /insights' does not exist |
| 107 | invalid_header |
Client | 406 (Not Acceptable) |
Invalid Accept header; must be of form application/vnd.layer+json; version=x.y |
> Request with invalid app id
< 403 (Forbidden)
{
"id": "invalid_app_id",
"code": 2,
"message": "The request did not contgain a valid app id",
"url": "https://developer.layer.com/insights.md#authentication",
"data": {
"app_id": "layer:///apps/de456a87-11e4-b09d-164230d1df67"
}
}> Request with valid app id but not enabled
< 403 (Forbidden)
{
"id": "invalid_app_id",
"code": 2,
"message": "The App ID provided is not permitted to access the Insights API.",
"url": "https://developer.layer.com/insights.md#authentication",
"data": {
"app_id": "layer:///apps/de456a87-11e4-b09d-164230d1df67"
}
}> Request without an Authorization header or no bearer token in header
< 401 (Unauthorized)
{
"id": "authentication_required",
"code": 4,
"message": "The request could not be completed because the Authorization header does not contain a Layer Bearer Token.",
"url": "https://developer.layer.com/insights.md#authentication",
}> Request invalid bearer token in Authorization header
< 403 (Forbidden)
{
"id": "authentication_required",
"code": 4,
"message": "The request could not be completed because the Authorization header does not contain a valid Layer Bearer Token.",
"url": "https://developer.layer.com/insights.md#authentication",
}> Request with valid authorization, but incorrectly formatted body
< 400 (Bad Request)
{
"id": "invalid_request",
"code": 10,
"message": "The request body did not contain a valid JSON document.",
"url": "https://developer.layer.com/insights.md#requests"
}> Request with valid authorization and JSON body, but invalid data
< 400 (Bad Request)
{
"id": "invalid_request",
"code": 10,
"message": "The request body is not a valid JSON document.",
"url": "https://developer.layer.com/insights.md#requests"
}> Request with invalid value for interval
< 422 (Unprocessable Entity)
{
"id": "invalid_property",
"code": 105,
"message": "The interval must be 'hour', 'day' or 'month'",
"url": "https://developer.layer.com/insights.md#requests",
"data": {
"property": "interval"
}
}> Request with missing metrics list
< 422 (Unprocessable Entity)
{
"id": "missing_property",
"code": 104,
"message": "At least one metric must be specified.",
"url": "https://developer.layer.com/insights.md#requests",
"data": {
"property": "metrics"
}
}> Request with invalid value for metrics
< 422 (Unprocessable Entity)
{
"id": "invalid_property",
"code": 105,
"message": "A metric name was invalid.",
"url": "https://developer.layer.com/insights.md#requests",
"data": {
"property": "metrics"
}
}> Request with invalid name for dimensions
< 422 (Unprocessable Entity)
{
"id": "invalid_property",
"code": 105,
"message": "A dimension name was invalid.",
"url": "https://developer.layer.com/insights.md#requests",
"data": {
"property": "dimensions"
}
}> Request with dimensions not valid for metric
< 422 (Unprocessable Entity)
{
"id": "invalid_property",
"code": 105,
"message": "The dimensions are not valid for these metrics.",
"url": "https://developer.layer.com/insights.md#requests",
"data": {
"property": "dimensions"
}
}