entity sync spec

entity-sync.md

incremental entity fetch

entity-change-log

Description:

This endpoint returns a paginated list of changes in the system. Calling systems can use this to periodically fetch the list of entities that have been modified over a given time period.

In common usage the caller would periodically poll this endpoint, using the to value returned from the previous call as the from value on the next call.

After receiving the change events, the caller can optionally call other endpoints as appropriate to fetch the current value for that entity.

Example request:

GET /contact-change-log?from=1495052932000&to=1495139308014&entityType=contact

Example response:

{
    "to": 1495139308014,
    "rows": [
        {
            "date": 1495052932001,
            "entityType": "contact",
            "eventType": "create",
            "entityId": "3029-2320202-032"
        },
        {
            "date": 1495052932010,
            "entityType": "contact",
            "eventType": "delete",
            "entityId": "30393-2320202-032"
        },
        {
            "date": 1495052932020,
            "entityType": "group",
            "eventType": "update",
            "entityId": "9999-2320202-032"
        }
    ],
    "next": "akzkd93920s0sd0s0"
}

Query string params:

• from - Required. Date to search logs from (inclusive).
• to - Optional. Date to search logs to (exclusive). If not provided, current time is used as the upper bound.
• entityType - Optional. If provided, only entities of this type are returned. If omitted, events for all entities are returned. May be specified more than once, in which case events for ANY of the entityTypes specified should be returned.
• limit - Optional. If provided, specifies the max # of rows to return per request. (default=1000, max=1000).
• next - Optional. If provided, the next page of results will be returned.

Time format:

All times are encoded as Unix time in milliseconds (e.g. as returned by new Date().getTime() in Javascript or System.currentTimeMillis() in Java). This avoids time zone considerations and can be easily converted to native date formats in most languages.

Response

Sort order: Results are returned in ascending time order.

Return payload is a JSON object with this shape:

{
    "to": 1495139308014,
    "rows": [ ],
    "next": ""
}

Fields in this object:

• to - The upper bound date (exclusive) reflected by this result set. The caller may store this and use it as the from parameter to fetch the next range of changes. This value should always be returned.
• rows - Current page of results.
• next - Token that can be used on a subsequent request by the caller to fetch the next page of results. This field should be omitted from the response if no more pages are available.

Each row element is a JSON object with this shape:

{
    "date": 1495052932001,
    "entityType": "contact",
    "eventType": "create",
    "entityId": "3029-2320202-032"
}

Fields in this object:

• date - Unix time in milliseconds that the event occurred in the system of record
• entityType - Type of entity. This will likely be some enumeration.
• eventType - Type of event. Enumeration with these values:
  • create
  • update
  • delete
• entityId - Immutable identifier for entity in the system of record.

Issues

• How should entity associations be communicated? For example, contact/group association is not necessarily a property of either entity

  • Could be modeled as a separate entityType, but there are two entityId values that are relevant
  • Could be implemented as two log rows, one per entity entityType=contact-group-assoc entityId=<contact-id>, entityType=group-contact-assoc entityId=<group-id>
  • Alternately this could be implemented with an array of entity type/id pairs, perhps with associate and dissociate eventTypes.

    {
        "date": 1495139308014,
        "eventType": "associate",
        "entities": [
            {
                "entityType": "contact",
                "entityId": "abc"
            },
            {
                "entityType": "group",
                "entityId": "zyx"
            }
        ]
    }