notifications-apis.md

Notifications APIs

Based on current designs, we need new APIs that support retrieving lists of notifications, filtered by the requesting user, that allow for:

• Listing all recent (e.g., most recent 50) notifications, grouped by origin.
• Listing all notifications within an origin, with optional paging.
• Deleting all notifications, irrespective of origin.
• Deleting all notifications within an origin.
• Deleting a single notification.

With that need, I'm thinking of the following new endpoints, respectively:

• GET /notifications
• GET /notifications/:origin (with optional ?range=50)
• DELETE /notifications
• DELETE /notifications/:origin
• DELETE /notifications/:origin/:id

Technically we shouldn't need the origin for that last one, but I've included it to more easily differentiate it from DELETE /notifications/:origin.

Example Responses

Note that ...body below represents the body of a notification message, which could take different forms, each of which is detailed in Example Notification Bodies below.

GET /notifications

[
  {
    origin: "core",
    notifications: [
      {
        ...body
      },
      {
        ...body
      }
    ]
  },
  {
    origin: "cnunciato",
    notifications: [
      {
        ...body
      }
    ]
  }
]

GET /notifications/:origin

{
  range_start: 0,
  range_end: 49,
  total_count: 123,
  data: [
    {
      ...body
    },
    {
      ...body
    }
  ]
}

DELETE /notifications
-> 204

DELETE /notifications/:origin
-> 204

DELETE /notifications/:origin/:id
-> 204

Example Notification Bodies

Initally we're interested in the following set of notifications:

1. Builder started building a package in one of my origins, based on a GitHub webhook.
2. Builder built (or failed to build) a package in one of my origins.
3. Builder promoted a package in one of my origins (e.g., to the unstable channel, after a build).
4. A user promoted a package in one of my origins.
5. A user demoted a package in one of my origins.
6. A user submitted a build request for a package in one of my origins.
7. A user uploaded a package to one of my origins.
8. A package that is a dependency of one of the packages in any of my origins was promoted to stable.

Suggested responses for these are enumerated below.

1. Job Started

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "job-started",
  category: "info",
  origin: "core",
  data: {
    package: "node"
    group_id: "12345",
    job_id: "67890",
    status: "Processing",
    source: "vcs:github",
    message: "This is the first line of the commit message"
  }
}

2. Job Finished (or Failed)

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "job-finished",
  category: "error",
  origin: "core",
  data: {
    package: "node",
    version: "1.2.3",
    release: "2017010100000000",
    group_id: "12345",
    job_id: "67890",
    status: "Failed",
    source: "vcs:github",
    message: "This is the first line of the commit message",
  }
}

3. Package Promoted (by Builder)

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "package-promoted",
  category: "info",
  origin: "core",
  data: {
    package: "node",
    version: "1.2.3",
    release: "2017010100000000",
    source: "bldr",
    channel: "unstable"
  }
}

4. Package Promoted (by User)

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "package-promoted",
  category: "info",
  origin: "core",
  data: {
    package: "node",
    version: "1.2.3",
    release: "2017010100000000",
    channel: "stable",
    source: "user:cnunciato",
    message: "Releasing! Yay!"
  }
}

5. Package Demoted

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "package-demoted",
  category: "warning",
  origin: "core",
  data: {
    package: "node",
    version: "1.2.3",
    release: "2017010100000000",
    channel: "stable",
    source: "user:cnunciato",
    message: "Some stuff happened, so we had to demote."
  }
}

6. Job Scheduled

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "job-scheduled",
  category: "info",
  origin: "core",
  data: {
    package: "node"
    group_id: "12345",
    job_id: "67890",
    source: "user:cnunciato",
    message: "Do we feel like a message might be useful here, too?"
  }
}

7. Package Uploaded

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "package-uploaded",
  category: "info",
  origin: "core",
  data: {
    package: "node",
    version: "1.2.3",
    release: "2017010100000000",
    channel: "unstable",
    source: "user:cnunciato"
  }
}

8. Dependency Promoted

{
  id: 1234567890,
  timestamp: 1512687331658,
  type: "dep-promoted",
  category: "info",
  origin: "core",
  data: {
    package: "node",
    version: "1.2.3",
    release: "2017010100000000",
    channel: "stable",
    dependent: {
      origin: "cnunciato",
      name: "ghost,
      version: "1.2.3",
      release: "2017010100000000"
    }
  }
}

This is awesome! Thanks so much for taking the time to write all this up. It's really comprehensive.

I have a few questions about the example responses.

• Where does category come from?
• For the timestamp, is an integer timestamp preferred?
• Where do the various messages come from?

@raskchanky The category is just something that seemed useful in order to distinguish messages that are informational from those that might be considered more important to users, like build failures or dependency rebuilds/promotions -- things a user would probably want to take some action on.

On the dates, I personally like the simplicity of integer timestamps because no timezones or client-side parsing irritations, but if we aren't using them elsewhere, regular ol' ISO date strings are okay, too.

The messages will either be submitted by users (e.g., when I promote a package, I might provide an optional message in the dialog or at the CLI describing the reason for that promotion) or by Builder with a webhook callback from GitHub.

This looks like a good start. There are some additional things to discuss and close on before we commit to an API since this will potentially have a huge impact on the service scalability (eg, we need to decide on how/where we are going to store the activity feed, what model we are going to use to build it (read time vs. write time), what front end we will use for notifications, etc).

For the API, the main issue I see is that this proposal looks like a polled model. We might want to consider investigating whether we we should design around a 'web push' model instead. That decision will also inform the overall architecture of the notification solution.

@chefsalim I totally agree and share that concern, yeah. In some form, we're going to need to be able to get these on demand, or close to that, but it'd be ideal not to have to poll for changes and just be notified on an open connection somehow, via socket or SSE. When we looked at this a while back, the Rust options for both were pretty limited, but that might've changed over the past few months. Definitely something we should investigate though.

That said, we can poll somewhat intelligently, or not even poll at all at first, in order to keep the traffic to a minimum.