gistfile1.txt

Even on the fastest gpus that we could realistically use, railbird's video
processing pipeline can only run at about 2x real time. If we were to wait for
users to finish recording their tracking sessions to start uploading video to
our backend it could take hours for them to have complete statistics, at which
point they will likely have moved on to thinking about something else. The
obvious solution here is streaming, but unfortunately, pretty much every
streaming protocol in existence is optimzed to minimize latency at all costs.
They achieve this by adjusting the bit rate or dropping frames to synchronize
the stream closely with the recording time, whenever the connection bandwith
becomes spotty or cuts out momentarily. These sacrifices in the name of
minimizing latency are problematic because maintaining a steady frame rate and
uninterrupted visual continuity of the pool table events is critical for our
computer vision systems to reliably understand what is happening on the table.

We solved this problem by simply inventing our own protocol, which we like to
describe as a live upload, rather than a streaming protocol. It's quite simple:

* Video is encoded into short self contained chunks, where we are careful to make sure to split at keyframes.

* Our native camera module provides a callback that indicates when a new chunk is ready. These callbacks have this signature:

#+begin_src typescript
function handleUploadChunkReady({ filepath, index }: { filepath: string; index: number })
#+end_src

* The backend provides this gql api:

#+begin_src graphql
type GetUploadLinkReturn {
  uploadUrl: String!
  headers: [Header]!
}

type Mutation {
  createUploadStream(videoMetadata: VideoMetadataInput!): Int!
  getUploadLink(videoId: Int!, segmentIndex: Int!): GetUploadLinkReturn!
}
#+end_src

* Notice that this uri has a quirk in that you have to request an upload link from the gql api and then make a restful PUT request to the returned link

** Not super important, it is a detail that had some smaller impacts on the details of our design

The question is pretty open ended, and we're not looking for any answer in
particular: How would you design the code that handles this interaction?