Skip to content

Instantly share code, notes, and snippets.

@briandoconnor
Last active June 24, 2021 19:03
Show Gist options
  • Save briandoconnor/55eeae7b8560b3df691e495c227ef039 to your computer and use it in GitHub Desktop.
Save briandoconnor/55eeae7b8560b3df691e495c227ef039 to your computer and use it in GitHub Desktop.
DRS Pagination
- See issue [#29](https://github.com/ga4gh/TASC/issues/29)
- See issue [#325](https://github.com/ga4gh/data-repository-service-schemas/issues/325)
- Question 1: are we settled on using the GitHub pagination style?
Query params:
- optional:
- per_page
- page
Links: header
- required:
- next
- optional:
- last
- previous
- first
PageNum: header
- page (for information) / self-link
Consistency and maintaining state in APIs... implementation and spec clarification might be needed
Error Codes:
- what happens when the "next" request is no longer valid? Depends on API but we may need common use of error code
Look for the RFC that "Link:" is based on... benefit with clients? https://github.com/ga4gh/TASC/issues/29#issuecomment-838673382
- Notes
- RFC 5988 mentioned in this page -> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Link
- https://github.com/ga4gh/TASC/issues/29#issuecomment-838673382
- Question 2: At what level are we paginating? For object responses or within?
- do we paginate at the object level? Pages of objects?
- do we paginate within an object as well (since bundles can be very large)
Notes:
- DRS server using bundles: NCBI DRS server (SRA). expand=true is extremely slow.
- Expand param not compatible with pagination?
- link header for list of objects
- don't include expand?
- have a child endpoint to page children?
- example from DRS SRA: https://github.com/ga4gh/fasp-scripts/blob/master/notebooks/drs/SRA_IDs_and_bundling.ipynb
- is it a breaking change if we are talking about a new endpoint? can we keep the expand parameter for GET single DrsObject, but disallow expand for GET or POST list of DrsObjects
- connections... being able to travel between parents/children... just need the context to know where you are (next, previous, parent, child)
- I hate to add another breaking change... But what if we had a new object type of bundle and it is a json file that is hosted publicly on drs that someone just downloads and looks through
Next Steps:
- link GitHub approach for listing batch operation objects
- explore alternative endpoint for paging/navigating through nested "contents" for a give object, which can then use the same paging approach (explore other primatives to move in the set for DRS, for example parent/child)
{
"id": "string",
"name": "string",
"self_uri": "drs://drs.example.org/314159",
"size": 0,
"created_time": "2019-08-24T14:15:22Z",
"updated_time": "2019-08-24T14:15:22Z",
"version": "string",
"mime_type": "application/json",
"checksums": [
{
"checksum": "string",
"type": "sha-256"
}
],
"access_methods": [
{
"type": "s3",
"access_url": {
"url": "string",
"headers": "Authorization: Basic Z2E0Z2g6ZHJz"
},
"access_id": "string",
"region": "us-east-1"
}
],
"contents": [
{
"name": "string",
"id": "string",
"drs_uri": "drs://drs.example.org/314159",
"contents": [
{}
]
}
],
"description": "string",
"aliases": [
"string"
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment