Last active
March 28, 2019 12:08
-
-
Save MonkeyDo/2bb3add231979834e1b3fd034f109cd3 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Auth | |
- No need for an application page, we will use MetaBrainz’ (currently on MusicBrainz website (https://musicbrainz.org/account/applications). | |
- No need to generate or store tokens, only handling auth with existing token for existing user. No need to add a DB table to store tokens. | |
- Possibly a UI change will be needed with a shortcut or simple page that redirects to MBs applications page | |
Tech stack | |
- Any good reason why we should use Koa over Express, considering we already use Express for the web server and can possibly reuse some code? https://github.com/koajs/koa/blob/master/docs/koa-vs-express.md | |
- I don’t think we need Kong. The auth and rate-limiting can be done in Express with the help of packages (links below). Kong will introduce extra requests = extra latency | |
- Caching with Redis can also be done with Express (links below) | |
Rate-limit for ExpressJS with Redis: https://www.npmjs.com/package/rate-limit-redis with https://github.com/nfriedly/express-rate-limit | |
Another rate limiter for Express https://github.com/animir/node-rate-limiter-flexible/wiki/Express-Middleware | |
Exponential back-off: https://www.npmjs.com/package/express-slow-down | |
The webserver uses Passport for Express to manage auth: http://www.passportjs.org/ | |
Search | |
- Search endpoint is likely to only return a simplified representations (bbid, default alias) rather than full entities. | |
- Once Solr is implemented we can develop the schema and DB access to return full entities | |
What to return in the queries? | |
- Anything that is not an Entity needs to be returned with the associated Entity (if requested). | |
- Anything that is an Entity needs to be fetched as a separate request (no mechanism to automatically fetch linked entities) | |
- The aim is to save on SQL joins wherever possible. Passing a flag to request data linked to an entity, as described for relationships in the proposal, is fine. | |
- Instead of ‘include’ subqueries that MB has (https://musicbrainz.org/doc/Development/XML_Web_Service/Version_2#Subqueries), get the client to batch multiple separate requests (fetch author, process relationships by type, new request to fetch array of Work entities bbids). Avoid SQL joins as much as possible to save time and send only specifically requested data = lower latency for everyone | |
- `GET publication/{id:BBID}/edition` for example does not follow that logic: instead, a user needs to fetch the publication with `editions` flag set to true, then fetch Editions by bbids from the array returned by the first request OR see below: | |
Entity endpoints should allow to browse by linked entities: | |
- For example: I should be able to search for Works by Author X using `GET /work?author={authorBBID}` |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@mwiencek The proposal I was referring to was from 2018. In the meantime the same applicant has published a new proposal for this year.
Could you also have a look at that new proposal, and chime in, especially concerning plans for ws/3 (I wasn't at the summit discussion so i only have a vague idea)?
So instead of includes, for example have separate endpoints, something like
/{entity}/aliases
,/{entity}/relationships
?