"Roll out non-breaking changes to biz ops schema without requiring us to redeploy any applications"
- Schema change propagates to all consumers on creation of git semver tag
- No dramatic differences in how biz-ops-schema library is used in each app
- API updates before others
- Able to easily work with local data when npm linking/working offline
- Nothing picks up major releases of schema without installing new version of node module
- If the API is on v2 and other apps still on v1, nothing breaks (major releases will typically mean the structure of the data in the schema files/library has changed, but the actual API endpoints won't have changed, so should still work fine for all consumers)
- A semver release x.y.z of biz-ops-schema releases a new version of the packaage and deploys aschema file to {s3 bucket}/vx/latest.json
- The file's structure will be:
{ schema: //the entire schema as json version: //git commit hash }
- The biz-ops-schema library can be configured with a BASE_URL to poll for schema updates
- The API will be configured with {s3 bucket}
- All other apps will be configured with biz-ops-{staging}.api.ft.com/schema (or could go via api.ft.com/biz-ops, but want to avoid having to pass creds. *new thought - coudl the API post to {s3 bucket}/{prod|staging}/{version}? How would that work?)
- Any app with the library installed will poll
${BASE_URL}/v${version of library, read from package.json}/latest.json
- When the content retrieved from the app is detected to have a different
version
property, the cached data is replaced wit the new - The library also fires an event on update (might need an on before update event too). In the API this is listened to and used to
- generate a new graphQL API, which replaces the last one
- serve the new schema from its
/schema/vN/latest.json
endpoint
- If a request comes in for a different version of the schema to the one the api is using, it is proxied through to s3
So for all apps except the API, there's no change really, other than having to add a little config to the biz-ops-schema library In the API there's a bit to do, but it's not ridiculously difficult, though perhaps a little unintuitive and weird
For managing particularly tricky things I'm thinking there should also be an easy way to put an application into 'just use the schema you have installed' mode. That might actually be a better way to manage major version changes than proxying through to s3 for previous schemas: disable autoupdate until everything is on the same version, then turn back on... but that requires a level of coordination this whole project is designed to avoid
We have a balance here between each app having direct access to the schema (and needing to be rebuilt each time it changes) and the api being the host of the schema and each consumer calling the api for meta data.
The benefit of the former is that anyone can experiment with the schema at a data level or at a meta level without needing a private version of the api (they just need a local copy of the schema). The effect of the latter is the api owns everything and becomes a bottleneck to quick local development and latency for consumers due to the need for network requests to the api.
What we really need is dynamic npm inclusion at runtime! We would then have the new schema automatically included into the api and its consumers each time we published the npm.