Skip to content

Instantly share code, notes, and snippets.

@posix4e
Created January 24, 2019 00:31
Show Gist options
  • Save posix4e/9468f2939da84a174b05044db72ba598 to your computer and use it in GitHub Desktop.
Save posix4e/9468f2939da84a174b05044db72ba598 to your computer and use it in GitHub Desktop.
New key descriptions
Keys will be prefixed by the name of the processing strategy
Input keys will begin with the prefix input. And all keys nessecary ....
Output keys are keys meant to be dumped to s3, and are written once. Aka, you listen to output dump and delete it.
This flexabiliy will mean we can process with cron, events or listeners.
Temp keys are keys that are being written to as a job is being completed, and have values, meant to be extended,
but data is never removed from a temp key. A good example of a temp key is the list of captchas (ekeys) associated
with a task when the recording oracle detects the task is complete, it renames it from temp to output for dumping.
Temp keys can be dumped to get partial results on a bucket. If temp keys are orphaned, they automatically expire after a week.
- A session key is a temp key until the session is completed and then it renamed as an output key.
- For instance all changes to the client key are logged in a list for temp for the session, and then when the session is finito,
it is rolled up and inserted into output to the session output data, and also active keys for later client connections.
Active keys: can be overwritten and changed, but the source of truth is always stored in postgresql and synced down.
- For instance the webmaster can change the siteconfig's difficulty
- Interestingly although client_confidence can be an output key, site confidence is an active key.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment