Puzzmo 2.0 RFC - Feb 19th
How can we architect the comms between the API, games and puzzles? This is a reasonably educated guess, which I'm opening for ideas and comments!
To make this work, we need to move a chunk of logic which is hardcoded into the server into something which can vary either:
- through an admin (e.g. a custom leaderboard admin panel for curating a daily meta-leaderboard)
- codegen'd with variations in the same way to do the bonus infra on a daily 3 weeks in advance
- from a puzzle (e.g. giant typeshift, with a leaderboard based on how many words you find with an “S” in it)
This likely means that we likely need a sort of schema for defining how a leaderboard / game variant can work, as the two are somewhat interlinked, and so the game may receive info from both the puzzle file and the Puzzmo API for bootstrapping
Variants being dynamic means we can’t operate on the same ‘pick a prefix from a single folder’ either. Perhaps we could work with something like:
spelltower4
├─ variants/
├─ 1-100.txt
├─ 1-101.txt
└─ ...txtWhere the defaults are still plucked from the root, but that special versions can come from subfolders. It kinda depends on how controlled we want the algorithm, if any variant is fine for a particular slot on a daily (e.g. instead of the bonus) then it could be a reasonable tradeoff WRT complexity.
If it's not, then we might have to think of tagging systems either in puzzle file names, folder names etc. Solvable, but more complex.
The puzzle files themselves probably need to change, I’m starting to side on “it’s time to use JSON” which sucks because multi-line JSON is unreadable to humans (and for admin reasons, it's important we keep some readability on puzzle text files).
We may want to use the front-matter header system seen in Jekyll/Hugo, where a puzzle file could instead look like:
---
{ "thing": 1, "other": 2 }
---
5
5
5
25
SGEOS
ICLDO
ALHTA
COA**
**C**
SLEDSIt’s trivial to implement and while being a little bit messy, I think it’s a pretty reasonable trade-off of readability and automation. We could consider supporting this front-matter at both the start and the end of a file too.
JSON is probably the right thing to do here, it’s built into all runtimes we use for Puzzmo, doesn't require a dependency for clients and isn’t an eval.
I think schema-wise we may want to be looking at something in this space:
type PuzzleHeader = {
_v: 1
name?: string // For showing in puzzmo.com
description?: string // Markdown text which shows on initial loading of the game
constraints?: { [game specific, imagine spelltower min score, RBC min turns etc] },
leaderboards?: [
{ name: string, staticID: string, type: string, bitID: string, formatString?: string... }
],
metadata?: { [game or variant specific] }
}The front-matter is basically for talking to the API, which does not understand the puzzle file format.
This front-matter object would likely get tweaked via the API before being passed to the game via the bootstrap data, for example the API may pass in some default leaderboard metadata, or constraints (“we need to hide the hint button etc”.)
It may not even be the individual games responsibility to know the front-matter exists (e.g. the app / api / jig handle that and the game still sees the original puzzle file)
The API will probably do strict validation of this JSON during imports from the pool. It would be considered an "untrusted" input, because of its scope for varienty.
Today, a game is completed and all of the leaderboard processing happens on the server. We take info from pipeline stats (temporary), variables defined on a gameplay (permanent) and then create a db entry for each leaderboard that we want to apply a score for a game.
If we want to do any sort of histogram, we need to be able to look at all gameplays for a puzzle and generate the buckets based on a number in a field on a table. So, for current leaderboards and completion histograms the definition of a leaderboard includes the field which can be used to lookup the value on a gameplay (like metric1).
This field lookup being dynamic in the API is ok, I use typescript to not get it wrong, this being dynamic and coming from the puzzle requires some ahead-of-time validation and is a light worry.
This is done so that a histogram is accurate for all games, not just ones with leaderboard scores, which is a paid subset of all games played. We show the histogram to non-paying folks, and I think give their location. If this isn’t so important, we can move to make a leaderboard record be the source of truth here. This removes the dynamic nature of the current system, because we’re working against a different model (a leaderboard record’s value is always a number in the same field)
But, I think we may end up needing to have the game create and send up the leaderboard entries instead. If we want to have weird leaderboards, the code which decides how many “s”es were used in the typeshift really should be at the source. Otherwise we are syncing three systems (generation/game/api) to set a single leaderboard entry, and there’s a lot of space for that to go awry.
This feels somewhat risky, like I trust we can safely make changes to the API (and write tests, use staging etc) for all our leaderboard code and have it deployed in ~30m. Having it in the puzzle and game systems moves important puzzmo app infra into a place where we have low visibility and different team priorities.
To make this work I wouldn’t be surprised to see us changing the completion data to look like:
type PuzzleCompletion = {
pipelineData: any[] // still necessary for user stats / api stuff
gameplay: Gameplay // same as before, with same metrics etc
bits: [ {
id: string
value: string
puzzmoPoints?: number
stringValue?: string
transitory?: true
}]
leaderboards: [{ ... see comment below }]
checksum: string
}A few example bits:
Spelltower
bits: [{ id: "time", value: 325112 }, { id: "score", value: 3455 }, { id: "longest-word", value: 8, stringValue: "scottish", puzzmoPoints: 200 }, { id: "time-to-almost-clear", value: 1233, puzzmoPoints: 400 }, { id: "full-complete", value: 1, puzzmoPoints: 1000 }]Then if the game knows about its weird variant, it simply adds extra bits which are marked as transitory
bits: [{ id:"letters-with-s", value: 4, transitory: true }, { id: "time", value: 325112 }, { id: "score", value: 3455 }, { id: "longest-word", value: 8, stringValue: "scottish", puzzmoPoints: 200 }, { id: "time-to-almost-clear", value: 1233, puzzmoPoints: 400 }, { id: "full-complete", value: 1, puzzmoPoints: 1000 }]“bits” is very open to naming bike-shedding, but it needs to feel far from “user stats” so that we’re not muddying the technical glossary.
On the long term, we may be able to switch out all pipelineData into bits, some of which are transitory and not important to keep around outside of completion pipeline processing.
e.g. a leaderboard tracking info across many games
Today a leaderboard is defined on a game basis, sorta in its own file where we give it a bunch of context from the game completion (pipeline data) and then look at what’s available to make leaderboards from. A leaderboard is ‘created’ when the first record is added to it (which is why on partners we only show “no one has submitted to this leaderboard yet)
We could instead have a weekly/daily leaderboard which is pre-computed, and take into account all the bits from all of today’s daily games and can be a mix of things like:
- Time to perfect completions on spelltower and flipart
- Which is bits should exist from the user today: spelltower: ‘full-complete’, flipart: ‘speed-run’
- Score is aggregate of: spelltower ‘time’, flipart ‘time’
I’m sure a lot of our weird leaderboard ideas can be a summary of:
- A filter step
- An aggregate step
Which we can define in the db, allowing them to either be randomly generated or human created by non-programmers.
The user stats history system would be based on bits. Perhaps we store the last 30 values in the db, and then for paying folks a per-month JSON blob in cold storage in the CDN. The 30 values is probably enough for the histograms, but not really enough for ‘today you got the lowest time’
We will have some tension here because we may need to index these against difficulty. I don’t have an answer for this ATM, and the attempts I have made so far in puzzmo’s user stats (Mon-Fri best of X) are OK, but not satisfying.