Skip to content

Instantly share code, notes, and snippets.

Last active Sep 21, 2020
What would you like to do?

Toy example for token tracking:

Toy example for token tracking.

Top: Sequential revisions R1, R2, R3, and R4 of a dummy article from left to right. Color of underlining indicates origin revision. Bottom: The token tracking data after revision R4, showing all tokens that have so far appeared in the article, if currently deleted or not. Each token is linked to a unique token ID (note some identical string values for distinct tokens), the revision ID of original introduction, and each revision in which a token was possibly a) deleted and b) reinserted (both are lists). Arrows in the table indicate undo actions among revisions (not yet part of the API output).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment