GRLs (generalized regex lexers; made-up name) are a common lexer type which are represented by a
- deterministic state-based finite automaton with regex at eges (similar to generalized non-det. finite automaton),
- such that edges can not only transition to a new state but also push or pop multiple states and also issue/yield token types (e.g. "keyword", "string", "punctuation").
- additionally, the set of outgoing edges for a state is actually ordered ("list of rules"; what matches first, is gone first),
- and you can also import the rule lists from other states (e.g. have a common "comment rule list")
(Is there a formal name for it? See also https://github.com/mozilla/skywriter/wiki/Common-JavaScript-Syntax-Highlighting-Specification, a spec that apparently never took off)
| Product | Highlighting Software | Language of Lexer | Supported Lexer Types | Official Folder/Repo of Languages |
|---|---|---|---|---|
| LaTeX with minted | Pygments | Python | GRL | pygments/pygments @ GitHub |
| GitLab viewing | Rouge | Ruby | GRL | rouge-ruby/rouge @ GitHub |
| GitLab single-file editing | Ace | JavaScript | GRL | ajaxorg/ace @ GitHub |
| GitLab multi-file editing | Monaco | TypeScript | GRL | microsoft/monaco-language @ GitHub.com |
| GitHub viewing | Linguist | XML | GRL | github/linguist @ GitHub.com |
| GitHub editing | CodeMirror | JavaScript | GRL and raw lexer API (eat, eatUntil, ...) | codemirror/CodeMirror @ GitHub.com |
| VSCode, Visual Studio Code | TextMate Grammars | (enhanced?) plist files | GRL | VSCode Marketplace |
| JetBrains IntelliJ | Custom | Java, but creatable from JFlex EBNF | All (since only Java API required) | JetBrains Marketplace |
| Notepad++ | Custom | XML | weak; can only specify common string occurrences and operators to match; not even regex-based | notepad-plus-plus/userDefinedLanguages @ GitHub.com (semiofficial) |
| jEdit (not major product; but relevant for our purposes) |
Custom | XML | GRL | ?, see also http://www.jedit.org/users-guide/writing-modes.html |
Relevant and implemented for MMT surface syntax:
- ✓ Pygments (for LaTeX minted in papers)
- ✓ Rouge (for GitLab online highlighting)
- ✓ CodeMirror (for Jupyter and GLIF, see https://github.com/KWARC/GLIF/blob/master/glif_kernel/js/kernel.js)