Here is the lexer description file:
[a-zA-Z_]\w* identifier
\d+ integer
\+ ADD
\- SUBTRACT
\s+ whitespace
And here is what I type into stdin:
this - is + an - expr3ss10n
Here is a transcript of the output:
stephen at greed in ~/repos/cky (master)
$ bin/release/main -l etc/expression.txt
this - is + an - expr3ss10n
identifier: at index=0, length=4
whitespace: at index=4, length=1
SUBTRACT: at index=5, length=1
whitespace: at index=6, length=1
identifier: at index=7, length=2
whitespace: at index=9, length=1
ADD: at index=10, length=1
whitespace: at index=11, length=1
identifier: at index=12, length=2
whitespace: at index=14, length=1
SUBTRACT: at index=15, length=1
whitespace: at index=16, length=1
identifier: at index=17, length=10
whitespace: at index=27, length=1
This brought to you using 100% C with no libraries except libstephen and my own regular expression code!