Skip to content

Instantly share code, notes, and snippets.

@moritz
Created September 5, 2017 19:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save moritz/46b8ff0a9f44200a0a23869bbf63a90e to your computer and use it in GitHub Desktop.
Save moritz/46b8ff0a9f44200a0a23869bbf63a90e to your computer and use it in GitHub Desktop.
Parsing an indention-based language (like Python)
my $sample = q:to/EOF/;
a = 1
if a:
x = 1
y = 2
if x + 1 < 3:
z = x + y
a = z * 2
b = 5
EOF
constant NEW_INDENTATION = Any.new;
grammar Pythonesque {
method handle_indentation($match) {
my $current = $match.Str.chars;
my $last = @*INDENTATION[*-1];
if $last === NEW_INDENTATION {
my $before = @*INDENTATION[*-2];
if $current > $before {
@*INDENTATION[*-1] = $current;
@*SCOPES.push([]);
}
else {
die "Inconsistent indentation: expected "
~ "more than $before, got $current spaces";
}
}
elsif $current > $last {
die "Inconsistent indentation: expected "
~ "at most $last, got $current spaces";
}
elsif $current < $last {
my $idx = @*INDENTATION.first(:k, { $current == $_ });
if defined $idx {
for $idx + 1 .. @*INDENTATION.end {
@*INDENTATION.pop;
@*SCOPES.pop;
}
}
else {
die "Unexpected indentation level: $current.";
}
}
}
token TOP {
:my @*INDENTATION = (0,);
:my @*SCOPES = ([],);
<line>*
}
token line {
^^ ( \h* ) { self.handle_indentation($0) }
<statement> $$ \n*
}
token ws { \h* }
proto token statement { * }
rule statement:sym<if> {
'if' <expression> ':'
{ @*INDENTATION.push(NEW_INDENTATION) }
}
token statement:sym<expression> {
<expression>
}
rule expression { <term> + % <operator> }
token term { <identifier> | <number> }
token number { \d+ }
token identifier { <:alpha> \w* }
token operator {
<[-+=<>*/]> | '==' | '<=' | '>=' | '!='
}
}
say so Pythonesque.parse($sample);
try Pythonesque.parse(q:to/EOF/);
if 1:
a = 2
b = 3
c = 4
EOF
note $!.message;
try Pythonesque.parse(q:to/EOF/);
if 1:
a = 2
b = 3
EOF
note $!.message;
try Pythonesque.parse(q:to/EOF/);
if 1:
a = 2
b = 3
EOF
note $!.message;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment