Skip to content

Instantly share code, notes, and snippets.

@aarongough
Created July 13, 2010 22:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aarongough/474659 to your computer and use it in GitHub Desktop.
Save aarongough/474659 to your computer and use it in GitHub Desktop.
=begin
%%{
machine simple_lexer;
integer = ('+'|'-')?[0-9]+;
float = ('+'|'-')?[0-9]+'.'[0-9]+;
assignment = '=';
identifier = [a-zA-Z][a-zA-Z_]+;
main := |*
integer => {
emit(:integer_literal, data, token_array, ts, te)
};
float => {
emit(:float_literal, data, token_array, ts, te)
};
assignment => {
emit(:assignment_operator, data, token_array, ts, te)
};
identifier => {
emit(:identifier, data, token_array, ts, te)
};
space;
*|;
}%%
=end
%% write data;
# %% this just fixes our syntax highlighting...
def emit(token_name, data, target_array, ts, te)
target_array << {:name => token_name.to_sym, :value => data[ts...te].pack("c*") }
end
def run_lexer(data)
data = data.unpack("c*") if(data.is_a?(String))
eof = data.length
token_array = []
%% write init;
%% write exec;
puts token_array.inspect
end
@bluemont
Copy link

I think you meant to name this file ragel_lexer.rl since it has to be processed by Ragel first. Thanks! Your article over at http://thingsaaronmade.com/blog/a-simple-intro-to-writing-a-lexer-with-ragel.html has been very helpful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment