Skip to content

Instantly share code, notes, and snippets.

@agudulin
Created February 28, 2012 19:59
Show Gist options
  • Save agudulin/1934741 to your computer and use it in GitHub Desktop.
Save agudulin/1934741 to your computer and use it in GitHub Desktop.
Flex: lexer for python code
%{
#include <string>
#include <iostream>
using namespace std;
#define YYSTYPE string
#include "pyparser.tab.h"
%}
identifier [_a-zA-Z][_a-zA-Z0-9]*
comment "#"[^\n]*
quote1 "\'"[^'\\]*"\'"
quote2 "\""[^"\\]*"\""
%%
and|elif|global|or|assert|else|if|except|pass|break|import|print|exec|in|raise|continue|finally|is|return|for|lambda|try|del|from|not|while yylval = yytext; return DEFINED;
class yylval = yytext; return CLASS;
":" yylval = yytext; return COLON;
"." yylval = yytext; return DOT;
"(" yylval = yytext; return OBRACKET;
")" yylval = yytext; return EBRACKET;
{identifier} yylval = yytext; return ID;
{comment} ; /* skip comment */
{quote1} ; /* skip messages in single quotes */
{quote2} ; /* skip messages in double quotes */
[ \t]+ ; /* skip whitespaces */
\n\r|\r\n|\n|\r ; /* skip new line */
. yylval = yytext; return OTHER;
%%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment