ambarishsatheesh

Iterator-based parsing is an efficient and straightforward way of writing recursive-descent parsers in Python. Here’s an outline:

use an iterator to split the source into a stream of tokens or token descriptors.
pass the iterator’s next method and the first token to the toplevel parser class.
use separate functions, where appropriate, for individual grammar rules. pass the next method and the current token on to these functions as well.
to check the current token, inspect the token argument. to fetch the next token, call the next method.
Here’s a simple example. This is a limited but hopefully safe version of Python’s eval function, which handles strings, floating point values, integers, and tuples only. Tuples can be nested.

 
import cStringIO, tokenize