Skip to content

Instantly share code, notes, and snippets.

@willharrison

willharrison/gist:3f61be462b916d546601 Secret

Created May 8, 2015 23:02
Show Gist options
  • Save willharrison/3f61be462b916d546601 to your computer and use it in GitHub Desktop.
Save willharrison/3f61be462b916d546601 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.rb
def tokenize(code)
code.chomp! # Remove extra line breaks
tokens = [] # This will hold the generated tokens
indent_stack = []
while i < code.size
chunk = code[i..-1]
if identifier = chunk[/\A([a-z]\w*)/, 1] # <= what is this doing?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment