Skip to content

Instantly share code, notes, and snippets.

@asterite
Created October 14, 2010 17:40
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save asterite/626637 to your computer and use it in GitHub Desktop.
Save asterite/626637 to your computer and use it in GitHub Desktop.
An S-Expression lexer/parser. Later i found out with regexps it's much faster to code and it actually works faster.
#!/usr/bin/ruby
class Lexer
def initialize(str)
@str = str
@p = 0
end
def next_token
while true
case c
when nil
return @token = nil
when ' ', "\n", "\r", "\t"
@p += 1
next
when '('
@p += 1
return @token = :'('
when ')'
@p += 1
return @token = :')'
when '"'
start = @p
while c! != '"' && c != nil
end
@p += 1
return @token = @str[start + 1 ... @p - 1]
when '0' .. '9', '+', '-'
start = @p
@p += 1
while true
case c
when '0' .. '9'
@p += 1
else
break
end
end
return @token = @str[start ... @p].to_i
when 'a' .. 'z', 'A' .. 'Z', '_'
start = @p
while true
case c
when 'a' .. 'z', 'A' .. 'Z', '0' .. '9', '_'
@p += 1
else
break
end
end
return @token = @str[start ... @p].to_sym
else
raise 'Invalid character'
end
end
end
def c
x = @str[@p]
x ? x.chr : nil
end
def c!
@p += 1
c
end
end
class Parser < Lexer
def self.parse(str)
Parser.new(str).parse
end
def parse
exp = []
while true
case next_token
when :'('
exp << parse
raise 'Missing closing parenthesis' if @token != :')'
when :')'
break
when String, Fixnum, Symbol
exp << @token
when nil
break
end
end
exp
end
end
p Parser.parse('(this (is a number 1( example "s-expression")))')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment