notes and information about the config parser system being worked on for MMDLib

config parser - example file.md

The following is an example of a valid config file - modified to include comments where the original of this (used in the test suite) does not.

// top level section
/*
 * The original formulation of the concept only allowed sections at the top level
 * After a bit of discussion, it was decided that this was a bit limiting.
 */
section {
  # key to test operation and list parsing
  key = [ list, op(! ident ), ident2 ]
  # two keys, to test basic key-value pair testing and the important "all" key
  # though "all" is kinda forgotten at this point and might go away
  magic = xyzzy
  all = ident3
  // subsection to make sure that works
  blech {
    magic = abcd
  }
}

Note that, as the comment in the example says, at one time the design called for there to be an "all" keyword for convenience. Handling that properly and making sure the various potential corner cases didn't happen is making me re-think its utility.

config parser - notes on parser states.md

Opening Notes

• the arrow (->) means "transition to new state"

• the double-colons (::) means "take the following actions"

• DISCARD means to discard the current token

• CONSUME means to use the token in some manner

• RETURN means "clean up and return the section of the parse tree you've created"

• ENTRY means "do this on first entering the state"

• ANY is "any token at all"

• Tokens:

  • IDENTIFIER
  • OPEN_BRACE
  • CLOSE_BRACE
  • OPEN_LIST
  • CLOSE_LIST
  • OPEN_PARENS
  • CLOSE_PARENS
  • OPERATOR
  • STORE

States, Actions and Transitions

START:     ENTRY :: PEEK AT NEXT TWO TOKENS
           IDENTIFIER OPEN_BRACE  -> SECTION
           IDENTIFIER STORE -> KEYVALUE
			
SECTION:   ENTRY :: CONSUME, DISCARD
           IDENTIFIER OPEN_BRACE -> SECTION
           IDENTIFIER STORE -> KEYVALUE
           CLOSE_BRACE ANY :: DISCARD, RETURN
			
KEYVALUE:  ENTRY :: CONSUME, DISCARD
           IDENTIFIER OPEN_PARENS -> OPERATION
           OPEN_LIST IDENTIFIER -> LIST
           IDENTIFIER ANY :: CONSUME, RETURN
			
LIST:      ENTRY :: DISCARD
           IDENTIFIER SEPERATOR :: CONSUME, DISCARD
           IDENTIFIER OPEN_PARENS -> OPERATION
           SEPERATOR ANY :: DISCARD
           CLOSE_LIST ANY :: DISCARD, RETURN
			
OPERATION: ENTRY :: CONSUME, DISCARD
           OPERATOR IDENTIFIER :: CONSUME, CONSUME
           CLOSE_PARENS ANY :: DISCARD, RETURN

config parser - notes on tokenizing.md

• Whitespace of any type is just a separator. All characters from ASCII below 33 (so 0 to 32) act as little more than separators between tokens
• IDENTIFIER meets the requirements of this regular expression: /[a-zA-Z_]{1}[a-zA-Z0-9_]*/
• other tokens, despite the suggestive naming of most, could be anything that will not trigger the IDENTIFIER regular expression