Command Parsing

cmd.org

Command Parsing

Overview

The usual purpose of parsing the command is to accept or present a command. The specification says, that:

define-command defines two functions. The first function has the same name as the command name, and implements the body of the command. It takes as arguments the arguments to the command as specified by the define-command form, as required and keyword arguments.

The name of the other function defined by define-command is unspecified. It implements the code used by the command processor for parsing and returning the command’s arguments.

Commands are parsed by functions in variables *command-parser*, *command-unparser* and *partial-command-parser. These functions are expected to know the (unspecified) interface of the second function defined by define-command.

Historically McCLIM for each defined command created three parsing functions used by the command line parsers. Now we adopted an approach similar to CLIM-TOS - the parser iterates over arguments and calls the argument and the delimiter callbacks. This solution is more flexible.

The name required arguments is a bit misleading because all arguments may have default values. In that case they don’t need to be supplied and in this sense they may be optional. Moreover keyword arguments are not optional unless they have a default value.

define-command must arrange for the function that implements the body of the command to get the proper values for unsupplied keyword arguments.

Names required arguments and keyword arguments are appropriate for the lambda list of the function implementing the command body but to avoid confusion (from the command parsing perspective) we’ll call them positional arguments and option arguments.

McCLIM parser implementation

The parser for command arguments is invoked with a function parse-command and accepts four arguments:

command name

a symbol

argument parser

a parser for the argument descriptions

delimiter parser

a parser for delimiters

stream

a stream to invoke operations on

The stream may be some other stream than the one passed to parse-command, for example accepting-values may supply an encapsulating stream. The argument parser should alwyas return the argument value. This is especially important when parsing option arguments where the keyword implies the ptype of the value.

The argument parser accepts the stream, the argument ptype and all description arguments except for the :gesture that is used for defining the presentation translators and is not expected to be evaluated. Moreover the argument name is supplemented as a value of the query-identifier keyword value.

The delimiter parser accepts the stream and a keyword telling what will be parsed next. Note that sometimes the delimiter parser is called twice in a row between arguments, that’s because there are two groups of delimiters:

argument delimiters

• :arg - positional argument
• :key - the option name
• :val - the option value

section delimiters

• :pos - positional arguments
• :opt - option arguments
• :end - no more parsing

Consider the following command:

(define-command com-foobar ((name 'string)
                            (age  'integer)
                            &key
                            (nsfw 'boolean :when (>= age 18)))
  (if (and (>= age 18) nsfw)
      (format t "Hello ~a! Let's hang out and have a beer!" name)
      (format t "Hello ~a! What a lovely day, isn't it?" name)))

First positional arguments are parsed:

positional arguments

del:pos del:arg arg:str del:arg arg:int

After parsing all positional arguments, depending on the value of age:

<= 17

del:end

>= 18

del:opt del:key arg:(member :nsfw) del:val arg:bool del:end

It is a responsibility of the arg parser to return an object of the correct presentation type (or escape the context), the consequences of returning something else are unspecified.

When the parser asks for a keyword then the presentation type is (completion ,valid-keys). Then depending on the key its value is queried. Here is a small parser example with sample output:

(flet ((arg-parser (stream ptype &rest args &key prompt &allow-other-keys)
         (declare (ignore stream))
         (format t ";;; Parsing ~a - ~s~%" ptype args)
         (cond ((presentation-subtypep ptype 'string) "jd"
                (format t "~a: " prompt)
                (read-line))
               ((presentation-subtypep ptype 'integer)
                (format t "~a: " prompt)
                (parse-integer (read-line) :junk-allowed t))
               ((presentation-subtypep ptype 'boolean)
                (yes-or-no-p "nsfw?"))
               ((presentation-subtypep '(completion nil) ptype)
                (with-presentation-type-parameters (completion ptype)
                  (format t "One of ~s please!: " sequence)
                  (read)))
               ((error "uh, erm. sorry!"))))
       (del-parser (stream type)
         (declare (ignore stream))
         (format t ";;; Delimiter ~s~%" type)))
  (climi::parse-command 'com-foobar #'arg-parser #'del-parser *query-io*))

;;; Delimiter :POS
;;; Delimiter :ARG
;;; Parsing STRING - (:QUERY-IDENTIFIER NAME :PROMPT "name")
name: JD
;;; Delimiter :ARG
;;; Parsing INTEGER - (:QUERY-IDENTIFIER AGE :PROMPT "age")
age: 32
;;; Delimiter :OPT
;;; Delimiter :KEY
;;; Parsing (COMPLETION (NSFW)) - (:PROMPT NIL)
One of (:NSFW) please!: :nsfw
;;; Delimiter :VAL
;;; Parsing BOOLEAN - (:QUERY-IDENTIFIER NSFW :WHEN T :PROMPT "nsfw")
nsfw? (yes or no) yes
;;; Delimiter :END

Command Unparser

command-unparser must be a function of three arguments, a command table, a stream, and a command to “unparse”. It prints a textual description of the command its supplied arguments onto the stream. The default value for command-unparser is the value of command-unparser.

The command unparser has three positional arguments: the command table, the stream and the command. It is expected to present a (possibly partial) command to the stream. It is invoked from the pmethod present specialized on the command.

The specification of the command unparser does not accept arguments acceptably and for-context-type - this is most likely the ommission. CLIM-TOS implements these as keyword arguments, McCLIM applies all remaining arguments to the function present.

This function iterates over positional arguments and when the argument has a value (either present in the command object or a default one) then that value is presented. Otherwise a string standing for the unsupplied value is printed.

After all positional arguments are presented and there are no unsupplied arguments, then options are presented. First ones occuring in the command object and then remaining keyword-value pairs. This is because option arguments may be conditionally unavailable depending on the positional arguments (see the argument description key :when).

Command parser

command-parser must be a function of two arguments, a command table and a stream. It reads a command from the user and returns a command object, or nil if an empty command line was read. The default value for command-parser is the value of command-parser.

It is not clear whether all command parsers should called from the pmethod accept, because when the stream is interactive then per specification the input editor will be opened (i.e when there is no interactor and the menu-command-parser is used). An alternative is to call the parser directly in the explicitly created input context. For example:

(with-input-context (`(command :command-table ,table))
    (object)
    (funcall *command-parser* table stream)
  (t object))

After accepting the command name this function iterates over positional arguments and call accept (or equivalent) and if suitable supplementing a default value. After all positional arguments are read, compute the applicable keys and read keyword-value pairs.

At any time the operator may type the activation gesture. When all arguments are accepted then the command should be returned. On the other hand what should happen if the activation gesture is inserted before the complete command is read? There are many possibilities:

1. ignore the gesture (alternatively treat it is a delimiter)
2. invoke the partial parser supplying the object read so far
3. return the partial command (and let the caller decide what to do)
4. try to complete the command with default values and do 2. or 3.

Command partial parser

partial-command-parser must be a function of four arguments, a command table, a stream, a partial command, and a start position. The partial command is a command object with the value of unsupplied-argument-marker in place of any argument that needs to be filled in. The function reads the remaining, unsupplied arguments in any way it sees fit (for example, via an accepting-values dialog), and returns a command object. The start position is the original input-editor scan position of the stream, when the stream is an interactive stream. The default value for partial-command-parser is the value of partial-command-parser.

Command line parsers considerations

• what the activation gesture does?
• should the partial parser really continue the cli?
• auto-insert the delimiter when the object is selected?
• ignore subsequent delimiters? (only when there is no default value?)
• how to make backspace work sanely?
• in accepting values modifying positional value may remove the option

Menu parsers considerations

• how to hint that the parser is reading something?
• activation and abort gestures doesn’t seem to work
• should we eagerly supplement defaults? rather not
• if not, should we supplement them on delimiter gesture?
• what the activation gesture does?