Doing worker processing with EmacsLisp

worker-elisp.el

(defmacro worker-elisp (output-stream &rest lisp)
  "Evaluate the LISP in a child Emacs sending output to OUTPUT-STREAM.

The child Emacs has a 'load-path' exactly as the 'load-path' of
the parent Emacs at execution.

The OUTPUT-STREAM could be a buffer, a function or a process.

If the OUTPUT-STREAM is a process it may have a process property
':send-string-function' evaluating to a function to send data to
the process.  The function should take the same arguments as
the standard Emacs Lisp 'process-send-string'.

Furthermore, if the OUTPUT-STREAM is a process, when the child
Emacs finishes an EOF is sent to the process.  If the
OUTPUT-STREAM has a process property ':send-eof-function' then
that is used to send the EOF.  The function should take the same
arguments as the standard Emacs Lisp 'process-send-eof'.

An example:

 (worker-elisp http-connection
   (require 'creole)
   (creole-wiki some-file-name))

Presuming http-connection is a process (in the manner of Elnode,
for example) this will cause a child Emacs to be created, within
which 'some-file-name' will be loaded and converted from
WikiCreole to HTML and then sent to the standard output
stream. The child's standard output stream is connected directly
to the 'http-connection'.  In this case, presumably the
'http-connection' would have functions attached to the properties
'':send-string-function' and ':send-eof-function' to do HTTP
chunk encoding and to end the HTTP connection correctly."
  (declare (indent defun))
  (let ((loadpathvar (make-symbol "load-path-form"))
        (childlispvar (make-symbol "child-lisp"))
        (lispvar (make-symbol "lisp"))
        (filtervar (make-symbol "filter-function"))
        (cmdvar (make-symbol "command"))
        (procvar (make-symbol "process"))
        (namevar (make-symbol "process-name"))
        (bufvar (make-symbol "buffer"))
        (outvar (make-symbol "output-stream")))
    `(let* ((,outvar ,output-stream)
            (,lispvar ',@lisp)
            (,loadpathvar
             (format "(setq load-path %s)"
                     (pp-to-string load-path)))
            (,childlispvar  ; the lisp to run
             (format "\"(progn %s %s)\""
                     load-path-form
                     (apply 'pp-to-string lisp)))
            (,cmdvar "emacs -Q -batch --eval '(eval (read t))'")
            (,namevar (concat
                       (number-to-string (random))
                       (number-to-string (float-time))))
            ;; We have to make a buffer unless the output-stream is a buffer
            (,bufvar (cond
                      ((bufferp ,outvar) ,outvar)
                      (t
                       (get-buffer-create (format "* %s *" ,namevar)))))
            (,procvar (start-process-shell-command ,namevar ,bufvar ,cmdvar)))
       ;; If the output stream is not a buffer we need a filter function
       (cond
        ;; Plain old filter function supplied directly
        ((functionp ,outvar)
         (set-process-filter ,procvar ,outvar))
        ;; A process - setup a filter function
        ((processp ,outvar)
         (set-process-filter
          ,procvar
          (lambda (process data)
            (if (not (equal "closed" (process-status ,procvar)))
                (funcall
                 ;; Does the output-stream have a specific function?
                 (or (process-get ,outvar :send-string-function)
                     'process-send-string)
                 ;; The data to sent to the output-stream process
                 ,outvar data))))))
       ;; Now setup the sentinel
       (set-process-sentinel
        ,procvar
        (lambda (process signal)
          (let ((send-eof-function
                 ;; Does the output-stream have a send-eof?
                 (or (and (process-p ,outvar)
                          (process-get ,outvar :send-eof-function))
                     'process-send-eof)))
            (cond
             ((equal status "finished\n")
              (when send-eof-function
                  (funcall send-eof-function ,outvar)))
             ((string-match "exited abnormally with code \\([0-9]+\\)\n" status)
              (when send-eof-function
                (funcall send-eof-function ,procvar))
              (delete-process process)
              (kill-buffer (process-buffer process)))
             ;; Any other signal status is ignored
             (t)))))
       ;; And finally send the child Emacs the Lisp to evaluate
       (process-send-string ,procvar ,childlispvar))))