clara_changelog_prototype.clj

(ns clara.rules.changelog
  "Support for extracting and replaying a sequence of changes. The extract sequence represents the logical difference
   between the state of the given session and a previous point in time, but does not reflect every intermediate step
   along the way. For instance, if a given fact is inserted and retracted a million times prior to getting the change
   sequence, that sequence will contain no entries reflecting that fact. This is done to prevent unbounded growth of
   the log.

   Each item in the change sequence is one of three structures, with schemas defined below. They are:

   * Fact Change -- an arbitrary Clojure structure in the working memory -- and the number of instances in the memory.
   * Accumulator Change -- the result of an accumulator computation.
   * Activation -- a record of a pending activation in the rule engine.

   The structure of these items is designed such that previous entries can be discarded to preserve space. The fact-change
   entry contains an absolute count of facts, so any previous entries with the same fact instance in the
   log can be discarded. Similarly, the accumulator change specifies the full state for the node id and accumulator
   bindings.

   This approach allows for self-compacting log implementations. This is drawn from the compacting log model
   offered by Apache Kafka at https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction, but is
   decoupled from the underlying storage. Therefore an arbitrary storage system to be used."

  (:require [clara.rules.memory :as mem]
            [schema.core :as s])
  (:import [clara.rules.engine Token]))


;; Schema defining the number of the given type of fact in the working memory.
(def fact-change-schema {:type :fact
                         :fact s/Any
                         :count s/Int})

;; Schema defining an accumulator result in the changelog.
(def accum-change-schema {:type :accum
                          :node-id s/Int
                          :bindings {s/Keyword s/Any}})

;; Schema defining a pending activation in the changelog.
(def activation-schema {:type :activation
                        :node-id s/Int
                        :tokens [Token]})

;; Prismatic schema defining the changelog entry.
(def change-schema (s/conditional
                    #(= :fact (:type %)) fact-change-schema

                    #(= :accum (:type %)) accum-change-schema

                    #(= :activation (:type %)) activation-schema))

(defn get-changes
  "Returns a sequence of change records for the given session. The sequence contains all changes
   since the last time clear-changes was used on the given session, or all changes for the history
   of the session if clear-changes has not be called."
  [session]
  ;; Implement
  )

(defn clear-changes
  "Returns a session equivalent to the given one, but with the change log cleared. This is generally
   used by a consumer that first calls and persists get-changes to a durable store, and therefore is
   able to safely discard the earlier log.

   After calling clear-changes on the session, users can simply discard the previous session and
   the unneeded changes will be garbage collected."
  [session])

(defn mk-session-from-changes
  "Creates a new session from a sequence of changes. The rule sources and options are the same as those
   given to clara.rules/mk-session.

   The given sequence of should represent all changes since the creation of the session, although redundant
   changes can be discarded as described in the namespace-level documentation."
  [change-seq & sources-and-options])

This seems fairly straightforward to support; any serializable store (we have Cassandra available) would work if you keep an index of checkpoints and have a lifecycle policy to age out entries after a point in time.