Created
May 22, 2014 03:04
-
-
Save rbrush/1dd983a4069759729e34 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(ns clara.rules.changelog | |
"Support for extracting and replaying a sequence of changes. The extract sequence represents the logical difference | |
between the state of the given session and a previous point in time, but does not reflect every intermediate step | |
along the way. For instance, if a given fact is inserted and retracted a million times prior to getting the change | |
sequence, that sequence will contain no entries reflecting that fact. This is done to prevent unbounded growth of | |
the log. | |
Each item in the change sequence is one of three structures, with schemas defined below. They are: | |
* Fact Change -- an arbitrary Clojure structure in the working memory -- and the number of instances in the memory. | |
* Accumulator Change -- the result of an accumulator computation. | |
* Activation -- a record of a pending activation in the rule engine. | |
The structure of these items is designed such that previous entries can be discarded to preserve space. The fact-change | |
entry contains an absolute count of facts, so any previous entries with the same fact instance in the | |
log can be discarded. Similarly, the accumulator change specifies the full state for the node id and accumulator | |
bindings. | |
This approach allows for self-compacting log implementations. This is drawn from the compacting log model | |
offered by Apache Kafka at https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction, but is | |
decoupled from the underlying storage. Therefore an arbitrary storage system to be used." | |
(:require [clara.rules.memory :as mem] | |
[schema.core :as s]) | |
(:import [clara.rules.engine Token])) | |
;; Schema defining the number of the given type of fact in the working memory. | |
(def fact-change-schema {:type :fact | |
:fact s/Any | |
:count s/Int}) | |
;; Schema defining an accumulator result in the changelog. | |
(def accum-change-schema {:type :accum | |
:node-id s/Int | |
:bindings {s/Keyword s/Any}}) | |
;; Schema defining a pending activation in the changelog. | |
(def activation-schema {:type :activation | |
:node-id s/Int | |
:tokens [Token]}) | |
;; Prismatic schema defining the changelog entry. | |
(def change-schema (s/conditional | |
#(= :fact (:type %)) fact-change-schema | |
#(= :accum (:type %)) accum-change-schema | |
#(= :activation (:type %)) activation-schema)) | |
(defn get-changes | |
"Returns a sequence of change records for the given session. The sequence contains all changes | |
since the last time clear-changes was used on the given session, or all changes for the history | |
of the session if clear-changes has not be called." | |
[session] | |
;; Implement | |
) | |
(defn clear-changes | |
"Returns a session equivalent to the given one, but with the change log cleared. This is generally | |
used by a consumer that first calls and persists get-changes to a durable store, and therefore is | |
able to safely discard the earlier log. | |
After calling clear-changes on the session, users can simply discard the previous session and | |
the unneeded changes will be garbage collected." | |
[session]) | |
(defn mk-session-from-changes | |
"Creates a new session from a sequence of changes. The rule sources and options are the same as those | |
given to clara.rules/mk-session. | |
The given sequence of should represent all changes since the creation of the session, although redundant | |
changes can be discarded as described in the namespace-level documentation." | |
[change-seq & sources-and-options]) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This seems fairly straightforward to support; any serializable store (we have Cassandra available) would work if you keep an index of checkpoints and have a lifecycle policy to age out entries after a point in time.