Datomic 0.9.5927 observations and questions

notes.md

Query basics

{:db/ident       :meta/tag
 :db/valueType   :db.type/tuple
 :db/tupleAttrs  [:meta/tag-namespace :meta/tag-key :meta/tag-value] ;; all unique strings
 :db/cardinality :db.cardinality/one
 :db/unique      :db.unique/identity}

Then

{:meta/tag-namespace "Test"
 :meta/tag-key       "Test"
 :meta/tag-value     "Test"}

Lookup refs work as I would expect!

(d/entity db [:meta/tag ["Test" "Test" "Test"]]) 
; #:db{:id 17592186574549}

I can query with hardcoded values:

(d/q '[:find ?t :in $
       :where [?t :meta/tag ["Test" "Test" "Test"]]]
     db) 
; #{[17592186574549]}

And I can construct tuples outside and pass them in:

(d/q '[:find ?t :in $ ?tt
       :where [?t :meta/tag ?tt]]
     db ["Test" "Test" "Test"]) 
; #{[17592186574549]}

Construct tuples inside Datalog - this doesn't work:

(d/q '[:find ?t :in $ ?tn ?tk  ?tv
       :where [?t :meta/tag [?tn ?tk ?tv]]]
     db "Test" "Test" "Test") 
; #{}

You have to use the tuple helper:

(d/q '[:find ?t :in $ ?tn ?tk  ?tv :where
     [(tuple ?tn ?tk ?tv) ?tt]
     [?t :meta/tag ?tt]]
   db "Test" "Test" "Test")
; #{[17592186574549]}

Raw index access:

(seq (d/datoms db :avet :meta/tag ["Test" "Test" "Test"]))

(#datom[17592186574549 1116 ["Test" "Test" "Test"] 13194140063444 true])

Partial tuples (padded with nil) works with seek-datoms:

(seq (d/seek-datoms db :avet :meta/tag ["Test" nil nil]))

(#datom[17592186574549 1116 ["Test" "Test" "Test"] 13194140063444 true])

---

Adding tuple attrs to existing entities

To backfill composite tuple attributes, you have to re-transact the source values.

Composite attributes are entirely managed by Datomic–you never assert or retract them yourself. Whenever you assert or retract any attribute that is part of a composite, Datomic will automatically populate the composite value.

The transactor won't transact anything that's already currently true, but it will trigger the behaviour that creates the composite tuple.

Update: Sample Datomic Client code that does this, from Cognitect:

https://github.com/cognitect-labs/day-of-datomic-cloud/blob/master/src/datomic/dodc/composites.clj

---

Length of tuples

Homogenous tuples, although they are variable length, you can only have at most 8 values. This means that although it does provide a way to set the order of a bunch of values, it can't be used to set an arbitrary order of child entities e.g. sections in a page if there are more than 8 children.

Still, it is useful if you know you'll never have more than 8 arbitrarily ordered children.

---

Using refs in tuples

The docs don't list :db.type/ref as a valid value, but the transactor does accept :db/tupleType :db.type/ref, and it accepts entity ids as tuple values, whether as raw Longs or lookup-refs.

d/entity won't process entitys referenced in tuples, and you can't use d/pull to walk into entities referenced this way.

When an entity ID appears in a tuple, it doesn't also create a :vaet index datom.

My guess is refs are treated as longs once they're validated as entity ids, and no further ref-like behaviour occurs.

About "using refs in tuples", do you think it is some kind of limitation/bug? I have a use case where a composite natural key has one :db.type/ref element because I dediced to model an enumeration as database idents as suggested by the documentation.

{:db/ident :template/provider+name
 :db/cardinality :db.cardinality/one
 :db/unique :db.unique/value
 :db/valueType :db.type/tuple
 :db/tupleAttrs [:template/provider :template/name]}

where :template/provider is a ref and :template/name is a string. Queries that restrict the templates on the individual attributes work (i.e. where conditions on :template/provider and :template/name) but not if I try to use the composite attribute (e.g. [?e :template/provider+name [:provider/some-provider "some-name"]]). Also I can't refer to the template using the composite key when transacting an entity that refers to a template:

(d/transact conn {:tx-data [#:infocard{...
                                       :template [:template/provider+name [:provider/some-provider "some-name"]]
                                       ...}]})

I guess we'd have to hear from @stuarthalloway on this one :)

+1 would like to hear more on this.

I really appreciate the effort here to document the behavior of composite tuples in Datomic. There is so much unfulfilled promise in composite tuples, especially with refs. (@mikew1 , sent you a personal reply to this)

@mikew1 and others: I cobbled together a tx-function that adds support for using composite tuples with refs in transactions:

(defn- resolve-composite-refs
  [db v-or-nym]
  (if-let [[identifier value] (when (= :lookup-ref (nym-type v-or-nym)) v-or-nym)]
    [identifier (if-let [t-attrs (-> (d/pull db '[:db/tupleAttrs] identifier) :db/tupleAttrs)]
                  (mapv (fn [t-attr v] (if (= :db.type/ref (vtype db t-attr))
                                         (let [rid (-> (d/pull db '[:db/id] v) :db/id)]
                                           (assert rid (str "Lookup ref cannot be resolved" v-or-nym))
                                           rid)
                                         v))
                        t-attrs value)
                  value)]
    v-or-nym))

;; This should be possible without a database function, but Datomic composite tuples
;; are bafflingly limited with ref types and unique identity/value.
;; Reference: https://gist.github.com/robert-stuttaford/e329470c1a77712d7c4ab3580fe9aaa3
;; Reference: https://forum.datomic.com/t/troubles-with-upsert-on-composite-tuples/1355
(defn t-ref
  "Return equivalent transaction data to the given `[op e a v]` where any constituent lookup refs of
  composite tuple lookup-refs are resolved into eids.  This works around a known limitation of
  composite tuples combined with unique values references"
  [db [op e a v]]
  (let [e (resolve-composite-refs db e)
        v (if (= :db.type/ref (vtype db a))
            (resolve-composite-refs db v)
            v)]
    [[op e a v]]))

Here's an example of usage:

`(~'st.datomic/t-ref
                     [:db/add "address" :st.location/county
                      [:st.county/state+county-numeric [:ansi.fips/state-numeric ~state] ~county]])