Modifying Datomic schema for composite tuples

modify-composite-tuples.md

Say you define a composite tuple like so:

#:db{:ident :foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar :baz],
     :cardinality :db.cardinality/one}

Now suppose you want to change [:bar :baz] to [:bar :quux]. The Datomic docs say "You can never alter :db/tupleAttrs", so simply transacting the following won't work:

#:db{:ident :foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar :quux],
     :cardinality :db.cardinality/one}

You'll have to create a whole new composite attribute instead. You might have the idea to simply do this:

#:db{:ident :new-foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar :quux],
     :cardinality :db.cardinality/one}

Don't do that. The :foo composite tuple is still in Datomic's schema, and it includes the :bar attribute. So if you transact an entity like this:

{:bar "testing"
 :quux 123}

It will be turned into this:

{:bar "testing"
 :quux 123
 :foo ["testing" nil]
 :new-foo ["testing" 123]}

This would be especially bad if you're using :db.unique/identity for the composite tuple, like I am in the example. Instead, you should create a new attribute to use instead of bar:

#:db{:ident :new-foo,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:new-bar :quux],
     :cardinality :db.cardinality/one}

Conclusion: Any time you need to modify a composite tuple, you should create a new attribute for the tuple and a completely new set of attributes for :db/tupleAttrs.

Since this is likely to result in a lot of attribute creation/renaming, I've adopted the convention of appending -<number> to the end of attributes that are just renamings of old attributes, like so:

#:db{:ident :foo-0,
     :valueType :db.type/tuple,
     :unique :db.unique/identity,
     :tupleAttrs [:bar-0 :quux],
     :cardinality :db.cardinality/one}

Note: I've been doing this in development where:

1. I transact the schema to a dev db, but
2. I haven't yet transacted any other data that actually uses the attributes (I've only been testing with d/with).

If you've already transacted data that uses the attributes, that would complicate things of course. If you need to make a change to a composite tuple that's already being used in prod, you'll have to make sure you do things in a backwards-compatible way. I haven't thought much about this yet.

Alternate solutions, which I haven't tried, include:

• Set up your dev environment so you don't transact schema or data at all; you just use d/with for everything (might be complicated and/or annoying).
• Write some code to easily migrate data from one db to another, then every time you want to make a disallowed schema change, move all your data over to a new dev db. I think this would work best in combination with the solution I've presented above. During development, you rename attributes using the -<number> convention, but before you deploy to prod, you get rid of all the -<numbers>s and migrate to a new dev db.