Ideas about inheritance with declarative modeling for dynamodb

bloop-inheritance-concept.py

# Expanded from Account in http://bloop.readthedocs.org/en/latest/


class Account(engine.model):
    class Meta:
        abstract = True  # Don't create/validate as a literal table in Dynamo
        read_units = 100
        write_units = 10

    id = Column(UUID, hash_key=True)
    name = Column(String)
    email = Column(String)
    by_email = GlobalSecondaryIndex(
        hash_key='email', projection='keys_only',
        write_units=1, read_units=5)


class GoogleAccount(Account):
    class Meta:
        # Implicit abstract = False
        # tables are created unless explicitly declared as an abstract base
        read_units = 10
        table_name = "accounts-goog"


class GooglePremiumAccount(GoogleAccount):
    premium = Column(Boolean)
    
    # Meta.table_name is inherited from the parent
    # this has Meta.table_name = "accounts-goog"


class FacebookAccount(Account):
    class Meta:
        read_units = 200
    permissions = Column(Set(String))
    
    # Implict table name from class name
    # parent has no table_name

Open question: how does table name inference work with inheritance?

This is easy without inheritance - use explicit Meta.table_name if provided, fall back to class name.

Inheritance can be used for different tables with shared base attributes/layouts, or defining two views of the same table. There are a few ways to handle implicit table names with inheritance, but they are often at odds with 'obvious' for at least one of these cases.

Assume a base model Base and a deriving model Sub. Some options for picking the table name for Sub:

---

, here are two sample paths for picking a table name for Sub:

1. Use Sub.Meta.table_name
2. Fall back to Base.Meta.table_name
3. Fall back to Sub.__name__

This makes sense in the multi-view pattern - a base class defines common attributes that all models care about, and each submodel uses that same table to read/write. A little less obvious with abstract models, since an abstract model isn't explicitly tied to a table name.

---

1. Use Sub.Meta.table_name
2. Fall back to Sub.__name__

This requires explicitly duplicating the table_name from Base to Sub. There is some benefit to requiring users to explicitly declare their intention to access a table through two different patterns, but otherwise this breaks with the cloning behavior for other Meta attributes (Base.Meta.read_units is used for Sub.Meta.read_units if the latter is not defined).

---

Require explicit table names for all models, unless the model is abstract, or inherits from another model. The abstract property is already required for the Abstract Model pattern, so this is only a change in that we no longer fall back to the class name.

This is a breaking change against the current behavior; it will require more lines for simple tables, but will take care of the eventual confusion that comes from any implicit mapping system.

In this case, each class's Meta will either have abstract = True or table_name = "Some String" at which point, inheriting from the parent is clear and matches the behavior for other Meta attributes.

Inheriting Columns and Indexes

Originally I planned to do a union of the columns and indexes (fields from here on) taking the base's fields when provided. There are a few cases to consider:

1. Model name conflicts. This should be a simple resolution; following python, last definition wins.
2. Dynamo name conflicts. Two attributes point to the same dynamo column. These can be de-duped, but there are valid reasons to parse a field in multiple ways.
3. Subclassing model does not want access to an attribute. Possible fix: pass some_attribute = None to remove the column from the subclass. This lacks some clarity, since specifying None is a common pattern to (document the existence of and) provide defaults for instances of a class.

Inheriting Meta

All information (besides abstract and table_name discussed above) can safely be inherited from the immediate parent class, with the exception of bloop_init which will always take an explicit value or default to the immediate class's __init__ method.

Single table per engine

Supporting the streaming api has a host of complications when a single table backs more than one model in an engine. How are changes populated into the correct model?

1. The user specifies a handler per model, or passes a list of models with non-overlapping tables to watch for changes. The first is probably a lot of boilerplate, or error-prone. The latter doesn't solve the case where two models are backed by the same table for the same engine.

Alternatively, the engine can disallow mapping more than one model (within that engine) to a single table. That is, there will always be at most one model for a given table, through a given engine. This would require a way to watch multiple engines at once; perhaps the following signature:

# User function that is called when a change passes through the streaming api
# Signals a stop to processing once 100 changes are observed.
# Return Truthy values to stop processing, Falsey values to continue
changes = 0
def handle_update(engine, model, old, new):
    global changes
    changes += 1
    # Stop processing after 100 changes
    if changes > 99:
        return True
    else:
        return False

# Blocking call that monitors all tables registered for any number of engines
bloop.stream.monitor(handle_update, legacy_engine, engine_compat)