Ideas about inheritance with declarative modeling for dynamodb

bloop-inheritance-concept.py

# Expanded from Account in http://bloop.readthedocs.org/en/latest/


class Account(engine.model):
    class Meta:
        abstract = True  # Don't create/validate as a literal table in Dynamo
        read_units = 100
        write_units = 10

    id = Column(UUID, hash_key=True)
    name = Column(String)
    email = Column(String)
    by_email = GlobalSecondaryIndex(
        hash_key='email', projection='keys_only',
        write_units=1, read_units=5)


class GoogleAccount(Account):
    class Meta:
        # Implicit abstract = False
        # tables are created unless explicitly declared as an abstract base
        read_units = 10
        table_name = "accounts-goog"


class GooglePremiumAccount(GoogleAccount):
    premium = Column(Boolean)
    
    # Meta.table_name is inherited from the parent
    # this has Meta.table_name = "accounts-goog"


class FacebookAccount(Account):
    class Meta:
        read_units = 200
    permissions = Column(Set(String))
    
    # Implict table name from class name
    # parent has no table_name

Inheriting Columns and Indexes

Originally I planned to do a union of the columns and indexes (fields from here on) taking the base's fields when provided. There are a few cases to consider:

1. Model name conflicts. This should be a simple resolution; following python, last definition wins.
2. Dynamo name conflicts. Two attributes point to the same dynamo column. These can be de-duped, but there are valid reasons to parse a field in multiple ways.
3. Subclassing model does not want access to an attribute. Possible fix: pass some_attribute = None to remove the column from the subclass. This lacks some clarity, since specifying None is a common pattern to (document the existence of and) provide defaults for instances of a class.

Inheriting Meta

All information (besides abstract and table_name discussed above) can safely be inherited from the immediate parent class, with the exception of bloop_init which will always take an explicit value or default to the immediate class's __init__ method.

Single table per engine

Supporting the streaming api has a host of complications when a single table backs more than one model in an engine. How are changes populated into the correct model?

1. The user specifies a handler per model, or passes a list of models with non-overlapping tables to watch for changes. The first is probably a lot of boilerplate, or error-prone. The latter doesn't solve the case where two models are backed by the same table for the same engine.

Alternatively, the engine can disallow mapping more than one model (within that engine) to a single table. That is, there will always be at most one model for a given table, through a given engine. This would require a way to watch multiple engines at once; perhaps the following signature:

# User function that is called when a change passes through the streaming api
# Signals a stop to processing once 100 changes are observed.
# Return Truthy values to stop processing, Falsey values to continue
changes = 0
def handle_update(engine, model, old, new):
    global changes
    changes += 1
    # Stop processing after 100 changes
    if changes > 99:
        return True
    else:
        return False

# Blocking call that monitors all tables registered for any number of engines
bloop.stream.monitor(handle_update, legacy_engine, engine_compat)