An introduction to DataMapper Property API v. 2.0 (draft)

dm_property_2.0.rb

##
#
# The most important aspect of the new Property API is that
# property inheritance should not be used to instruct
# adapters how to persist data. It means, for example, that 
# you should not use Integer as the base class for your custom
# property just because it is persisted as an integer. Right now
# Enum inherits from Integer because it is stored as an integer.
# It is a mistake, because a value of Enum property can be either
# a string or a symbol, not integer. The fact that it is persisted
# as an integer should not be reflected in the inheritance.
#
# So, why? It's because adapters should know how to persist
# data, it's outside of the dm-core's scope. All the logic required
# to make the decision how to persist data should be implemented
# on the adapter level. That's the new deal. On the property level
# we only care about resource attributes and how they work and behave.
#
# Definitions:
#
#   typecast - A property typecasts a value that comes from 
#              the user input. *it is not called* when retrieving
#              data from the datastore. For example Time property
#              knows how to typecast strings, hashes and other
#              representations to a time object. String always does
#              #to_s. Integer does #to_i etc.
#
#   load - A value is loaded once retrieved from the datastore (means:
#          returned by the adapter). For all the built-in core properties
#          it is required for the adapter to return their values as an 
#          instance of the expected class. For example String doesn't do
#          #to_s on a value returned by the adapter because we expect
#          that value to already be a string. Same goes with Integer, Float,
#          Time, DateTime and all the properties that comes with dm-core.
#          Note that it is possible that multiple classes can be used for 
#          loaded values. Like JSON, where we can have an array or a hash.
#          It is important for validations to handle that properly (that's why
#          in 1.0.x auto validation of a type is skipped in case of custom properties,
#          which no longer will be required in 2.0)
#
#   dump - Dump is *always* called before passing values down to the adapter
#          to execute a query or any other statement. It should be used
#          only if we want to persist a value in a different form then when
#          it's loaded. A good example is a Time object that we want to 
#          persist as a string because our legacy database has such schema.
#          A bad example is that we want to do the same, because our backend
#          datastore doesn't support storing time objects. If that's the case
#          then the adapter should handle that internally.
#
#   load_as (new) - It's a new way of configuring a property. It sets class or
#                   many classes that are used to represent a loaded value. So,
#                   with String it's ::String, with Integer it's ::Integer, but with
#                   JSON it can be ::Array or ::Hash.
#                   Notice that this option is used by auto-validation to infer the
#                   correct type validator. Also, this is primitive method/setting
#                   successor.
#
#   dump_as (new) - Another new way of configuring a property. Same as load_as but
#                   in the context of dumping values. In case of RDBMS adapters
#                   *this setting* is used to figure out the schema because *dumped*
#                   values are being returned down to the adapter
#
#   marshall (new) - It comes with the base Object property as a helper for 
#                    the adapters which can use it if they don't know how to
#                    persist a given property value. It is never called inside
#                    dm-core

#   unmarshall (new) - Also comes with the base Object property and also can
#                      be used by the adapter to unmarshall data before returning
#                      them back to dm-core
#

#####
#
# Object - the base class which provides marshal / unmarshall 
# which should be used by an adapter if it doesn't know how
# to persist a property.
#
# An example adapter code might do something like that with a 
# given dumped property value:
#
#   value_to_persist = if property_supported?(property)
#     value
#   else
#     property.marshal(value)
#   end
#
# That means that we could have SQlite adapter that has to
# marshal an array property value and MongoDB adapter 
# that can persist that value without the need to marshal it.
#
module DataMapper
  class Property
    class Object < Property
      load_as ::Object
      dump_as ::String

      def marshall(value)
        [ Marshal.dump(value) ].pack('m') unless value.nil?
      end

      def unmarshall(value)
        case value
          when ::String
            Marshal.load(value.unpack('m').first)
          when ::Object
            value
          end
      end
    end
  end
end

#########################
#
# Some Property examples:
#

######
#
# Time stored as a string
#
module DataMapper
  class Property
    class TimeAsString < Time
      dump_as ::String

      def dump(value)
        value.to_s
      end
    end
  end
end

######
#
# JSON stored as a string and loaded either as a hash or an array
#
module DataMapper
  class Property
    class JSON < Object
      load_as ::Hash, ::Array
      dump_as ::String

      def load(value)
        JSON.load(value)
      end

      def dump(value)
        JSON.dump(value)
      end
    end
  end
end

I think your JSON example has dump and load the wrong way around :)

But, this is interesting, and a good step forward I think.

Fixed, thanks!

This looks a lot nicer!

Just looking over this again, and wondering how dm-migrations and dm-validations will interact with it. Say, for example, your TimeAsString above. Does dm-migrations see the dump_as ::String and deduce from that, that the field in the database should be a VARCHAR? And likewise, auto-validation in dm-validations needs to be aware of the fact that subclassing takes place, so TimeAsString doesn't fail validation because it contains a Time instead of a String. These are the types of issues we're facing with the current Property API. Well... in the current property API you'd probably end up subclassing String to implement TimeAsString, since dm-migrations would create the wrong type in the database otherwise, but as a side-effect of subclassing String, auto-validation would fail because it isn't actually a String. So you end up overriding #custom? to get it working. It all gets quite fiddly :)

@d11wtq yes, that's the significant change - migrations care only about dump_as setting as it tells in what form a value is pushed down to the adapter. In case of TimeAsString a migration will create a varchar field. On the other hand validations only care about load_as since that tells us in what form a value will be presented in the "resource land". So, if you have a JSON with load_as ::Array, ::Hash then an auto-validation of type will check if a value is either Array or Hash.

That's exactly why I want to make this change so you guys no longer have to deal with overriding valid? or primitive? or setting custom? to true as those are only workarounds of the problems we're facing with the current implementation.

@solnic, excellent! I think DM properties (or Virtus attributes, as the case may become) are pretty awesome and being able to easily customize them is a big selling point.

@d11wtq actually it's quite possible we're gonna integrate Virtus with DM1 :)