Example metaclass implementation for Python 3 for the construct3 library

metastruct.py

class CustomCollection():
	def __init__(self):
		super().__init__()
		self.items = []
		
	def __setitem__(self, key, value):
		if key == '_':
			self.items.append(value)
		else:
			self.items.append((key, value))
	
	def __getitem__(self, key):
		if key == '_':
			raise KeyError('_')
		return dict(self.items)[key]
		

class _MetaStruct(type):
	""" metaclass for use by magic struct"""

	@classmethod
	def __prepare__(metacls, name, bases):
		return CustomCollection()

	def __new__(cls, name, bases, classdict):
		if bases:
			return Struct(*classdict.items())
		else:
			return super().__new__(name, bases, classdict.items())

class MagicStruct(metaclass=_MetaStruct):
	""" syntactic sugar for making Structs"""
	pass

class ExampleStruct(MagicStruct)
	named = uint64l
	_ = Embedded(Flags(uint64l,
		flag1=0x01,
		flag2=0x20,
	))

• how do you suppose to embed two structs? _ is already taken and OrderedDict won't help here. Also, there's Padding which doesn't get a name in the containing struct.
• suppose i wish to nest structs, how would that be done?

class FooStruct(Struct):
    a = int8
    b = int16
    class Nested(Struct):
        c = int32
        d = int64

"Nested" becomes a field of the struct, since it's inserted into the class' dict... but it completely breaks the semantics of what a class is. that's bad.

• now what about the other constructs? e.g., Sequence is like an "unnamed struct", i.e., a tuple. that would clearly not work
• and how would "more complex" constructs, such as Switch, work?

class EthernetHeader(Struct):
    src = MacAddr()
    dst = MacAddr()
    type = ubint16
    next = Switch(this.type,
        (0x1234, IPHeader),
        ...
    )

so we're back to square one -- you just "peeled" the first level of parentheses.

moreover, today i can define "small structs" wherever i want, e.g., inside the Switch body. it's not a good practice for large structs, but using the metaclass approach i would have to create a class somewhere, give it a global name, and refer to it -- instead of just defining it where it's needed.

anyway, these are some of the issues from the top of my head. i'm sure you'd run into more if you really go down that road. of course you could work around any of these, but it's all too artificial, confusing to newcomers and requires tons of magic. construct 3 aims to be a "cleaner version" of construct 2 (which does suffer from some amount of magic), so i'd rather not go this way.

that being said, i encourage you to continue with your work and if we see it becomes consistent and compositional, i'd be happy to include it in construct (perhaps as an alternative form). please keep me posted. thanks.

how do you suppose to embed two structs? _ is already taken and OrderedDict won't help here. Also, there's Padding which doesn't get a name in the containing struct.

look again, I'm not using an OrderedDict, I'm using a CustomCollection, you can reuse names as much as you want, if the name is anything but _ it's passed to Struct.__init__ as a k,v tuple, and if it's _ it's just passed as v.

__prepare__ is pretty magical, it's lets you override the instance of dict used by class as the locals() while evaluating class body.

"Nested" becomes a field of the struct, since it's inserted into the class' dict... but it completely breaks the semantics of what a class is. that's bad.

This doesn't bother me since other schema-definition with metaclasses work this way. (see Flatland, SQLAlchemy, etc.), You're already breaking semantics by just using metaclasses to have class make an instance instead of a sub-class (though the Zen of metaclasses is that a sub-class is a new instance of type)

now what about the other constructs? e.g., Sequence is like an "unnamed struct", i.e., a tuple. that would clearly not work

yeah probably not you could make a metaclass interface, but it would be silly.

and how would "more complex" constructs, such as Switch, work?

class EthernetHeader(Struct):
     src = MacAddr()
     dst = MacAddr()
     type = ubint16
     class next(Switch(this.type)):
          @Switch.condition(0x1234)
          class IPHeader(Struct):
              #inline defintion of IPHeader, name would be ignored
              pass
         # - or - #
          _ = Switch.condition(0x1234)(IPHeader) #external types, would also be needed for simple types like 

ok that's pretty ugly. I suppose the solution to metaprogramming is MORE METAPROGRAMMING:

since __prepare__ lets us change how locals() every variable access or retrieval inside the class block can be overridden, I think you could make this work (with enough patience)

class EthernetHeader(Struct):
     src = MacAddr()
     dst = MacAddr()
     type = ubint16
     if this.type == 0x1234:
         next = IPHeader
     elif this.type == 0x2345:
         next = SomethingElse

(the access of this and the assignments to next can both be overriden)

But I fear that way lies madness.

(nm, there's no way to execute all if blocks, you'd have to use with or macros, but I think my point about madness still stands.)

I'm gonna play with the code a little bit and update it, see If I can get something reasonable for Switch (w/o macros)

How about:

next = Switch(this.type)
next[0x1234] = IPHeader

@next.condition(0x2345)
class next(MagicStruct):
pass

next[Switch.default] = uint64l

that could be made to work (w/o macros), but now there's machinery to do Switch inside the Struct metaclass, which I suppose isn't composable.