The conceptual overview of content in Magenta along with it's manifestation as a directory tree. The "map" provides a visual overview of the relationships between the 6 concepts, artist, asset, subset, family, version and instance.
Previous
To enable a publish of any kind from any artist performing any task during shots, the idea of a "task" was delegated to the /work directory. Now the same is true for assets too. Less to worry about.
| Node | Description |
|---|---|
 |
A transformation; i.e. input A turns into output B |
 |
Transformation from -> to |
 |
Instance, the inverse of a file. May have one or more representations |
 |
A representation, such as a .jpeg file or .exr sequence |
 |
Updated individually |
 |
A comment |
 |
Artist |
 |
Item |
 |
Subset |
 |
Family |
 |
Package |
 |
Versioned package |
The package is a group of subset versions.
/film directoryFor example the animator imports a layoutPkg to start animating. He produces a animPkg to be used by lighters who produce a lightPkg to be used by compositors who produce compPkg used by grading and editing.
The package compensates for the lack of association between subsets, such as ben01 and table01, where each subset is versioned independently. A package wraps one or more of these versions.
ben01 animPkg
v001 /-------> v001
v002 |
v003 -------------------|
table01 |
v001 -------------------/
v002One of potentially many ways of persisting the same set of data. At 10,000 feet, a project can be represented as either a Blu-Ray, or terabytes of models, animation, audio, renders and settings.
Representations are relevant mainly when using the same set of data in different software.
For example, that high resolution hero model works great for renders, but less so for an asset browser. There, a smaller more accessible file format is preferable, such as a PNG. Perhaps a proxy version, such that it can still be rotated, but stored as an OBJ rather than ABC for simplified parsing in the Python based application.
The idea is that, whenever you save data, you save it in every way you can think to use it. Every model is not only saved as an MB, even though it will primarily be used in Maya. It is also saves as an OBJ, because you never know if you someday will want to open it in Mudbox. It's saved as a PNG, because you might not have access to either Maya nor Mudbox when time comes to simply look at the model.
Think of it like portrait. The subject - such as a person - remains the same, but for every portrait strokes and colours differ slightly. Some are caricatures, some are line drawings. Some might be sculptures, or even 3d scans. Vastly different results, each representing the same subject. The same data.
In this case, the subject could be a model, whereas a representation could be an exact representation stored as an .obj, or an approximate one, such as a .png or poly-reduced alembic. Same data, different representations.
A "subset" is part of an Asset.
A subset consists of one or more Pyblish "instances".
Sometimes, part of an Asset needs to update independently of one another.
For example, in a shot with two characters only one has changed and needs publishing. Another example, a shot where only one render layer has changed and needs publishing.
Instead of publishing the entire Item, needlessly consuming time, space and cpu cycles, a subset may be published and versioned individually.
The name "subset" reflect that it is not the whole Item, but only part of it.
| """Source code to represent a directory tree as JSON""" | |
| import os | |
| import json | |
| def hierarchy_to_dict(path): | |
| if os.path.isfile(path): | |
| return "" | |
| d = {n: hierarchy_to_dict(os.path.join(path, n)) for n in os.listdir(path)} | |
| return d | |
| d = hierarchy_to_dict(r'C:\Users\marcus\Dropbox\projects\MagentaHierarchy0.7') | |
| print(json.dumps(d, indent=2, sort_keys=True)) |
By the way, still very pleased with the latest ideas that are on the drawing board. It seems it's getting solid.
That is awesome! I'm really excited about realising this map, I think it could be the foundation for quite a few pipelines. Even the likes of Framestore but also smaller ones.
It seems there could be multiple instances for a representation. How come?
It's the opposite, a single instance - such as a Rig - consists of many representations - such as an .mb and an .fbx.
How to define a representation? Think of it like portrait. The subject - such as a person - remains the same, but for every portrait strokes and colours differ slightly. Some are caricatures, some are line drawings. Some might be sculptures, or even 3d scans. Vastly different results, each representing the same subject. The same data.
In this case, the subject could be a model, whereas a representation could be an exact representation stored as an .obj, or an approximate one, such as a .png or poly-reduced alembic. Same data, different representations.
Ok, that clarifies things. How would you extract different "representations" in Pyblish?
I can imagine they might be the same, but also different families (other formats might require a different set of validations). This is actually supported by Pyblish now, instances in multiple families... as such the families might export their own representations.
Does this mean that the terminology for "family" is currently becoming a conflict here? Or would representations always be of a single family? If so how would you setup the different "representations" output for it?
Two extractors of the same instance are by definition producing two representations. The instance is the data that remains the same, whereas the outputted files/data/whatever are the representations.
Families and representations don't necessarily have anything in common. A family of rig can still be represented by both an .mb, as an .obj, without the controllers and deformers, and a .png.
Representations are more closely related to file formats; but even there it can differ. Such as a low resolution version of a model, and an even lower resolution of the same model, both of which are .obj.
Great questions; it's getting much clearer for me just typing it out!
The images refer to a "representation".
We should have a definition for it as well.
How does that differ from an "instance"? It seems there could be multiple instances for a representation. How come?
By the way, still very pleased with the latest ideas that are on the drawing board. It seems it's getting solid.