Create a gist now

Instantly share code, notes, and snippets.

@mottosso /CHANGES.md Secret
Last active Aug 23, 2017

Embed
What would you like to do?
Asset Map 0.7

Asset Map

  • v0.7 - Uniting assets and shots

The conceptual overview of content in Magenta along with it's manifestation as a directory tree. The "map" provides a visual overview of the relationships between the 6 concepts, artist, asset, subset, family, version and instance.

Previous


image


Motivation

To enable a publish of any kind from any artist performing any task during shots, the idea of a "task" was delegated to the /work directory. Now the same is true for assets too. Less to worry about.

Changes from 0.6

  • Assets and shots united
    Previously, assets was publised into a task, whereas shots were not. Now neither is.

  • Item is Asset
    The term "Item" was replaced by "Asset". Shots are also assets.

Node Description
A transformation; i.e. input A turns into output B
Transformation from -> to
Instance, the inverse of a file. May have one or more representations
A representation, such as a .jpeg file or .exr sequence
Updated individually
A comment
Artist
Item
Subset
Family
Package
Versioned package

Asset

A directory with /work and /publish directories.

Work and Publish represents private and public files respectively. The private files are worked on in isolation by an individual artist who then "publishes" to the equivalent /public directory where other artist then may fetch it.

Instance

The inverse of a file.

When a file has been opened/imported into an application, the result is the "instance" of that file.

Package

The package is a group of subset versions.

  • Artists use packages directly
  • Artists publish packages in-directly
  • A new version of a package is created for each published Subset, such as ben01
  • Packages are only relevant within the /film directory

For example the animator imports a layoutPkg to start animating. He produces a animPkg to be used by lighters who produce a lightPkg to be used by compositors who produce compPkg used by grading and editing.

Motivation

The package compensates for the lack of association between subsets, such as ben01 and table01, where each subset is versioned independently. A package wraps one or more of these versions.

ben01                              animPkg
 v001                     /------->  v001
 v002                    |
 v003 -------------------|
table01                  |
 v001 -------------------/
 v002

Representation

One of potentially many ways of persisting the same set of data. At 10,000 feet, a project can be represented as either a Blu-Ray, or terabytes of models, animation, audio, renders and settings.

Motivation

Representations are relevant mainly when using the same set of data in different software.

For example, that high resolution hero model works great for renders, but less so for an asset browser. There, a smaller more accessible file format is preferable, such as a PNG. Perhaps a proxy version, such that it can still be rotated, but stored as an OBJ rather than ABC for simplified parsing in the Python based application.

The idea is that, whenever you save data, you save it in every way you can think to use it. Every model is not only saved as an MB, even though it will primarily be used in Maya. It is also saves as an OBJ, because you never know if you someday will want to open it in Mudbox. It's saved as a PNG, because you might not have access to either Maya nor Mudbox when time comes to simply look at the model.

Analogy

Think of it like portrait. The subject - such as a person - remains the same, but for every portrait strokes and colours differ slightly. Some are caricatures, some are line drawings. Some might be sculptures, or even 3d scans. Vastly different results, each representing the same subject. The same data.

In this case, the subject could be a model, whereas a representation could be an exact representation stored as an .obj, or an approximate one, such as a .png or poly-reduced alembic. Same data, different representations.

Subset

A "subset" is part of an Asset.

A subset consists of one or more Pyblish "instances".

Motivation

Sometimes, part of an Asset needs to update independently of one another.

For example, in a shot with two characters only one has changed and needs publishing. Another example, a shot where only one render layer has changed and needs publishing.

Instead of publishing the entire Item, needlessly consuming time, space and cpu cycles, a subset may be published and versioned individually.

The name "subset" reflect that it is not the whole Item, but only part of it.

"""Source code to represent a directory tree as JSON"""
import os
import json
def hierarchy_to_dict(path):
if os.path.isfile(path):
return ""
d = {n: hierarchy_to_dict(os.path.join(path, n)) for n in os.listdir(path)}
return d
d = hierarchy_to_dict(r'C:\Users\marcus\Dropbox\projects\MagentaHierarchy0.7')
print(json.dumps(d, indent=2, sort_keys=True))
@BigRoy

This comment has been minimized.

Show comment
Hide comment
@BigRoy

BigRoy Feb 18, 2016

The images refer to a "representation".

We should have a definition for it as well.

How does that differ from an "instance"? It seems there could be multiple instances for a representation. How come?

By the way, still very pleased with the latest ideas that are on the drawing board. It seems it's getting solid.

BigRoy commented Feb 18, 2016

The images refer to a "representation".

We should have a definition for it as well.

How does that differ from an "instance"? It seems there could be multiple instances for a representation. How come?

By the way, still very pleased with the latest ideas that are on the drawing board. It seems it's getting solid.

@mottosso

This comment has been minimized.

Show comment
Hide comment
@mottosso

mottosso Feb 18, 2016

By the way, still very pleased with the latest ideas that are on the drawing board. It seems it's getting solid.

That is awesome! I'm really excited about realising this map, I think it could be the foundation for quite a few pipelines. Even the likes of Framestore but also smaller ones.

It seems there could be multiple instances for a representation. How come?

It's the opposite, a single instance - such as a Rig - consists of many representations - such as an .mb and an .fbx.

How to define a representation? Think of it like portrait. The subject - such as a person - remains the same, but for every portrait strokes and colours differ slightly. Some are caricatures, some are line drawings. Some might be sculptures, or even 3d scans. Vastly different results, each representing the same subject. The same data.

In this case, the subject could be a model, whereas a representation could be an exact representation stored as an .obj, or an approximate one, such as a .png or poly-reduced alembic. Same data, different representations.

Owner

mottosso commented Feb 18, 2016

By the way, still very pleased with the latest ideas that are on the drawing board. It seems it's getting solid.

That is awesome! I'm really excited about realising this map, I think it could be the foundation for quite a few pipelines. Even the likes of Framestore but also smaller ones.

It seems there could be multiple instances for a representation. How come?

It's the opposite, a single instance - such as a Rig - consists of many representations - such as an .mb and an .fbx.

How to define a representation? Think of it like portrait. The subject - such as a person - remains the same, but for every portrait strokes and colours differ slightly. Some are caricatures, some are line drawings. Some might be sculptures, or even 3d scans. Vastly different results, each representing the same subject. The same data.

In this case, the subject could be a model, whereas a representation could be an exact representation stored as an .obj, or an approximate one, such as a .png or poly-reduced alembic. Same data, different representations.

@BigRoy

This comment has been minimized.

Show comment
Hide comment
@BigRoy

BigRoy Feb 18, 2016

Ok, that clarifies things. How would you extract different "representations" in Pyblish?

I can imagine they might be the same, but also different families (other formats might require a different set of validations). This is actually supported by Pyblish now, instances in multiple families... as such the families might export their own representations.

Does this mean that the terminology for "family" is currently becoming a conflict here? Or would representations always be of a single family? If so how would you setup the different "representations" output for it?

BigRoy commented Feb 18, 2016

Ok, that clarifies things. How would you extract different "representations" in Pyblish?

I can imagine they might be the same, but also different families (other formats might require a different set of validations). This is actually supported by Pyblish now, instances in multiple families... as such the families might export their own representations.

Does this mean that the terminology for "family" is currently becoming a conflict here? Or would representations always be of a single family? If so how would you setup the different "representations" output for it?

@mottosso

This comment has been minimized.

Show comment
Hide comment
@mottosso

mottosso Feb 18, 2016

Two extractors of the same instance are by definition producing two representations. The instance is the data that remains the same, whereas the outputted files/data/whatever are the representations.

Families and representations don't necessarily have anything in common. A family of rig can still be represented by both an .mb, as an .obj, without the controllers and deformers, and a .png.

Representations are more closely related to file formats; but even there it can differ. Such as a low resolution version of a model, and an even lower resolution of the same model, both of which are .obj.

Owner

mottosso commented Feb 18, 2016

Two extractors of the same instance are by definition producing two representations. The instance is the data that remains the same, whereas the outputted files/data/whatever are the representations.

Families and representations don't necessarily have anything in common. A family of rig can still be represented by both an .mb, as an .obj, without the controllers and deformers, and a .png.

Representations are more closely related to file formats; but even there it can differ. Such as a low resolution version of a model, and an even lower resolution of the same model, both of which are .obj.

@mottosso

This comment has been minimized.

Show comment
Hide comment
@mottosso

mottosso Feb 18, 2016

Great questions; it's getting much clearer for me just typing it out!

Owner

mottosso commented Feb 18, 2016

Great questions; it's getting much clearer for me just typing it out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment