USD

gistfile1.txt

As 2017 draws to a close and we prepare the last release for the year, 0.8.2, we wanted to additionally provide an update that includes a recap of the areas in which we’ve been focusing our development, as well as an outline of the problems on which we are either actively engaged or have on the books for the next six months horizon.

The major focus areas of development have been: 
Steering some optimization focus from reading USD to authoring USD
Finalizing UsdShade for both MaterialX conformance and usefulness in conjunction with instancing 
Preparing Hydra to handle the needs of “full quality” renderers as back-ends.
Removing impediments to preserving aggregate instances throughout our pipeline
Adding and fleshing out schemas, such as lights and skinning

Diving into a bit more detail,, we’ll break it down roughly by area.

USD Core

Authoring: Capabilities, and Performance

The number of clients demanding advanced and performant authoring of USD scene description has been growing.  We have been addressing this need on a number of fronts:
Copying Scene Description - properties can now be copied robustly in Usd using UsdProperty::FlattenTo(), and SdfPrimSpecs can be copied within a layer or between layers using SdfCopySpec.  Copying prims at the Usd-level is future work that is more difficult to usefully parameterize.
Flattening LayerStacks - Thanks to earlier groundwork that replaces the SdfListOp add and reorder statements with append and prepend, we now have the ability to flatten a layer and all of its nested subLayers into a single layer, while preserving all resolved composition arcs authored in the LayerStack, using UsdUtilsFlattenLayerStack.
Faster editing - we have identified a few areas in Usd’s “change processing” (i.e. the re-indexing and rebuilding of the UsdStage that happens in response to changes to SdfLayers that contribute to the stage) that can be improved to dramatically lower the response time for certain common forms of editing.  We will be tackling these issues shortly.
Batch Editing - faster editing response time for single edits only goes so far.  To dramatically improve the time to create tens of thousands to millions of prims from scratch, we require a Usd-level, batch editing API, so that edits can be accumulated to be processed all at once, allowing Usd’s multithreaded composition features to come into play.  Leveraging batch editing features will require use of a new API, but we believe we should be able to keep all the existing API’s working as they currently do, in a non-batch context.
Optimizing Data Access in Crate

After gaining valuable network metadata on Usd consumption from the rendering of Coco, we’ve been making a series of improvements, some of which required advancing the file format version for Crate.
Data Layout - modulo the effects that deduplication has on sample layout, timeSamples are laid out so that the data for same/similar times is collocated over all attributes, so that fetching all the data to render a single frame requires mostly contiguous pages in the file.
Optional pre-fetch management. Crate can be tasked (via environment variable) to manage its own network pre-fetch, rather than relying on the OS’s default settings.  For typical asset and cached datum sizes at Pixar, we found that our OS’s several megabyte-sized prefetch to often be wasteful in the context of high-demand farm rendering, and allowing crate to decide its own prefetch can substantially eliminate wasted traffic.  We are still profiling with this feature.
Lossless compression - We have deployed a combination of compression techniques for a crate file’s “footer” or structural section that is read in greedily when a crate file is opened.  Some crate files become substantially smaller, reducing network bandwidth requirements, without any noticeable latency degradation in the “warm cache” case.  We are proceeding to add compression for integer arrays.
Quantization - We are investigating adding support for bounds-adjusted quantization for floating-point vector array values.
Core Schema Work

We recently added an expressive UsdCollectionAPI schema to the core, which allows collections to be encoded compactly as a set of “paths to include” and “paths to exclude”, with the ability to specify whether the paths must match objects exactly, or match all prims at or below the paths, or all prims and properties below the paths.  The older, UsdGeomCollectionAPI will be deprecated in favor of the new, core schema.

“API” schemas are getting upgraded.  The set of API schemas that have been applied (using the API) to a prim will soon be recorded in core, list-edited metadata on the prim, which will make it possible to answer UsdPrim::HasA<UsdCollectionAPI>() queries, for example.  We may, in the future, be able to use this information to allow properties defined in API schemas to have fallback values, like those in Typed schemas.
Usd Geometry Schemas

The geometry schemas have been fairly stable.  Upcoming work includes:
Adding support in UsdGeomPointBased and UsdGeomPointInstancer to compute positions using velocities/angularVelocities.
Possibly adding the ability to inherit primvars down namespace, or adding a more general mechanism for inherited properties, given their usefulness in modulating shading and rendering; inherited attributes can provide a concise specification of overrides for objects unrelated in any way other than their location in namespace, which is not a categorization easily mimicked using composition arcs.  Namespace inheritance as an override mechanism for properties inside aggregate instances has had mixed success in our Renderman-based pipeline, due to its scattershot application (e.g. overriding a primvar called roughness on an instance root will cause all primvars named roughness on all prims inside the instance to be overridden; there is no way to scope the override to a subset of prims.  We believe we can create a specification that would allow such scoping, but are not convinced it justifies the extra complexity and expense.  If you have experience or thoughts to share on this matter, please let us know!
Generalized visibility. One of the last important groups of overrides we must be able to make on aggregate instances without breaking the instancing is being able to modify visibility of prims within the instances.  When we speak of visibility in this context, we do not mean the simple invisible/inherited visibility provided by UsdGeomImageable; rather, we refer to the more general visibility described by MaterialX, in which a “visibility statement” consists of:
A set of observed objects (expressed as a collection)
An optional set of observer objects (also expressed as a collection)
A type of visibility, canonical types being: camera, illuminates, shadows, secondary/bounce
This would provide us with a uniform mechanism for expressing light and shadow linking that also captures what Renderman calls “trace groups”... and also gives us a way to modify visibility of prims within instances, due to the nature of collections and relationships: this is important in our pipeline for being able to encode the results of frustum-pruning calculations without losing the benefits of instancing.
Usd Shading Schemas

We have been working towards two large (and frequently overlapping) goals in UsdShade: 
Reaching a satisfactory compliance with MaterialX such that all its important concepts have some native encoding in USD and its shaders/ops can be recorded in UsdShade shading networks
Ensuring that we have sufficient expressivity to feed “full-quality” renderers directly from USD, with a particular emphasis on keeping aggregate instancing intact, without losing the ability to have per-instance look variation.

Setting aside, for now, the tantalizing and rich problems of encoding/standardizing material layering, and standardizing bxdf and/or lobe vocabulary and structure (which we hope to attack one day, but not in the next six months), we see the remaining work falling into the following buckets:

Material Binding

We require a model for material binding that works well with aggregate instances, and helps to better organize the different types of shading that may often be simultaneously present in a scene.  We recently finalized such a proposal, which we are about to begin implementing.  Please see Material Assignment in UsdShade with Collections and Purpose for details.

Shader Registry

Given that we intend for Hydra to be able to arbitrate data for arbitrary back-end renderer implementations, and that we wish for the shading networks expressed in USD to be as sparsely authored as possible (i.e. not containing input values that are identical to an OSL shader’s default value for the input), we must provide a portable shader registry available to Hydra and all other clients of USD, such that complete shader data can be queried based on a shader’s authored id attribute.  Some aspects of the shader registry will include:
Provide a uniform specification of shader interface - inputs, outputs, default values, UI metadata.
Be extensible by plugin to decoding new kinds of shader interface specifications, but by default able to consume OSL metadata, Renderman “args” files, and a usd-based schema, which is what we will likely use for encoding MaterialX shader interfaces.
Be efficiently queryable from multiple threads simultaneously.
Be an oracle for finding asset implementations of shaders; for example, MaterialX shaders may provide both OSL and glsl implementations.

If you have concerns, questions, or ideas about a USD shader registry, please contact us!
Material Overrides

The last issue is the one over which we feel we have the least concrete grasp.  The problem is specific to aggregate instancing since, unlike in Katana, where “Material Override” is an actual node and ops whose purpose is to override material parameters at specific points in namespace, USD does not require a material-specific overriding mechanism: its standard override mechanism applies equally well to materials, shading networks, and geometry.

But aggregate instancing is special, because of the specialized association we can generally make between instances in the renderers that are our masters and bottlenecks in the VFX industry.  The ability to share tessellated geometry between instances is a huge advantage (for complex assets) in raytracing engines.  Although the true cost of per-instance variation in materials is still being debated and researched in our own pipeline, it is generally much less than losing sharing of geometry between instances.  
Although dynamic deduplication of gprims in renderers can rediscover sharability that was not expressed in USD, we gain great benefits in USD performance itself by deploying aggregate instancing as aggressively as possible - it was with this in mind that we designed a skeletal skinning schema that allows for entire skinned characters to be instanced.

The big question, therefore, is how much mechanism we need to provide to be able to override material properties inside aggregate instances?  Are the features described above sufficient (material binding, illumination/shadowing/visibility, and primvar or attribute inheritance)?  Or do we need to add a full-featured “MaterialOverride” schema that allows ancestor prims to override material interface properties of descendant prims, necessitating a “material property resolve” step required above-and-beyond USD’s normal value resolution in order to resolve material interface properties?

We have received mixed feedback on this issue internally - if you have experience or perspective to share, please contact us!
Usd Lighting Schemas

We incorporated some great external feedback into our nascent UsdLux lighting schemas, recently.  We are currently in the process of marshalling the UsdLux data to Hydra so that it can be consumed by backend renderers.  We have also begun baking out Gaffer nodes in Katana to (highly pixar-specialized) UsdLux-derived schemas in USD, though this work is not in a sharable state.
Usd Skeletal Schemas

We are beginning to flesh out the computation API’s for UsdSkel, with implementations adapted from the system used for skinning on Coco, as we try to transition our crowds pipeline to using the UsdSkel schemas.  Particular challenges that were not addressed in the original design document are what to do about ad hoc props, and the implied desire for constraints on skels.

We are also continuing our collaboration with Apple on adding blend shape support, as well as supported I/O from Maya.
Hydra and UsdImaging

The two over-arching themes of the work going into open-source Hydra are: 
Completing Hydra’s divorce from OpenGl and making it fully capable of handling full-quality renders, with fully-expressive shading networks, and motion-blur.
Continuing work on scalability features
Some specifics…
Material System

As the UsdShade binding and network-encoding model has been maturing, Hydra has been keeping pace with the introduction of materials and shading networks, with the intent of allowing front-ends like UsdImaging to provide “high level, user-level semantics” shading networks that can be transformed, when needed, by specific back-end renderers as they are consumed, while allowing for continued change-tracking in live re-rendering contexts.

An important part of the material system, currently being finalized, is a standardization for texture and uv orientation.

Geometry Model Card/Box Representations

Usd’s native instancing mechanism and, to a greater degree, the UsdGeomPointInstancer, allow us to specify and represent in-memory, enormous amounts of geometry very concisely.  It is common, in our pipeline, to create scenes that, while loading quickly and not taxing a GPU’s memory, nevertheless bring our preview renderer to a crawl for camera tumbling refresh, simply because we have overtaxed the GPU’s rasterization capabilities.

The solution we have deployed in the past in specific DCC’s is to allow pre-rendered texture cards to substitute for prototype geometry in PointInstancers.  We have recently generalized this capability for UsdGeom, allowing any UsdGeomModel - whether it is a PointInstancer prototype or a “free” model - so that we can stipulate, via attributes authored in the scenegraph, that some models should be drawn as cards, or untextured bounding boxes, or points.
Usdview

Following the change to UsdShade’s shader connection encoding to use “attribute connections” instead of sidecar relationships, usdview’s property browser recently got a major upgrade that displays relationships and attributes-with-connections as openable items, whose nested children are the target paths of the relationship or connection.  Further, any target path can now be used to navigate selection, via RMB context menu.

Much work has recently gone into improving usdview’s software architecture, teasing logical units of functionality apart and enforcing MVC structure.  This process will take awhile, as a) the starting point was a quick hack that grew and grew, b) the project is generally resource-starved.  But we have a near-term goal of replacing the current, internally-accessible-only, very loose, plugin system with a structured, efficient, command plugin mechanism available to all.

Houdini Plugins

While there have not been any major new features going into the Houdini USD plugins recently, there has been a fair amount of refinement and extension work, including the following:
Better overlay support for the PointInstancer
Improvements to the way primvars are imported and handled
Replacing the original lock-based stage-sharing mechanism with a simpler, more performant, “masked stage” cache mechanism (the UsdStage masking feature did not exist when the original sharing mechanism was deployed).

Additionally, we’ve had a fantastic ongoing dialog with SideFx since Siggraph, and they have already contributed code to enable our plugins to work with 16.5 as well as 16.0!
Maya Plugins

The Maya plugins are the the first place we will be deploying material collection creation and collection-based bindings, and we will ensure these provide a working solution to binding materials to native Maya aggregate instances.

Excitingly, we have also made progress recently on exporting MASH and traditional instancers to UsdGeomPointInstancers.  We would like to push further and enable the use of pxrUsdReferenceAssemblies as masters (which would result in prototypes-with-references in USD), but have no concrete timing, yet.
Katana Plugins

Many forms of “session overrides” on the operant Stage for a pxrUsdIn are now specifiable using a system based on graph-state variables.  These include, but are not limited to, deactivations, variant selections, and inherited attributes such as velocityScale that affect the internal computations that populate katana attributes from USD data.

Much work has gone into fleshing out the translation of UsdGeomPointInstancers into instance arrays, complete with velocity-based interpolation.

The most exciting news is that we are investigating a new representation in Katana, inspired by our complexity management techniques used first in Presto, and then Maya, which has the potential to allow us to keep the benefits of USD instancing live in Katana, in a more robust way than the prototype that attempts to create Katana instance sources for USD masters.

The pattern, which we intend to deploy first with instances in USD, but could be applied more generally, is to consider instance locations to be (mostly) untouchable (by CEL and most ops) USD archives that get emitted directly to the renderer.  What this means is that pxrUsdIn will prune population of locations beneath instances, so the user node-graph will not see any such locations.  Very late-firing, implicit resolvers will create the locations at render-time, allowing only a few ops to run on them, representing the few quantities we desire to be overridable within aggregate instances.  Initial scalability tests are promising, and this approach to handling USD instancing in Katana seems more robust than the earlier attempt.