This extension inserts an optional layer in between bufferView
s and buffer
s. This layer, called decompressedView
, performs Open3DGC decompression on part of a buffer
before it is accessed by the bufferView
.
In this example (a ball), we have:
- 960 indices (2 bytes/index): 1920 bytes
- 162 vertices (12 bytes/position, 12 bytes/normal): 3888 bytes
- 5808 bytes total uncompressed data
- 1194 bytes total compressed data (as compressed by collada2gltf -c Open3DGC -m binary, default compression parameters)
Uncompressed dae and various binary formats here.
In an engine, this can be loaded bottom-up (creating buffers first, and so on):
buffer [<- decompressedView] <- bufferView <- accessor <- mesh
(This is, roughly, the approach Cesium uses.)
or top-down (lazily loading objects as necessary):
mesh -> accessor -> bufferView -> [decompressedView ->] buffer
(This seems to be the approach MontageJS uses.)
As proposed, all bufferView
s, regardless of whether they come from compressed data, will represent data that can be passed directly to gl.bufferData
; decompression happens before this. No information needs to travel up or down the chain for decompression to be performed.
For discussions on alternative ideas, see Open alternative ideas.
For pseudocode for functionality added by this proposal, see Pseudocode.
{
"allExtensions": [
"mesh_compression_open3dgc"
],
Since an extra layer is inserted between bufferView
and buffer
, that layer
is stored here, in "extensions"
.
"extensions": {
"mesh_compression_open3dgc": {
"decompressedViews": {
"decompressedView_ball": {
A decompressedView
behaves much like a buffer
or bufferView
, but must be processed before it is used. It points to some data in a buffer
, which would be decompressed, then referenced by a bufferView
. In practice, decompressedView
s will map one-to-one with glTF meshes
. However, they are not extensions to the mesh
object, because it breaks the "bottom-up" method described above and breaks separation of concern in the "top-down" method.
"buffer": "buffer_ball_compressed", // buffer containing compressed data
"byteOffset": 0, // buffer offset pointing to compressed data
"byteLength": 1194, // compressed size
// no type; compressed data is always read as a straight byte array.
// TODO: useful but not necessary; will probably keep
"decompressedByteLength": 5808,
// Since Open3DGC distinguishes between floating-point and integral types
// but not between different sizes of those types, this defines the types
// of the arrays to decompress into.
"floatAttributeSizes": [ 5126, 5126 ],
"intAttributeSizes": [],
// Compression metadata
"mode": "binary"
}
}
}
},
accessor
s behave just as they normally do without the extension. There can be multiple accessor
s for mesh indices into the same bufferView
into a decompressedView
- each one is one primitive
.
"accessors": {
"accessor_ball_idx": {
"bufferView": "bufferView_ball_element",
"byteOffset": 0,
"byteStride": 0,
"count": 960,
"componentType": 5123, // uint16
"type": "SCALAR"
},
"accessor_ball_pos": {
"bufferView": "bufferView_ball_vertexdata",
"byteOffset": 0,
"byteStride": 0,
"count": 162,
"componentType": 5126, // float32
"type": "VEC3",
"max": [ 1, 1, 1 ],
"min": [ -1, -1, -1 ]
},
"accessor_ball_nor": {
"bufferView": "bufferView_ball_vertexdata",
"byteOffset": 1944,
"byteStride": 0,
"count": 162,
"componentType": 5126, // float32
"type": "VEC3",
"max": [ 1, 1, 1 ],
"min": [ -1, -1, -1 ]
}
},
bufferView
s for compressed data point into a decompressedView instead of a normal buffer. If there are any uncompressed meshes, they work exactly as usual.
Since the "buffer"
property isn't used, it could be omitted when using the extension.
"bufferViews": {
"bufferView_ball_element": {
// No buffer
"byteOffset": 0, // Offset into decompressedView
"byteLength": 1920, // Length in decompressedView
"target": 34963, // ELEMENT_ARRAY_BUFFER
"extensions": {
"mesh_compression_open3dgc": {
// The buffer property can go here, instead of outside.
"decompressedView": "decompressedView_ball"
}
}
},
"bufferView_ball_vertexdata": {
// No buffer
"byteOffset": 1920, // Offset into decompressedView
"byteLength": 3888, // Length in decompressedView
"target": 34962, // ARRAY_BUFFER
"extensions": {
"mesh_compression_open3dgc": {
// The buffer property can go here, instead of outside.
"decompressedView": "decompressedView_ball"
}
}
}
},
Buffers behave as usual.
"buffers": {
"buffer_ball_compressed": {
"byteLength": 1194, // compressed size
"type": "arraybuffer",
"uri": "ball.o3dgc.bin"
}
},
Meshes behave as usual.
"meshes": {
"geom-ball": {
"name": "ball",
// TODO: in core, consider moving "attributes" from below to here
"primitives": [ // TODO: in core, consider renaming "primitives"
{
"material": "whatever",
"mode": 4, // TRIANGLES. (See #341.) Implementation currently uses "primitive" as in glTF 0.8.
"indices": "accessor_ball_idx",
"attributes": {
"POSITION": "accessor_ball_pos",
"NORMAL": "accessor_ball_nor"
}
}
]
}
}
}
In the Open3DGC library, decompressing a single model results in individual homogeneous arrays for indices, positions, normals, and other attributes (which maps to a glTF mesh primitive). In order to define an output format which is efficient to upload and access from attributes, a format must be predefined for accessor
s to point into.
When decompressed, the resulting blob will look like this:
[--------model (primitive) 1---------]
[----indices----][positions][attrs...]
Those individual arrays will be be arranged into bufferView
s as follows:
[----------model (primitive) 1------------]
[------indices-------][positions][attrs...]
↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ ↓↓↓↓↓↓↓↓↓ ↓↓↓↓↓↓↓↓
[element array buffer][---array buffer---]
This allows an engine to simply construct a buffer from each of the decompressed arrays, then call gl.bufferData
exactly once for each bufferView
.
It is also possible to upload each part of an ARRAY_BUFFER
bufferView
using gl.bufferSubData
, avoiding the CPU/memory overhead of constructing the complete array for upload using gl.bufferData
.
This decompressed format will be defined by the extension specification. In particular, this defines the order in which arrays are stored into the GL buffer, and it defines the alignment of each of these arrays. For WebGL in particular, due to typed array semantics, this is an important consideration. See the pseudocode for constructing the decompressed blob.
-
Another, very different option is to connect model attributes directly with the homogeneous arrays that are returned by the decompression. Naively, this can be implemented with a single GL buffer for each attribute (positions, normals, etc.)
- Efficient implementation would require in-engine amalgamation of various arrays into one GL buffer. This can be done with
gl.bufferSubData
if sufficient metadata is provided, but prevents glTF from mapping closely to the GL API calls.
- Efficient implementation would require in-engine amalgamation of various arrays into one GL buffer. This can be done with
-
To allow for full expressiveness of layout (allowing multiple primitives to be stored in one
bufferView
, or attributes to be interleaved),accessor
s intobufferView
s could be used as if they are writable memory by the decompressor. Thus, some memory would be allocated by a bufferView, then the decompression step would write out index and attribute arrays into the accessors, allowing the format to be completely defined by information already stored in core glTF.- This may have significant conflicts with how a loader would be implemented without the extension (especially with regard to uploading to GL).
- There are also potential undefined behavior issues caused by this idea: since the loader must write into accessors, problems will arise if two accessors point into the same part of a bufferView - depending on which model or attribute is loaded first, the results might be completely different. The extension (or the glTF spec) would have to specify that either (a) accessors cannot overlap or (b) decompressing must always write the same data to the same place in memory.
-
I have considered structural compatibility with hypothetical hardware/API extensions for mesh compression. However, since many possible formats or structures could be used (including those described above plus considerations for interleaving), I deemed it not worth accounting for.
function loadGLTF() {
for (b in buffers) {
b.data = b.load();
}
for (d in decompressedViews) {
var decomp = Open3DGC.decompress(d.buffer.data[d.offset .. d.offset + d.length]);
d.data = produceDecompressedBlob(decomp.models); // (defined below)
}
for (bv in bufferViews) {
var data;
if (bv has extension open3dgc) {
data = bv.open3dgc.decompressedView.data[bv.offset .. bv.offset + bv.length];
} else {
data = bv.buffer.data[bv.offset .. bv.offset + bv.length];
}
gl.bufferData(bv.target, data, gl.STATIC_DRAW);
}
for (m in meshes) {
for (p in m.primitives) {
create VAO;
bind element array buffer from p.accessor;
for (a in p.attributes) {
bind array buffer from a;
}
}
}
// ...
}
This enforces an alignment scheme which is supported by Typed Arrays (e.g. Float32Array), whose offsets must be aligned with the type in the array.
function produceDecompressedBlob(decompressedModelObjects) {
var blob = [];
for (m in decompressedModelObjects) {
blob.padToMultipleOf(m.indices.BYTES_PER_ELEMENT);
blob.appendBytes(m.indices as bytes);
for (a in m.attributes) { // including positions and normals
blob.padToMultipleOf(a.BYTES_PER_ELEMENT);
blob.appendBytes(a as bytes);
}
}
}
decompressedBuffer
now can only contain one primitive, because multiple isn't possible with the schema I had. This is actually the same as how Fabrice had done the implementation. It may change again, but the schema will also have to change.- Removed [indices1][indices2][vertices1][vertices2] layout idea as that's no longer valid.
- Removed unnecessary
decompressedBuffer
fields. - Rename
decompressedBuffer
todecompressedView
. - Remove
bufferView.buffer
for bufferViews using compression. decompressedBuffer
can now hawe multiple primitives, but they must all use the same vertex data.