Last active
January 7, 2024 17:11
-
-
Save jamesu/9d25c16d5d11b402f9dc75d11df76177 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Tribes 1 Model Formats | |
| ---------------------- | |
| Tribes 1 uses file formats which are similar to file formats used in the Torque Game Engine, with the major difference being more use of tags and versioned objects serialized by class name (working on the same basic principle of class instantiation in torques' console system). Also a lot more emphasis on paletted textures. | |
| Currently this document only covers enough to render .dts and interior (.dig,.dis,.dil) shapes present in Tribes 1. For tribes 2 file formats, your best bet is to check out the earlier torque code. | |
| We'll refer to each field as [type] [name or tag value]. If successive fields need to be at a specific offset indicated by a field, that will be noted with "@nameOffset:" where appropriate. In addition if there is a list of typed data, it will be listed as "type[size]:" followed by the fields of that type. | |
| Common types | |
| ------------ | |
| * Tag | |
| uint32 storing a tag, usually followed by the size of the proceeding data. | |
| * ChunkSize | |
| uint32 where the last bit indicates if the file size should be dword aligned, in which case the amount of data you need to read is ((ChunkSize & ~0x80000000) + 3) & ~3. | |
| * ChunkList | |
| A successive list of chunks which will be read until a number of chunks have been read. The number of chunks needs to be read somewhere in the second chunk, otherwise processing will be aborted. | |
| * Chunk | |
| A simple tuple of [Tag, ChunkSize] where ChunkSize is the sum of the size of the stored fields. | |
| * Point2F | |
| float[2] representing a point. | |
| * Point3F | |
| float[3] representing a point. | |
| * Quat16 | |
| uint16[4] representing a quaternion. To decode you divide each component by 0x7fff. | |
| * QuatF | |
| float[4] representing a quaternion. | |
| * Mat3F | |
| float[3][3] representing a matrix in row-major format. | |
| * String | |
| uint16 length followed by the string data. If the last bit of the length is set, the actual size will be increased for dword padding i.e. the size to read is ((length & ~0x80000000) + 3) & ~3. | |
| * VersionedPersBlock | |
| A serialized object. Stored as follows: | |
| Tag 'PERS' | |
| uint32 chunkSize | |
| String className | |
| int32 version | |
| Volume Format | |
| ------------- | |
| Tribes '.vol' files are stored as follows: | |
| Tag 'PVOL' | |
| uint32 stringBlockOffset | |
| for each file: | |
| Tag 'VBLK' | |
| ChunkSize encodedSize | |
| @stringBlockOffset: | |
| Tag 'vols' (@stringBlockOffset) | |
| ChunkSize volsSize | |
| char[volsSize] stringData | |
| Tag 'voli' | |
| ChunkSize voliSize (should be multiple of entry size) | |
| for each file entry: | |
| uint32 id (always 0, not used in tribes) | |
| int32 filenameOffset (relative to @stringBlockOffset+8) | |
| int32 fileOffset (offset to relevant VBLK data) | |
| uint32 size | |
| uint8 compressType (0 = none, 1 = RLE, 2 = lzss, 3 = lha) | |
| File data is typically stored uncompressed, however older Dynamix games like Red Baron II actually use the compression. All compressed data is compressed in chunks of 500 bytes. | |
| For RLE, the following pseudocode should suffice: | |
| until input consumed: | |
| read byte | |
| if byte & 0x80: | |
| read repeat_byte | |
| emit repeat_byte byte & ~0x80 times | |
| else: | |
| read byte bytes | |
| emit bytes | |
| For LZH, using the standard algorithm without a starting size should work. | |
| Palette Format | |
| -------------- | |
| Tribes makes use of palettes. Normally a mission will load a single '.ppl' file which contains multiple palettes. Earlier games may make use of a simpler palette format with only 1 single palette in them. In addition microsoft palette files can be read. | |
| Tag 'PL98' | |
| uint32 numPalettes | |
| int32 shadeShift (shadeLevels = 1<<shadeShift) | |
| int32 hazeLevels | |
| int32 hazeColor | |
| uint8[32] allowedMatches (bit vector which marks colors that can be matched against) | |
| Data[numPalettes]: | |
| uint32[256] colors | |
| int32 index (batches up with index entry in bitmap) | |
| uint32 type (noremap=0, shadehaze=1, translucent=2, colorquant=3, alphaquant=4, additivequant=5, additive=6, subtractivequant=7, subtractive=8) | |
| [remap data] | |
| uint32 weightPresent | |
| if weightPresent != 0: | |
| float[256] colorWeights | |
| uint32 weightStart | |
| uint32 weightEnd | |
| Remap data size and content varies depending on the types of palettes used. Each maping table should be associated with the relevant palette data. Unless you are writing a software renderer however, the remap data is largely not useful. But it still needs to skipped past to properly read the palette. | |
| for each data: | |
| if type is shadehaze: | |
| uint8[256 * shadeLevels * hazeLevels] shadeMap | |
| if type is translucent or additive or subtractive: | |
| uint8[256 * 256] transMap | |
| for each data: | |
| if type is translucent, shadeHaze, additive or subtractive: | |
| uint8[256] colorIndex | |
| float[256] colorRed | |
| float[256] colorGreen | |
| float[256] colorBlue | |
| for each data: | |
| if type is noremap: | |
| uint8[256] colorIndex | |
| float[256] colorRed | |
| float[256] colorGreen | |
| float[256] colorBlue | |
| Phoenix Bitmap Format | |
| --------------------- | |
| Tribes uses its own bitmap format, which is similar to what you would get if you serialized a bitmap in torque... only it's mostly limited to paletted textures. It also supports loading microsoft bmp files. | |
| In the case of microsoft bitmap files tribes will assign a palette index to them from bfReserved2 if `bfReserved1 == 0xf5f7 and bfReserved2 != 0xffff`, in which case the colors from that palette will be used instead of those provided by the bitmap. | |
| Tribes '.bmp' files are serialized as follows: | |
| Tag 'PBMP' | |
| uint32 numPalettes | |
| ChunkList | |
| if Chunk = 'head': | |
| uint32 version | |
| uint32 width | |
| uint32 height | |
| uint32 bitDepth (should always be 8) | |
| uint32 attribute (normal=0x0, transparent=0x1, fuzzy=0x2, translucent=0x4, ownmeme=0x8, additive=0x10, subtractive=0x20, alpha8=0x40) | |
| [numChunks = version & 0x00ffffff] | |
| [stride = ((width * bitDepth >> 3)+3)&~3] | |
| if Chunk = 'DETL': | |
| int32 mipLevels (maximum = 9) | |
| if Chunk = 'DATA': | |
| uint8[Chunk.size] data (mip0, mip1, mip2, ...) | |
| If Chunk = 'piDX': | |
| int32 paletteIndex | |
| If Chunk = 'RIFF': | |
| [microsoft palette data] | |
| When loading bitmaps the colors used will be taken from a palette data entry equal to the paletteIndex value. It's also possible for a palette to be present in the bitmap data, in which case that can be used. | |
| If loading bitmap data into OpenGL, you'll want to use the alpha channel if transparent or translucent is set. For transparent images, alpha should be 0 or 255. You'll also want to set the following modes depending on the attribute flags: | |
| if (transparent) | |
| glEnable(GL_ALPHA_TEST); | |
| glAlphaFUnc(GL_GREATER, 0.65f); | |
| if (additive) | |
| glBlendFunc(GL_SRC_ALPHA, GL_ONE); | |
| else if (subtractive) | |
| glBlendFunc(GL_ZERO, GL_ONE_MINUS_SRC_COLOR); | |
| else | |
| glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA); | |
| Material Lists | |
| -------------- | |
| These are stored either in '.dml' files or at the end of '.dts' files. | |
| Tag 'PERS' | |
| uint32 chunkSize | |
| String className 'TS::MaterialList' | |
| uint32 numMaterials | |
| Material[numMaterials]: | |
| uint32 flags | |
| float alpha | |
| uint32 index | |
| uint8[3] rgb | |
| if version < 2: | |
| uint8[16] filename | |
| else: | |
| uint8[32] filename | |
| if version == 1 or 2: | |
| uint32 type | |
| float elasticity | |
| float friction | |
| if version == 1 or version > 3: | |
| uint32 useDefaultProps (assumed to be 1 if not present) | |
| DTS Shapes | |
| ---------- | |
| The format for the root shape object in '.dts' files is as follows: | |
| Tag 'PERS' | |
| uint32 chunkSize | |
| String className 'TS::Shape' | |
| int32 version | |
| uint32 numNodes | |
| uint32 numSequences | |
| uint32 numSubSequences | |
| uint32 numKeyframes | |
| uint32 numTransforms | |
| uint32 numNames | |
| uint32 numObjects | |
| uint32 numDetails | |
| uint32 numMeshes | |
| if version >= 2: | |
| uint32 numTransitions | |
| if version >= 4: | |
| uint32 numFrameTriggers | |
| float radius | |
| Point3F center | |
| if version > 7: | |
| Point3F minBounds (can be estimated with center + radius and refined with points) | |
| Point3F maxBounds | |
| Nodes[numNodes]: | |
| if version <= 7: | |
| int32 name | |
| int32 parentNode | |
| int32 numSubSequences | |
| int32 firstSubSequence | |
| int32 defaultTransform | |
| else: | |
| int16 name | |
| int16 parentNode | |
| int16 numSubSequences | |
| int16 firstSubSequence | |
| int16 defaultTransform | |
| Sequences[numSequences]: | |
| if version >= 5: | |
| int32 name | |
| int32 cyclic | |
| float duration | |
| int32 priority | |
| int32 firstTriggerFrame | |
| int32 numTriggerFrames | |
| int32 numIFLSubSequences | |
| int32 firstIFLSubSequence | |
| else if version >= 4: | |
| int32 name | |
| int32 cyclic | |
| float duration | |
| int32 priority | |
| int32 firstTriggerFrame | |
| int32 numTriggerFrames | |
| else: | |
| int32 name | |
| int32 cyclic | |
| float duration | |
| int32 priority | |
| SubSequence[numSubSequences]: | |
| if version <= 7: | |
| int32 sequenceIdx | |
| int32 numKeyFrames | |
| int32 firstKeyFrame | |
| else: | |
| int16 sequenceIdx | |
| int16 numKeyFrames | |
| int16 firstKeyFrame | |
| KeyFrame[numKeyframes]: | |
| if version < 3: | |
| float pos | |
| uint32 key (transform index - keyMask=0x3FFFFFFF, vis=0x80000000, valid=0x40000000) | |
| else if version <= 7: | |
| float pos | |
| uint32 key (transform index or mesh frame) | |
| uint32 matIndex (matMask=0x0FFFFFFF, vis=0x8000, visMatters=0x40000000, frameMatters=0x10000000, matMatters=0x20000000) | |
| else: | |
| float pos | |
| uint16 key (transform index or mesh frame) | |
| uint16 matIndex (matMask=0x0FFF, vis=0x8000, visMatters=0x4000, frameMatters=0x1000, matMatters=0x2000) | |
| Transform[numTransforms]: | |
| if version < 7: | |
| QuatF rot | |
| Point3F pos | |
| Point3F scale | |
| else if version == 7: | |
| Quat16 rot | |
| Point3F pos | |
| Point3F scale | |
| else: | |
| Quat16 rot | |
| Point3F pos | |
| Name[numNames]: | |
| char data[24] | |
| Object[numObjects]: | |
| if version <= 7: | |
| int16 name | |
| uint16 flags (0x1 = invisible by default) | |
| int32 meshIndex | |
| int16 nodeIndex | |
| Point3F offset (relative to node) | |
| int16 numSubSequences | |
| int16 firstSubSequence | |
| else: | |
| int16 name | |
| uint16 flags (0x1 = invisible by default) | |
| int32 meshIndex | |
| int32 nodeIndex | |
| uint32 offsetFlags | |
| Mat3F offsetRot | |
| Point3F offset (relative to node) | |
| int32 numSubSequences | |
| int32 firstSubSequence | |
| Detail[numDetails]: | |
| int32 rootNode | |
| float size | |
| if version >= 2: | |
| Transition[numTransitions]: | |
| if version < 7: | |
| int32 startSequence | |
| int32 endSequence | |
| float startPosition | |
| float endPosition | |
| float duration | |
| QuatF transformRot | |
| Point3F transformPos | |
| Point3F transformScale | |
| else if version == 7: | |
| int32 startSequence | |
| int32 endSequence | |
| float startPosition | |
| float endPosition | |
| float duration | |
| Quat16 transformRot | |
| Point3F transformPos | |
| Point3F transformScale | |
| else: | |
| int32 startSequence | |
| int32 endSequence | |
| float startPosition | |
| float endPosition | |
| float duration | |
| Quat16 transformRot | |
| Point3F transformPos | |
| if version >= 4: | |
| FrameTrigger[numFrameTriggers]: | |
| float pos | |
| int32 value | |
| if version >= 5: | |
| int32 defaultMaterials | |
| if version >= 6: | |
| int32 alwaysNode | |
| Mesh[numMeshes]: | |
| VersionedPersBlock object (always 'TS::CelAnimMesh') | |
| int32 hasMaterials | |
| if hasMaterials: | |
| VersionedPersBlock materialList (always 'TS::MaterialList') | |
| DTS Meshes | |
| ---------- | |
| Meshes are persisted in the main DTS shape file in the mesh list. The only type of mesh is 'TS::CelAnimMesh', formatted as follows: | |
| Tag 'PERS' | |
| uint32 chunkSize | |
| String className 'TS::CelAnimMesh' | |
| int32 version | |
| int32 numVerts | |
| int32 vertsPerFrame | |
| int32 numTextureVerts | |
| int32 numFaces | |
| int32 frames | |
| if version >= 2: | |
| int32 textureVertsPerFrame (for lower versions, should be assumed = numTextureVerts) | |
| if version < 3: | |
| Point3F scale (this was moved into the frame data for versions >= 3) | |
| Point3F origin | |
| float radius | |
| PackedVertex[numVerts]: | |
| uint8 x | |
| uint8 y | |
| uint8 z | |
| uint8 encodedNormal (index into the encoded normal table, same as the one in torque) | |
| Point2F[numTextureVerts] | |
| Face[numFaces]: | |
| int32 vertIndex0 | |
| int32 texIndex0 | |
| int32 vertIndex1 | |
| int32 texIndex1 | |
| int32 vertIndex2 | |
| int32 texIndex2 | |
| int32 matIndex | |
| Frame[numFrames]: | |
| int32 firstVert | |
| if version >= 3: | |
| Point3F scale | |
| Point3F origin | |
| To unpack the vertices, multiply each element by the frame scale and add on the frame origin, or in the case of older shapes, use the scale and origin present in the mesh data. Frame data for meshes is controlled at runtime by keyframe data. | |
| Rendering Shapes | |
| ---------------- | |
| Tribes will use the "Detail" data and the "alwaysNode" field to determine what to render. It's assumed the first node is always the "bounds" node, and the whole shape is moved by the inverse transform of that node to center the object. | |
| In pseudocode: | |
| select detail level based on size of projected radius | |
| renderNode(alwaysNode) | |
| renderNode(selectedDetail.rootNode) | |
| renderNode(node): | |
| calculate node transform | |
| render node objects relative to transform | |
| renderNode(node.children) | |
| Shapes have a default pose defined by the "defaultTransform" field in each Node which can be used in the absence of sequence data. Node transforms can simply be calculated by accumulating the transforms of the parent nodes, though keep in mind the translation component should be calculated by `parentPos + (parentRot * localPos)`. | |
| All object meshes should be rendered relative to the associated node transform, and offset by the "offset" field in the object. | |
| To incorporate sequence data, you need some sort of thread object which keeps track of the following: | |
| * Current sequence | |
| * Playback position | |
| * Any transitioning state (if transitioning between sequences) | |
| * Object states (mesh vert frame, mesh texvert frame, visiblity state) | |
| * Node visibility states | |
| For each animated node and object, there is a list of subsequences associated with each sequence where there is an animation track. | |
| Animation tracks for objects change the mesh vert frame, texture vert frame, and visibility. | |
| Animation tracks for nodes change the node transform and visibility. | |
| For nodes, simply select the closest keyframes and interpolate between them. You'll want to interpolate rotations between keyframes the same way as torque does (refer to the interpolate method on its QuatF class). | |
| For objects, simply change the frames or visibility as you pass the keyframe. | |
| Interiors | |
| --------- | |
| Interiors in tribes are split up into 4 files: | |
| - A ".dis" file which indexes everything required | |
| - One or more ".dig" files which store geometry | |
| - One of more ".dil" files which store lighting information | |
| - One ".dml" file which is a serialized material list | |
| DIS Index | |
| --------- | |
| Tag 'ITRs' | |
| uint32 chunkSize | |
| int32 numStates | |
| State[numStates]: | |
| uint32 stateNameIdx | |
| uint32 lodIdx | |
| uint32 numLods | |
| int32 numLods | |
| Lod[numLods]: | |
| uint32 minPixels | |
| uint32 geomNameIndx (index into names list) | |
| uint32 lightStateIdx | |
| uint32 linkableFaces | |
| int32 numLodLightStates | |
| LodLightState[numLodLightStates]: | |
| uint32 bits | |
| int32 numLightStates | |
| LightState[numLightStates]: | |
| uint32 bits | |
| int32 nameSize | |
| char[nameSize] names | |
| int32 materialListIdx (index into names list) | |
| bool linkedInterior | |
| Each Lod defines geometry for a particular detail level, shown at the `minPixels` level. This must be loaded from the file indicated by `geomNameIndx` from the same VOL file. | |
| Similarly a filename for a DML file is specified via `materialListIdx`, and a DIL file via `stateNameIdx` | |
| DIG Geometry | |
| ------------ | |
| Tag 'PERS' | |
| uint32 chunkSize | |
| String className 'ITRGeometry' | |
| int32 version | |
| int32 buildId | |
| float textureScale | |
| Point3F minBounds | |
| Point3F maxBounds | |
| int32 numSurfaces | |
| int32 numBSPNodes | |
| int32 numSolidLeafs | |
| int32 numEmptyLeafs | |
| int32 numPVSBits | |
| int32 numVerts | |
| int32 numPoint3Fs | |
| int32 numPoint2Fs | |
| int32 numPlanes | |
| Surface[numSurfaces] | |
| uint8 flags | |
| uint8 materials | |
| uint8 tsX | |
| uint8 tsY | |
| uint8 toX | |
| uint8 toY | |
| uint16 planeIdx | |
| uint32 vertIdx | |
| uint32 pointIdx | |
| uint8 numVerts | |
| uint8 numPoints | |
| BSPNode[numBSPNodes] | |
| uint16 planeIdx | |
| int16 front | |
| int16 back | |
| int16 fill | |
| BSPLeafSolid[numSolidLeafs] | |
| uint32 surfIdx | |
| uint32 planeIdx | |
| uint16 numSurfaces | |
| uint16 numPlanes | |
| BSPLeafEmpty[numEmptyLeafs] | |
| uint16 flags | |
| uint16 numSurfs | |
| uint32 pvsIdx | |
| uint32 planeIdx | |
| Point3F minBounds | |
| Point3F maxBounds | |
| uint16 numPlanes | |
| uint8[numPVSBits] | |
| Vertex[numVerts] | |
| uint16 pIdx | |
| uint16 tIdx | |
| Point3F[numPoint3Fs] | |
| Point2F[numPoint2Fs] | |
| Plane[numPlanes] | |
| float x | |
| float y | |
| float z | |
| float d | |
| int32 highestMip | |
| uint32 flags | |
| DIL Lighting Info | |
| ----------------- | |
| TODO | |
| Rendering Interiors | |
| ------------------- | |
| Tribes will render an interior first based on the active `State`, then dependent on the current suitable `Lod` level from the range specified by `lodIdx...(lodIdx+numLods)`. | |
| To render any particular `Surface`, you should use `Vertex` records specified from the range `vertIdx...(vertIdx+numVerts)`. These will specify a `tIdx` textureCoord which should be taken from the `Point2F` list, and a `pIdx` position from the `Point3F` list. | |
| Texture coords should additionally be scaled by the `tsX` and `tsY` values, and offsetted using the `toX` and `toY` values using the following algorithm: | |
| scale.x = (float)((int)surface.tsX+1) / texture_width | |
| scale.y = (float)((int)surface.tsY+1) / texture_height | |
| offset.x = (float)surface.toX / (float)texture_width | |
| offset.y = (float)surface.toY / (float)texture_height | |
| The surface normal is specified by the `planeIdx` Plane. Finally, the actual material is taken from the associated material list using the `materials` property. | |
| TODO: figure out how lighting info is mapped, and how BSP nodes are evaluated. | |
| ------------------- | |
| And that's it for now. Hopefully this should be enough to get anyone started with prodding around these assets! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment