Skip to content

Instantly share code, notes, and snippets.

@Yanrishatum
Created June 10, 2020 05:20
Show Gist options
  • Save Yanrishatum/8bcad688e9716a5dc3b3d4b6cf2c05b1 to your computer and use it in GitHub Desktop.
Save Yanrishatum/8bcad688e9716a5dc3b3d4b6cf2c05b1 to your computer and use it in GitHub Desktop.
HMD Community Specification

HMD Community Spec

HMD format is not really complicated and separated into two primary areas: header and binary data. Later contains raw buffer data contents of which dependant on context. Spec is rather messy, but at least provides info on how format is structured.

Basic types

// ValueSize represents the value type used to write
// For example `Byte` would mean that to read the size you need to read single byte.
// Note regarding Floats: In Header section if value is 0, make sure it's not -0 and always +0.

// Arrays contain their entry count as first value and then the
// amount of entries said value specified
// However, ValueSize is contextual.
// `Array<String, Byte>` means that `count` is a one byte and entry type is a String type
typedef Array<T, S:ValueSize> = {
  var count:S;
  var entries:Array<T>;
}

// Pointer represents a pointer to the specific type
// Note that for whatever reason they are offset by 1 in the file
// There are exception
typedef Pointer<T> = Int;
function readPointer() return readInt32() - 1;
function writePointer(index:Int) return writeInt32(index + 1)
// As mentioned - some pointers don't have an offset (Materials in Models for example)
typedef Pointer2<T> = Int;

// Strings preface their size as a single byte
// Size of 0xff means `null`
typedef String = Std.String;
function writeString(s) {
  if (s == null) writeByte(0xff);
  else {
    writeByte(s.length);
    writeString(s);
  }
}

// Cached strings are not important for writer,
// but they are an optimization for reader in order to reduce memory footprint.
typedef CachedString = String;

// Properties are an enum that does not have a lot of use atm.
typedef Props = Array<Property, Byte>;
typedef Property = Data.Property;
function writeProperty(prop) {
  switch (prop) {
    case CameraFOVY(v):
      writeByte(0);
      writeFloat(v);
      // Used for settings camera FOV I guess?
    case HasExtraTextures:
      writeByte(2);
      // Used for Material information
      // If set, Material will try to read out specular and normal map texture names.
  }
}

Header

# Some data have binary chunks outside linear header. They are described in `Binary:` subblock.
# `[a + b]:` denotes data offset
# `name[len, format]` denotes name and length of binary data and format of said data. Length in bytes
class Header {
  Bytes<3> signature; // Always should be "HMD"
  Byte version; // Current version is 3
  Int32 dataPosition; // Offset from beginning of the file to the binary data
  Props props;
  Array<Geometry, Int32> geometries;
  Array<Materials, Int32> materials;
  Array<Model, Int32> models;
  Array<Animation, Int32> animations;
}

// Geometry represents an Index and Vertex buffer couple.
// See addendum A for binary format information.
class Geometry {
  Props props;
  Int32 vertexCount;
  Byte vertexStride;
  Array<GeometryFormat, Byte> vertexFormat;
  Int32 vertexPosition;
  Array<Int32, Byte> indexCounts;
  Int32 indexPosition;
  Bounds bounds;
  
  class GeometryFormat {
    CachedString name;
    // See Data.GeomtryDataFormat
    Byte geometryDataFormat;
    // DFloat=1
    // DVec2 = 2
    // DVec3 = 3
    // DVec4 = 4
    // DBytes4 = 9
  }
  
  class Bounds {
    float xMin
    float yMin
    float zMin
    float xMax
    float yMax
    float zMax
  }
}

class Material {
  Props props;
  String name;
  String diffuseTexture;
  h2d.BlendMode<Byte> blendMode; // See h2d.BlendMode values, stored as single Byte
  Byte culling = 1; // Deprecated: Always 1
  Float killalpha = 1f; // Deprecated: Always 1
  
  // Those are only present in file if Material.props contains HasExtraTextures property.
  ?String specularTexture;
  ?String normalMap;
}

class Model {
  Props props;
  CachedString name;
  Pointer<Model> parent;
  CachedString follow; // Have no idea what it used for atm
  Position position;
  Pointer<Geometry> geometry;
  
  // Present only if Model.geometry > 0
  // For obvious reason of not needing any of those if there's nothing to render.
  ?Array<Pointer2<Material>, Byte> materials;
  ?Skin skin;
}

// See addebdum B for binary format information
class Animation {
  Props props;
  String name;
  Int32 frames;
  Float sampling;
  Float speed;
  Byte flags; {
    Bool loop = flags & 1;
    Bool hasEvents = flags & 2;
  }
  Int32 dataPosition;
  Array<AnimationObject, Int32> objects;
  
  // Present only if hasEvents flag is set
  ?Array<AnimationEvent, Int32> events;
  
  class AnimationObject {
    CachedString name;
    Byte flags; // EnumFlags of Data.AnimationFlags
    
    // Present only if flags contains HasProps
    ?Array<String, Byte> props;
  }
  
  class AnimationEvent {
    Int32 frame;
    CachedString data;
  }
}

class Position {
  Float x; // Position
  Float y;
  Float z;
  Float qx; // Rotation ?
  Float qy;
  Float qz;
  // Not always present for Skins, see Skin `hasScale` flag.
  // If not present, defaults to 1
  ?Float sx; // Scale
  ?Float sy;
  ?Float sz;
}

class Skin {
  CachedString name;
  // If name is null, return null and do not read further.
  
  Props props;
  Array<SkinJoint, UInt16> joints;
  Array<SkinSplit, Byte> split;
  
  class SkinJoint {
    Props props;
    CachedString name;
    UInt16 pid; {
      // hasScale will reflect the contents of Position data
      Bool hasScale = pid & 0x8000;
      pid &= 0x7ffff;
    }
    Pointer<SkinJoint> parent;
    Position position;
    Pointer<Int32> bind; // It's just an int, but have same offset as pointer
    // Present only if bind >= 0
    ?Position transpos;
  }
  
  class SkinSplit {
    Byte materialIndex;
    Array<UInt16, Byte> joints;
  }
}

Addendum

A : Geometry binary data

Geometry binary data is located at two offsets representing vertex buffer and index buffer respectively. The format of vertex buffer is based on GeometryFormat data. For example [Position, Normal, UV, Alpha] format would be [DVec3, DVec3, DVec2, DFloat], resulting in vertexStride being the sum of those entries - 9 in that case.
Position of the buffers in the file is dictated by Header.dataPosition and then extra offset - vertexPosition and indexPosition respectively.

Vertex buffer is a Float32Array, hence byte size of data would be vertexCount + vertexStride * 4, with beginning at Header.dataPosition + vertexPosition.

Index buffer is an UInt16Array containing vertex indexes for rendering. Due to Models being able to contain multiple Materials, indexes are stored as an array, with each entry denoting amount of indexes Material will render sequentially. And sum of those entries should be exact amount of indexes in the index buffer. Hence data size would be sum(indexCounts) * 2 and position at Header.dataPosition + indexPosition.

Expected order of buffer contents: Position, Normals, Tangents, UVs, Colors, SkinData

B : Animation binary data

Same as geometry binary data, animation binary data can be accessed by an offset, but actual data contents are heavily dependant on the AnimationFlags of each object.

Possible animation values: Position, Rotation, Scale, UV, Alpha and special props.

When there's only one frame of animation, SingleFrame flag should be set and only one frame of data should be present. Otherwise exact amount of frames as animation frame count. As with vertex buffer, contents are floats and present in the data only if respective flag is set:
Position, Rotation, Scale, UVs, Alphas, Property Values

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment