HXSL cheat-sheet

hxsl-cheatsheet.md

HXSL cheat-sheet

This is a WIP of a cheat-sheet that I will finish Eventually™

Types

Mapping of the shader types to Heaps types:

Float = Float
Int = Int
Bool = Bool
String = String
Array<T, Size> = Array<T>
Vec2 / Vec3 / Vec4 = h3d.Vector
IVec2 / IVec3 / IVec4 = Array<Int>
BVec2 / BVec3 / BVEc4 = Array<Bool>
Mat2 / Mat3 / Mat4 / Mat3x4 = h3d.Matrix
Sampler2D = h3d.mat.Texture
Sampler2DArray = h3d.mat.TextureArray
SamplerCube = h3d.mat.Texture // Should have Cube flag
Bytes2 / Bytes3 / Bytes4 = Int
// Channels will have an extra variable that specifies which channels are used.
Channel / Channel2 / Channel3 / Channel4 = h3d.mat.Texture + hxsl.Channel
Buffer<T, Size> = h3d.Buffer
{ ... }

Reference: hxsl.Types and hxsl.Macros

Syntax

Qualifiers

Qualifier	Example	Description
@param	@param var texture:Sampler2D;	Represents a uniform field and can be set per shader instance. Uniforms are not shared between shaders unless specified as shared or are explicitly borrowed.
@shared	@shared @param var tint:Vec4;	Marks the uniform as shared uniform and makes in the shader code outside the shader it was declared in. To access the shared uniform, declare it with the same name without qualifiers, such as: var tint:Vec4;
@borrow(path.to.Shader)	@borrow(h3d.shader.Base2d) var texture:Sampler2d	Borrows a @param from another shader. It has to be present in the shader list, otherwise runtime shader compilatioin errors will occur.
@var	@var var vertexElevation:Float;	Represents a varying field that have to be set in the vertex shader and can be accessed in the fragment shader as interpolation in the rendered triangle.
@private	@private var internalCalc:Float;	Marks the varying as private and prevents it from being accessed from other shaders.
@global	@global var time:Float;	Represents a global uniform that is shared between multiple shaders. Globals are not accessible in 2D filters. See h3d.pass.Default and h2d.RenderContext for a list of globals available for 2D and 3D shaders.
@const	@const var enableExpensiveFeature:Bool;	Represent a shader compile-time constant that will cause unique shader to be produce for each variation of the constant. Parts of the code that are gated behind const check will be optimized out. Can be either @param or @global and interpreted as @param if not specified. Only supports Int and Bool types. Suppports max:Int parameter, but as of writing I'm not nure about it's practical use. See here (Haxe Discord log) for some info.
@input	@input var secondaryNormal:Vec3	TODO: Explain how input buffer work.
@lowp / @mediump / @highp	@highp var preciseData:Vec3;	Sets the floating-point precision of the variable. Only usable on Float, and Vec types.
@nullable	@nullable @param var optionalData:Float	Allows compare operations agains null. Use-cases are unclear.
@ignore	@ignore var tempVar:IVec2;	Causes variable to be ignore by HXSL inspector.
@:import	@:import h3d.shader.NoiseLib;	Imports the specified shader variables and methods. Note that if imported shader relies on data being set to it - it has to be added to the shader list manually. Library type shaders that only declare methods don't have to be added directly.
@:extends	@:extends h3d.shader.ColorAdd	Same as @:import, but also imports the main functions such as vertex(), fragment(), and __init__.
@perObject	@global var global: { @perObject var modelView:Mat4 }	Marks a @global uniform as unique per rendered object and causes it to be treated as @param. TODO: Figure out exact specifics.
@perInstance([size])	@perInstance(4) @input var instanceColor:Vec4;	Marks an @input as unique per instance with size denoting the data stride. Used in instanced rendering.

TODO: var name:T auto-type.

Primary shader entry-points.

"Static" shaders

Both in 3D and 2D context, there's a so-called "base" shader that performs initial and final data processing between your own shaders. They are always present in their respective contexts when object is rendered.

• For 3D scenes, the h3d.shader.BaseMesh shader is considered as its base shader.
• For 2D scenes, the h2d.shader.Base2d shader is considered as its base shader.
• Exception: Filters/post-process shaders. They do not have any base shader and instead should extend h3d.shader.ScreenShader and are considered a complete full shader program instead of a part of it.

The following sections use 2D context / h3d.shader.Base2d as a base shader, when examples are mentioned.

Shared variables

HXSL shader pipeline treats each shaders as a piece of the resulting shader that does come calculation. There's no direct communication between the shaders, but it's possible to share and modify variables that are not uniforms (with exceptions, see qualifiers). However there are dependencies of variables that may result in shader compilation errors. For example, assigning @var var calculatedUV:Vec2 in the fragment() and then assigning var pixelColor:Vec4 will resulting in compilation error, because initializer of pixelColor depends on textureColor and calculatedUV, hence it only should be assigned in the initializer.

When variable is declared at the shader level, its considered shared until it is a constant or uniform (with exceptions). And to use that variable from another shader, all that is required is to declare variable under the same name and type. It's not mandatory to use @var qualifier when declaring shared variable sourced from outside, as variable merging will recognize it as such if at least one declaration has this qualifier. If variable is considered unique (see qualifiers) - it will receive another name, but currently its recommended to avoid collision due to the following issue:

// ShaderA
@param var color:Vec4;
// ShaderB
var color:Vec3;
// ShaderC
var color:Vec3;
// Resulting shader:
@param var color:Vec4;
var color2:Vec3;
var color3:Vec3;

When name collides, newest variable is renamed and given new name with incremented number, but currently it does not check if it can merge the chained variable, and will treat them as new ones every time, since they check against the unique variable.

Initializers

Initializer method, as name suggests, peforms the initialization and initial assignment of the variables. Core point of initializer is that it handles the dependencies of the variables it initizlies, as mentioned in [Shared variables], allowing you to interject in the calculation of them. For example, assigning calculatedUV in the initializer will affect textureColor and pixelColor, because they are dependant on calculatedUV and compiler will put all modifications of the UV before initializing the colors. But beware of circular dependencies, you can't assign calculatedUV using pixelColor.

It's still possible to assign initialized variables, in main calculation methods, but if you modified calculatedUV, you no longer can access textureColor or pixelColor, as that would put them in an undefined state.

To declare the initializer, use the following methods:

function __init__() {}
function __init__fragment() {}
function __init__vertex() {}

The frament and vertex initializers can be used to put some initialization code unique only to specific render step, but most often it's not necessary, as HXSL DCE should optimize out unused variables in the __init__.

Main entry-point for shader computation

Now that initializers are processed, only thing that remains is the actual shader calculation code. It's pretty straightforward, as only thing required is the declaration of the method with rendering step name:

function vertex() {}
function fragment() {}

Priority

This is a good time to metnion shader priority. Each shader can be assigned the priority, and vertex / fragment code is executed according to that priority (lower priority is executed first). For example Base2d is assigned the priority of 100 by RenderContext, making its code being the last one to be executed. Which is logical, as it does the final output processing such as conversion of scene coordinates to viewport coordinates.

Unconfirmed: Most likely initializer code uses the reverse priority to determine the order.

Sampler methods

Works for all Sampler and Channel types unless specified otherwise.

Method	Sample	Notes
SamplerT.get(uv:VecT):Vec4	var pixel:Vec4 = sampler2.get(uv);	Samples the texture pixels at the UV coordinate and returns the resulting color.
SamplerT.get(uv:VecT, lod:Int):Vec4	var pixel:Vec4 = sampler2.get(uv, 1);	Samples the specified texture LOD pixels at the UV coordinate and returns the resulting color.
SamplerT.fetch(pos:IntT, ?lod:Int):Vec4	var pixel:Vec4 = sampler2.fetch(pos, 1);	Fetches the exact pixels from the sampler (with optional LOD) at position. Not supported for SamplerCube.
SamplerT.grad(pos:VecT, dPdx: VecT, dPdy: VecT)	var pixel:Vec4 = sampler2.grad(pos, gradX, gradY)	Performs a texture lookup with explicit gradients. See GLSL textureGrad (Since #1127)
SamplerT.size(?lod:Int):Vec2	var size:Vec2 = sampler2.size();	Returns the underlying texture size of the sampler.

• Channel type supports exactly same types with exception that it returns either Float (for 1 channel) or VecN with N being the amount of channels.
• Due to implementation specifics, get cannot be used in a while loop. Use getLod(uv, 0) instead.

Methods

dFdx	dFdy	fwidth	radians	degrees
sin	cos	tan	asin	acos
atan	pos	exp	log	exp2
log2	sqrt	inverseSqrt	abs	sign
floor	ceil	fract	mod	min
max	clamp	mix	step	smoothstep
length	distance	dot	cross	normalize
reflect	int	float	bool	pow
vec2	vec3	vec4
ivec2	ivec3	ivec4
bvec2	bvec3	bvec4
mat2	mat3	mat4	mat3x4
saturate	pack	unpack	packNormal	unpackNormal
screenToUv	uvToScreen

Extra notes

Shader debugging

By adding -D shader_debug_dump flag, you can enable the debug shader dumping, which can be useful if something goes wrong and you don't understand why. When compiled with that flag - every unique shader combination will be dumped into a file, with fairly verbose showcase of what it does and when. The file will consist of multiple segments:

• Datas - it will list all the source shaders that are being compiled together. You can verify what shaders are even present on that step.
• Link - this is where all shaders are merged together. Here you can verify that merged version contains correct instruction ordering, i.e. check if shaders are executed in correct order.
• Split - this step separates the vertex and fragment shaders, as on high level HXSL does no distinction on variables that are shared between fragment/vertex shaders and ones that aren't. Split stage introduces such distinction.
• DCE - this step will eliminate any unused paremeters to reduce the shader footprint. For example in case of BaseMesh - most shaders never use camera.zFar, and thus it's being removed by DCE.
• Flatten - here program is being simplified even further by merging same-type variables into arrays (so all vec4 params are stores as one big vec4 param array). Doing so improves general performance of the shader on binding stage, as less operations have to be done in order to bind the data.
• Output - the last step of shader compilation, the actual output to the target language (either HXSL or GLSL, based on your backend). At this stage you can check the actual code that will be sent to the graphics driver.

Non-main methods

HXSL inlines all method calls that are not main ones (i.e. not framgnet / vertex). This can lead to errors, as those so-called "Helper" methods have to be inlinable, i.e. have all returns being "final" expression.

However, consider the following:

function myHelperMethod(a:Vec2, b:Vec2, cond: Float): Vec2 {
  if (cond < 0.5) return a;
  return b;
}

This won't compile with Cannot inline not final return error due to first return not being "final", as it's not a last expression in the method body. While Haxe properly handles such cases for inline methods - HXSL doesn't. Thus you should structure your code in a way that will either only have one return or last expression being a branch that each ends with a return:

if (cond < 0.5) return a;
else retrun b;
// Since everything is an expression, we can move return outside of if block:
return if (cond < 0.5) a;
else b;
// Ternary works as well!
return cond < 0.5 ? a : b;
// Or we can store the result in variable and return it
var result = cond < 0.5 ? a : b;
return result;

This is literally exactly what I was looking for, thanks for your service to the universe :)

It looks like the method SamplerT.size(?lod:Int):Vec2 doesn't exist, or maybe I'm doing something wrong...

class MyShader extends h3d.shader.ScreenShader {
    static var SRC = {
        @param var texture : Sampler2D;

        function fragment() {
            texture.size();
        }
    }
}

Fails to compile with

characters 13-27 : Sampler2D has no field 'size'

@volnt You clearly use haxelib version of Heaps. My cheat sheet is specifically for git version, which is months ahead of haxelib version, including extra HXSL features such as size support and @borrow qualifier.

Ah ! Didn't know about that, thank you :)