Description of Metal Shader Language

What Is Metal

# What Is Metal
Processing executed in parallel on the GPU can use a Shading language called Metal’s original Metal Shading Language (MSL).

The Metal shading language is a unified language that allows tighter integration between the graphics and compute programs.
MSL Filename Extension

A MSL filename extension is .metal. You can create a new metal file using the metal file template in Xcode.

Using .metal extension allows Xcode to recognize MSL files in your project, automatically build a default library at build time, and help you profile and debug source code with specialized Metal tools.

The built default library (default.metallib) is added to the application bundle. You can get the default library as follows:

let library = device.makeDefaultLibrary()// makeDefaultLibrary() is equivalent to:let filePath = Bundle.main.path(forResource: "default", ofType: "metallib")!
let library = try! device.makeLibrary(filepath: filePath)

Metal files can be built manually as follows: (Check 【Building a Library with Metal's Command-Line Tools】 for details)

```xcrun -sdk macosx metal -c MyLibrary.metal -o MyLibrary.air```
```xcrun -sdk macosx metallib MyLibrary.air -o MyLibrary.metallib```

## Metal Graphics Rendering Pipeline
https://miro.medium.com/max/720/1*UwWyrEbOtEIi19f4hRHpFA.webp
Metal Graphics Rendering Pipeline, Apple Developer Document by Apple Inc.

A render pipeline processes drawing commands and writes data into a render pass’s targets. A render pipeline has many stages, some programmed using shaders. Vertex Function and Fragment Function in rendering pipeline need to be written MSL. The main stages (from Primitives to Fragment Function) are as following flow:

https://miro.medium.com/max/720/1*YsVaZBmSIjPhS2S0NnHxSQ.webp
Main stages of the Metal graphics render pipeline, Apple Developer Document by Apple Inc.

    * A vertex group of primitives that are basic figures composed of vertices such as points, lines, triangles, etc. are passed to the vertex shader.
    * Vertex shader performs calculations such as coordinate transformation. The output vertex group is passed to the Rasterization.
    * Rasterization rasterizes vertices and passes raster-data to the Fragment shader
    * The fragment shader determines the color of each pixel.

Vertex Function and Fragment Function are described in detail in the Function section.

## Metal Coordinate Systems
Metal defines several standard coordinate systems to represent transformed graphics data at different stages along the rendering pipeline.

### Clip-space Coordinates
A vertex shader generates positions in clip-space coordinates. A 3D point in clip space coordinates is specified by a 4D homogeneous vector (x, y, z, w).

https://miro.medium.com/max/318/1*JVHnBpmwAWgzvF0jaklHWA.webp

Metal divides the x, y, and z values by w to convert clip-space coordinates into normalized device coordinates. it’s called perspective division. 
The following equality defines the relationship between the normalized device coordinates and clip coordinates.

https://miro.medium.com/max/442/1*2rnbm2lMQS27cRmyCcGQbA.webp

### Normalized Device Coordinates (NDC)
(x, y, z) indicated by the normalized device coordinate system is in the range of −1 to 1. Positive-z values point away from the camera. NDC use a left-handed coordinate system and map to positions in the viewport.


### Viewport coordinates
https://miro.medium.com/max/720/1*svoRX74-u7V3_D6q2Hojvg.webp

The rasterizer stage transforms NDC into viewport coordinates. A viewport is an area displayed on the screen. The (x,y) coordinates in this space are measured in pixels, with the origin in the top-left corner of the viewport and positive values going to the right and down.

### Texture coordinates
https://miro.medium.com/max/640/1*43WG4PfOjzRcc2qks-fQNQ.webp

Texture coordinates indicate floating-point positions that map locations on a texture image to locations on the geometric surface is represented as 2D or 3D vectors. Texture coordinates can also be specified using normalized texture coordinates. For 2D textures, normalized texture coordinates are values from 0.0 to 1.0 in both x and y directions.

## Function

Metal supports the following function attributes that specify how to use a function: vertex, fragment, and kernel. These function attributes are used at the start of a function, before its return type.

```vertex   void my_vertex_func(…) {…}```
```fragment void my_fragment_func(…) {…}```
```kernel   void my_kernel(…) {…}```

### Graphics Function
    * vertex: Metal executes a vertex function for each vertex in the vertex stream and generates per-vertex output.
    * fragment: Metal executes a fragment function for each fragment in the fragment stream and their associated data and generates per-fragment output.

### Compute function
    * kernel: A compute function called a “kernel” is a data-parallel function that is executed over a 1-, 2-, or 3D grid.

### Host Name Attribute
Starting from Metal 2.2, the ```[[hostname(name)]]``` attribute may be used with vertex, fragment, and kernel functions to override the default name of the function. 
Two distinct functions cannot have the same host name or else the compiler will raise a compile-time error.

```[[host_name("foo")]]  kernel void foo() {} // Metal API name is foo ```
```[[host_name("foo2")]] kernel void foo() {} // Metal API name is foo2```

### Templated Function(s)
Starting from Metal 2.2 it is now possible to define C ++like templates for vertices, fragments, and kernel functions. Since these cannot be called directly in the shader, users must explicitly instantiate the template to force the compiler to emit code for a given specialization.

```template<typename T>```
```kernel void bar(device T *x) { … } // Explicit specialization of `bar` with [T = int]```
```template kernel void bar(device int *);```

## Attribute Qualifier for Arguments and Variables
Attribute Qualifier for function arguments and values are as follows:

    ```[[position]]```:  The type is float4 and indicates coordinates (x, y, z, w).
    ```[[vertex_id]]```: ushort or uint to indicate the vertex index.
    ```[[stage_in]]``` : It is used as an argument of the fragment shader and has fragment data that is the structure used to draw one pixel of the display.
    ```[[buffer(index)]]```:  it specifies the buffer locations for the function arguments. A vertex function can read per-vertex inputs by indexing into a buffer(s) passed as arguments to the vertex function using the vertex and instance IDs.
    ```[[texture(index)]]```: Textures (including texture buffers).
    ```[[sampler(index)]]```: Samples that define how to access texture data.
    ```[[threadpositionin_grid]]```: The position of the thread in the grid.

### Address Space Qualifiers
Address Space Qualifiers is used to specify the memory area for variables and arguments and must be specified for pointer type and reference type arguments and variables.

device and constant can be specified in graphics function.

    * device: read / write
    * constant: read-only

threadgroup and thread can be specified in compute function.

    * ```threadgroup```: The entire thread in the thread group is shared.
    * ```thread```: Can’t be referenced from other threads.

# References

    * Metal Shading Language Specification (https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf)
    * Metal Programming Guide              (https://developer.apple.com/library/archive/documentation/Miscellaneous/Conceptual/MetalProgrammingGuide/Introduction/Introduction.html#//apple_ref/doc/uid/TP40014221-CH1-SW1)
    * Understanding the Metal Shading Language Filename Extension (https://developer.apple.com/documentation/metal/libraries/understanding_the_metal_shading_language_filename_extension)
    * Building a Library with Metal’s Command-Line Tools (https://developer.apple.com/documentation/metal/libraries/building_a_library_with_metal_s_command-line_tools)
    * Using a Render Pipeline to Render Primitives       (https://developer.apple.com/documentation/metal/using_a_render_pipeline_to_render_primitives)
    * Clip coordinates (https://en.wikipedia.org/wiki/Clip_coordinates)


# Appendix A: Metal Compiler Commands (xcrun -sdk macosx metal ...)
    
## Metal (create metallib) commands:
How To: generally create a metallib with xcrun

write a .metal file and pass it to xcrun with the -c flag, name the output (for example):
```xcrun -sdk macosx metal -c MyLibrary.metal -o MyLibrary.air```

take the output AIR file and run it against the compiler, name the output again:
```xcrun -sdk macosx metallib MyLibrary.air   -o MyLibrary.metallib```

## Metal (create dylib/symbol file with metallib)
How To: generate a metallib with a Symbol File

Generate a Metal library’s companion file in a single command by running the Metal compiler with the ```-frecord-sources=flat``` option:
```% xcrun -sdk macosx metal -frecord-sources=flat Shadows.metal PointLights.metal DirectionalLight.metal```

First, compile each Metal source file to a Metal AIR file by passing the ```-c``` option to the compiler:
```% xcrun -sdk macosx metal -c -frecord-sources Shadow.metal```
```% xcrun -sdk macosx metal -c -frecord-sources PointLights.metal```
```% xcrun -sdk macosx metal -c -frecord-sources DirectionalLight.metal```

Next, generate a Metal library by linking the AIR files:
```% xcrun -sdk macosx metal -frecord-sources -o LightsAndShadow.metallib Shadow.air PointLights.air DirectionalLight.air```

Separate the sources from the library and create a symbol companion file by running the ```metal-dsymutil``` command.
```% xcrun -sdk macosx metal-dsymutil -flat -remove-source LightsAndShadow.metallib```

The ```-remove-source``` option modifies the Metal library file by removing its embedded source information. 
At the same time, the command’s -flat option saves that source and other symbol information to a companion file. 

After the command finishes execution, the directory contains both the modified Metal library and its new symbol companion file:
```LightsAndShadow.metallib```
```LightsAndShadow.metallibsym```