This proposal outlines the initial design for a new real-time Non-Photorealistic Render Engine for Blender, with an initial focus on the features needed for cel shaded animation.
It follows the node-based workflow for materials already established in Cycles and EEVEE and shares most nodes with them.
The main exceptions are Shader nodes, supporting only those needed for interoperability and compositing between engines (Mix Shader, Emission, Transparent BSDF, and Holdout).
Additionally, it provides new nodes for NPR Shading and Line Rendering.
Therefore, the intended user workflow is to rely on compositing for most of the features already supported by Cycles, EEVEE, or the compositor itself.
Unsupported nodes follow the same behavior already present for incompatible nodes between EEVEE and Cycles: provide a fallback when possible (eg. Toon Shader in EEVEE) and a constant value when not (eg. ShaderToRGB in Cycles).
While the proposal of a new NPR Engine was requested by the Blender Foundation, this proposal is not by any means final or even guaranteed to happen in any form.
All the rendered images are actual screenshots taken using the Malt render engine, but not all the mentioned features are available yet in the public releases.
The engine includes a real-time line rendering system integrated into the Material Nodes. This allows controlling the line individually for each material and drive it with any shading feature (transparency, lighting, textures…) using regular Material Nodes.
Since line styling doesn’t rely on computing stroke connections, it can provide much better performance than Freestyle or LineArt.
Line detection is performed in two different fashions:
-
Image-based detection is almost independent of the scene triangle count, so it can be used while working on scenes with any amount of geometry with minimal impact on render performance.
-
Geometry-based detection is slower (although still real-time for low million triangle counts), but offers higher quality detection (independent of resolution and camera distance), visually similar to Freestyle and LineArt.
Line Width driven by procedural noise. Line Color and Color Marker driven by scene lighting.
- Units [Screen Percentage | Pixel]: The units used for the Width socket in the Line Output node.
- Scale (Float): Allows scaling the line width globally (Useful for tweaking and handling multiple render resolutions).
- Max Width (Float): The max line width the engine will render, higher values get clamped to this one.
- Color (RGBA): Color of the line.
- Width (Float): The line width in screen % or pixel units (selected in Render Properties).
- Color Marker (RGBA): Custom mask for detecting lines at Color Marker boundaries.
Image Based:
- Is Object Silhouette (Float): Acts like a boolean. 1 at object boundaries, 0 otherwise.
- Is Color Marker Silhouette (RGBA): Acts like a boolean for each Color Marker channel. 1 at Color Marker boundaries, 0 otherwise.
- Depth Delta (Float): The max depth delta between the rendered and nearby pixels. (In World units)
- Normal Delta (Float): The max angle between the rendered and nearby pixels. (Normalized to the 0-1 range)
Geometry Based:
- Is Freestyle Edge (Float): Acts like a boolean. 1 at Freestyle edges, 0 otherwise.
- Is Contour Edge (Float): Acts like a boolean. 1 at contour edges, 0 otherwise. A contour edge separates front and back faces (Same behavior as Line Art contours)
- Edge Crease (Float): The angle between the rendered edge faces. (Normalized to the 0-1 range)
The Line Detection node is used in combination with the Max, Multiply, Greater Than, Map Range and Curve nodes.
💡 If Material nodes supported sockets with multiple inputs (like Geometry nodes), it could make easier to combine multiple line types (ie. combining more than 2 line types with a single Max node).
Additionally, using boolean sockets could make more sense for some of the outputs.
| Is Object Silhouette | Is Color Marker Silhouette |
|---|---|
 |
 |
| Depth Delta | Normal Delta |
|---|---|
 |
 |
| Is Contour Edge | Edge Crease |
|---|---|
 |
 |
The Line Width nodes are user-friendlier wrappers over the Line Detection functionality.
While they're not as powerful for complex line art styles, they offer a convenient alternative for common use cases.
There are two Line Width nodes based on the type of detections they perform:
- Line Width (Image Based) (Faster)
- Line Width (Geometry Based) (Better Quality)
Both nodes support Silhouette and Freestyle Edge detection despite their name, since those always offer the best quality and performance.
- Object Silhouette
- Color Marker Silhouette
- Freestyle Edge
- Depth Delta
- Normal Delta
- Depth Delta Threshold (Float): Depth Delta values greater than this one will be considered as a line.
- Normal Delta Threshold (Float): Normal Delta values greater than this one will be considered as a line.
- Width (Float): The final width of the detected line.
- Line Width (Float)
- Object Silhouette
- Color Marker Silhouette
- Freestyle Edge
- Depth Delta
- Normal Delta
- Crease Edge Threshold (Float): Crease Edge values greater than this one will be considered as a line.
- Width (Float): The final width of the detected line.
- Line Width (Float)
Combine multiple line types and sets the line width to 4.
Using the Line Detection node:
Using the Line Width node:
If it's an Object Silhouette, a Contour Edge, or the Edge Crease is greater than 0.5, draws a 4 units wide line.
Drive the line width based on depth detection.
Line Detection equivalent setup
Achieve temporally-stable organic lines with procedural noise.
Line Detection equivalent setup
Draw Lines at shadow edges.
Line Detection equivalent setup
Sets the Color Marker based on cel shaded diffuse lighting, so Color Marker Silhouettes are detected at shadow edges.
While the Color Marker socket in the Line Output node creates a counter-intuitive node flow (ie. an output node modifies the behavior of the Color Marker socket in Line Detection), this is not too different to how the Displacement socket in the Material Output node can change the surface position and normal.
💡 Alternatively, a Color Marker input could be added to the Line Detection/Width nodes instead, but only one Line Detection/Width node per node tree could use it. This could be enforced by hiding or disabling the Color Marker input in any other Line Detection node once one is in use.
Set different widths and colors for each line type.
Since the line is part of the material nodes, each material can set its own style.
The engine introduces new nodes based on the most common NPR techniques already used in EEVEE, while avoiding the ShaderToRGB limitations and providing support for new features.
Shading nodes have a built-in color ramp that is applied individually for each light source contribution, so they work consistently with arbitrary combinations of light colors and intensities.
The World Shader lighting is exposed as a separate node, so the user can choose how to integrate it with the rest of the lighting and avoid the banding issues caused by mapping it to a color ramp.
In addition, since the use of specific lights for different elements of the scene is usually needed (especially in Cel Shaded styles), the engine provides full support for Light Linking and per Material and Shading Node control over shadows (including disabling self-shadows).
- Normal
- Color Ramp : Drives the diffuse reflectance.
- Ramp Offset (Float): Offsets the ramp sampling point.
- Max Contribution [Bool]: Use only the max light contribution instead of adding them.
- Shadows [Dropdown]: [Inherit from Material / Enable Shadows / Disable Self-Shadows / Disable Shadows]
- Light Group: Overrides the Light Group that affects this shading node. Has the same UI as the materials. Inherits from Material if empty.
- Result (Color)
| Linear Color Ramp | Constant Color Ramp |
|---|---|
 |
 |
| Ramp Offset | Ramp Offset Input |
|---|---|
 |
 |
| Max Contribution Disabled | Max Contribution Enabled |
|---|---|
 |
 |
| Enable Shadows | Disable Self-Shadows | Disable Shadows |
|---|---|---|
 |
 |
 |
Specular only:
- Roughness
Common:
- Normal
- Color Ramp : Drives the specular reflectance.
- Ramp Offset (Float): Offsets the ramp sampling point.
- Max Contribution [Bool]: Use only the max light contribution instead of adding them.
- Shadows [Dropdown]: [Inherit from Material / Enable Shadows / Disable Self-Shadows / Disable Shadows]
- Light Group: Override the Light Group that affects this shading node. Has the same UI as the materials. Inherits from Material if empty.
- Result (Color)
Light contribution from world shader.
- Normal
- Result (Color)
Reflections from world shader.
- Normal
- Roughness
- Result (Color)
Diffuse cel shading offset by a halftone pattern.
Diffuse cel shading with world lighting contribution. The World lighting is sampled at a fixed normal to keep the flat look.
World lighting scaled by ambient occlusion, combined with diffuse and specular shading driven by color ramps.
All the shading is offset by a custom painterly mask (In this case a combination of Voronoi and fractal noise).
(The environment map Kuwahara filter has been applied in Krita)
💡 It may be helpful to support Color Ramps as a socket type, so multiple nodes can be driven by the same ramp.
Although the NPR Shading Nodes cover the most common use cases, one of the key characteristics of NPR is the diversity of styles and the development of project-specific techniques. These techniques often have inherent limitations and edge cases that make them challenging to integrate as built-in nodes in Blender, but they are still important to support. To allow more advanced techniques and enable community-driven experimentation, the engine adds GLSL support to the Script Node, along with the API needed to implement custom shading models.
- OSL (Path): The OSL source file used for Cycles.
- GLSL (Path): The new input for GLSL source file.
- Function [Dropdown]: Select one of the functions from the source file. (Allows to easily store and distribute user libraries)
The generated sockets and functions list are the superset from both source files, regardless of the engine currently being used.
(?) This is just a basic sketch to express the intent of the API.
vec3 get_position();
vec3 get_normal();
vec3 get_true_normal();
vec3 get_tangent(uint index); //(?)
vec3 get_bitangent(uint index); //(?)
vec2 get_uv(uint index);
vec2 get_vertex_color(uint index);
mat4 get_object_matrix();
mat4 get_camera_matrix();
mat4 get_projection_matrix();
vec2 get_resolution();
vec2 get_screen_uv();
struct Light
{
int type;
vec3 position;
float radius;
vec3 direction;
float spot_angle;
float spot_blend;
uint light_group;
};
uint get_lights_count();
Light get_light(uint index);
bool is_light_group_active(uint light_group);
struct LitSurface
{
vec3 light_color;
vec3 shadow_multiply;
vec3 N;// Surface normal
vec3 L;// Surface to light direction (normalized)
vec3 V;// Surface to camera (view) direction (normalized)
vec3 R;// -L reflected on N
vec3 H;// Halfway vector
float NoL;// Dot product between N and L
};
LitSurface compute_lit_surface(uint light_index, vec3 position, vec3 normal, bool shadows, bool self_shadows);
vec3 sample_world_diffuse(vec3 normal);
vec3 sample_world_reflections(vec3 normal, float roughness);
float sample_screen_depth(vec2 uv, bool nearest_interpolation);
vec3 sample_screen_normal(vec2 uv, bool nearest_interpolation);
vec4/*uint?*/ sample_screen_ID(vec2 uv);
vec4 sample_screen_custom_ID(vec2 uv);-
Color Ramp : Drives the light attenuation, sampled in a linear form. (Optional)(Spot and Point lights only)
-
Normalized Power : Alternative way to set up light power so final color results are more predictable.
For example, for Sun Lights, (Normalized Power = 1.0) == (Power = PI).
Power and Normalized Power are synced, like camera FOV/mm or rotation quaternion/euler.
(?) There's probably a much better naming for this.
- Light Links [List]: For each added light group:
- Name: The Light Group name.
- Receive Light [Bool]
- Cast Shadow [Bool]
| Object | Receive Light (A) | Cast Shadow (A) | Receive Light (B) | Cast Shadow (B) |
|---|---|---|---|---|
| Plane | ✔️ | |||
| Cubes | ✔️ | ✔️ | ✔️ | |
| Shader Ball | ✔️ | ✔️ | ||
| Monkey | ✔️ | ✔️ |
- Shadows [Dropdown]: [Enable Shadows / Disable Self-Shadows / Disable Shadows]
The NPR engine is a real-time GPU rasterizer implemented as a new DRW_engine.
It follows a Forward render path split into three geometry passes:
- Pre Pass. Renders Depth, Normal, IDs and motion vectors.
- Edge Pass. Renders Freestyle Edges, Contours and Crease angle data. (Rasterized in line mode)
- Main Pass. Renders the Surface Final Color, the Line Color and Line Width.
Then the line rendering is expanded to the intended width.
The main advantages of a Forward path are allowing custom shading models, full support of ShaderToRGB-like effects, and fine control of line stylization.
graph LR
%%PS[[Previous Sample]]
%%EC[[Environment Cubemap & Probes]]
classDef Graph color:#fff, stroke-width:5px;
class PS,EC Graph;
SP((Shadow Pass))
PP((Pre Pass))
AO((AO Pass))
EP((Edge Pass))
MP((Main Pass))
LC((Line Compositing))
%%POST((Post Process))
classDef GeoPass fill:#d43c00, color:#fff, stroke-width:4px, stroke:#fff;
class SP,PP,EP,MP GeoPass;
classDef ScreenPass fill:#0068de, color:#fff, stroke-width:4px, stroke:#fff;
class AO,LC,POST ScreenPass;
SM(Shadow Maps)
R_D(Depth)
R_N(Normal)
R_OID(Object IDs)
R_CID(Color Marker)
R_E(Edge Data)
R_AO(AO)
R_MV(Motion Vectors)
R_C(Surface Color)
R_LC(Line Color)
R_LW(Line Width)
F_MV(Motion Vectors)
F_D(Depth)
F_C(Color)
classDef Texture fill:#7f00d4, color:#fff, stroke-width:2px, stroke:#000;
class SM,R_D,R_N,R_OID,R_CID,R_E,R_AO,R_MV,R_C,R_LC,R_LW,F_MV,F_D,F_C, Texture;
%%PS & SM & EC --> PP & MP
%%PS & EC --> SP --> SM
SP --> SM --> PP & MP
PP --> R_D & R_N & R_OID & R_CID & R_MV
R_D --> EP --> R_E --> MP
R_D --> AO --> R_AO --> MP
R_D & R_N & R_OID & R_CID --> MP
MP --> R_C & R_LC & R_LW
R_D & R_MV & R_C & R_LC & R_LW --> LC
LC --> F_C & F_D & F_MV %%--> POST
Alpha Blended Transparency can be supported by repeating the same path for transparent objects and using Weighted Order Independent Transparency, which means screen space effects will only work correctly with the first transparency layer. Ideally, since this engine can't use Cycles as a fallback for final rendering, some form of optional Per Pixel Linked Lists or Depth Peeling support could be provided for the Pre Pass.
There are 2 main types of line detection techniques supported:
These are screen-space filters that rely on the Pre Pass textures for line detection. Their main advantage is that their performance is triangle-count independent. They also allow detection based on shading features like alpha-cut transparency and arbitrary shader values (Color Marker).
A implementation of these features can be found in the Malt render engine: https://github.com/bnpr/Malt/blob/186d5e4c78bc419663d0e0f6fb8c82f36d4aa797/Malt/Shaders/Filters/Line.glsl#L71
In the case of Depth and Normal based detection, their main disadvantage is the detection aliasing that can happen at grazing angles.
This can be mitigated by supersampling a separate line "weight" texture and discarding fragments where the weight is lower than a certain threshold.
This method provides more limited but higher quality results, at a lower (but still real-time) performance.
This type of line detection relies on rasterizing edges in line mode.
It requires a buffer with per-edge data, so there's extra processing needed at mesh loading.
struct EdgeData
{
bool is_freestyle_edge;
vec3 connected_face_a_normal;
vec3 connected_face_b_normal;
}These edges are rendered to a screen-space target that stores the crease angle and bit masks for is_freestyle_edge and is_edge_contour.
The Main Pass always outputs 1 pixel wide lines, along their intended width stored in a screen texture. These lines are later expanded in a screen-space shader along their appropriate depth and motion vectors, based on the pixel they expand from.
Arbitrary wide lines can be supported with a Jump Flood algorithm at the cost of losing multiple transparent lines and correct depth compositing. Alternatively, it can be used as an optimization to compute the pixel radius that needs to be checked.
Lines have the same limitation as any screen-space technique: occluded pixels don't generate lines, so very wide lines can cause visible pop-in when disoccluded.
Unlike in ShaderToRGB workflows, the color ramp is mapped for each light before multiplying by light color/power.
ShaderToRGB
foreach light
{
result += lambert * light_color * light_power;
}
result = remap(color_ramp, result);
NPR Shading
foreach light
{
result += remap(color_ramp, lambert) * light_color * light_power;
}
This allows mapping lights with arbitrary colors and powers while avoiding clamping or banding issues.
For supporting self-shadows, object ID shadowmaps are needed in addition to regular depth shadowmaps, To check if a shadow is a self-shadow the renderer simply checks if the ID matches.
vec3 npr_diffuse(
vec3 position, vec3 normal,
sampler1D color_ramp, float ramp_offset, bool max_contribution,
uint light_group, bool receive_shadows, bool receive_self_shadows)
{
vec3 result = vec3(0);
for(int i = 0; i < get_lights_count(); i++)
{
Light L = get_light(i);
if(L.light_group != light_group)
{
continue;
}
LitSurface LS = compute_lit_surface(i, position, normal, receive_shadows, receive_self_shadows);
if(LS.NoL <= 0)
{
continue;
}
float lambert = clamp(LS.NoL + ramp_offset, 0, 1);
vec3 light_contribution = texture(color_ramp, lambert).rgb * LS.light_color * LS.shadow_multiply;
if(max_contribution)
{
result = max(result, light_contribution);
}
else
{
result += light_contribution;
}
}
return result;
}