Source Generated JSON Serialization using fast path in ASP.NET Core

source-gen-json-aspnetcore.md

Source Generated JSON Serialization using fast path in ASP.NET Core

In this post I explore how ASP.NET Core works together with the source generated JSON serializer. In .NET 8 the streaming JSON serialization can be enabled to use serialization-optimization mode with ASP.NET Core. However, using the default settings does not enable this fast-path. In this post I will explore how to enable the fast-path serialization by exploring the inner workings of JsonTypeInfo.cs. For this post I use .NET 8 with the corresponding ASP.NET Core release.

The .NET 8 runtime comes with three built-in JSON serializer modes:

• Reflection
• Source generation (Metadata-based mode)
• Source generation (Serialization-optimization mode)

Reflection mode is the default one. This mode is used when someone invokes JsonSerializer.(De)SerializeAsync without additional arguments passed. Source Generation (Metadata-based mode) as the name suggests generates source code at compile time that contains the metadata that the Reflection would need to discover during its first invocation. This mode helps to enable JSON serialization in environments where reflection otherwise would not be possible, for example in case of AOT published applications. Source generation (Serialization-optimization mode) also generates code at compile time, but besides the metadata it also uses a generated fast-path method when serializing objects.

From .NET 8 an ASP.NET Core application can leverage the source generated serializers.

Performance Overview

Here I compare the source generating modes to Reflection mode. The Metadata-based mode is typically faster for the first invocation, as it does not have to generate the corresponding metadata for the types. Further invocations with the same types (and options) will not realize performance gains. The Serialization-optimization mode uses a fast-path for serialization, which can be faster for certain types. Based on my measurements the Serialization-optimization mode is faster than the Metadata-based mode by 0-20% depending on the shape of the data.

In an ASP.NET Core application the performance gain on a single request with the fast-path is likely to be outweighed by the network jitter. However, the application throughput will increase as less CPU resources are required per request. This allows more requests to be served compared to the other modes.

Setup

Metadata-based mode

Enabling source generated serialization consists of two steps:

1. Creating a partial context class derived from JsonSerializerContext, that is attributed with types to be source generated.
2. This context type should be set as the TypeInfoResolver or added to the TypeInfoResolverChain.

In an ASP.NET Core WebApi application:

using System.Text.Json.Serialization;

var builder = WebApplication.CreateBuilder(args);
builder.Services.ConfigureHttpJsonOptions(options =>
{
    // Set the TypeInfoResolver 👇
    options.SerializerOptions.TypeInfoResolver = MyContext.Default;
        options.SerializerOptions.Encoder = null;
});

var app = builder.Build();
app.MapGet("/plain", WithPlain);
app.Run();

static MyDto WithPlain() => new MyDto();

public class MyDto { public string Description { get; set; } = "hello"; }

// Derive a partial class from JsonSerializerContext 👇
[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)]
[JsonSerializable(typeof(MyDto))]
public partial class MyContext : JsonSerializerContext { }

Serialization-optimization mode

Enabling the fast-path with Serialization-optimization mode requires a few further options:

1. Most settings of JsonSourceGenerationOptions must be compatible with the serialization options used by ASP.NET Core's serializer options.
2. ASP.NET Core's options.SerializerOptions.Encoder should be set to null (by default SystemTextJsonOutputFormatter adds a JavaScriptEncoder.UnsafeRelaxedJsonEscaping encoder.
3. DefaultBufferSize must be at least twice the size of the serialized data. However, it cannot take an arbitrarily large value as it is pooled by ArrayPool which has an upper 'limit' on array sizes that are reasonable to pool. Another consideration against large buffer sizes is to have the pooled arrays allocated on the Small Object Heap (SOH) instead of the Large Object Heap (LOH).

JsonTypeInfo Internals

When a web application is started, the ApiExplorer of ASP.NET Core creates routes for the endpoints. As part of this process the return types' type info is fetched and passed to the request delegate factory. During this process the JsonSerializerOptions caches JsonTypeInfo. The first time a JsonTypeInfo is created it gets configured. The Configure method of JsonTypeInfo determines if the object is compatible with ASP.NET Core's JsonSerializerOptions. When it is compatible, and the JsonTypeInfo has a serializer handler (a delegate for the generated fast-path code), it sets the CanUseSerializeHandler property to true.

Every time SerializeAsync is invoked the method checks if CanUseSerializeHandlerInStreaming is enabled. When enabled fast-path serialization mode is used. However, by default it is disabled, hence the metadata-based serialization is used. Before SerializeAsync returns the result to its invoker, the above CanUseSerializeHandler property is tested. When it has the value of true, the OnRootLevelAsyncSerializationCompleted called is called. This method compares the size of current serialized response with the default buffer size:

private void OnRootLevelAsyncSerializationCompleted(long serializationSize)
{
    if (_canUseSerializeHandlerInStreamingState != 2)
    {
        if ((ulong)serializationSize > (ulong)(base.Options.DefaultBufferSize / 2))
        {
            _canUseSerializeHandlerInStreamingState = 2;
        }
        else if ((uint)_serializationCount < 10u && Interlocked.Increment(ref _serializationCount) == 10)
        {
            Interlocked.CompareExchange(ref _canUseSerializeHandlerInStreamingState, 1, 0);
        }
    }
}

Any time the response is larger than half of the buffer, value 2 is set to _canUseSerializeHandlerInStreamingState field. This represents an end state, which disables future fast-path serialization as CanUseSerializeHandlerInStreaming will return false. Otherwise, the method increments a counter. When a counter reaches 10, _canUseSerializeHandlerInStreamingState field is set to the value of 1. In this state CanUseSerializeHandlerInStreaming returns true, that makes SerializeAsync to leverage the fast-path mode. In this state debugger breakpoints set in the generated code will be hit.

During serialization both the fast-path mode and the metadata mode rent an array in the size defined by the default buffer size. This array is rented from ArrayPool<byte>.Shared. A key difference is that the metadata mode regularly flushes the buffer while the fast-path mode writes all data to the buffer before it is flushed to the response stream.

The final detail I glossed over is how JsonTypeInfo's compatibility with ASP.NET Core's JsonSerializerOptions is determined. This is decided during the configuration of the JsonTypeInfo. When compatibility is determined, JsonTypeInfo reads its Options.CanUseFastPathSerializationLogic property. This property is not debugger browsable, because it caches its value for the rest of the lifetime of the object. Having it evaluated too early (for example by being displayed in the debugger) might result in an invalid state.

The CanUseFastPathSerializationLogic property itself determines if the Options is compatible with all the chained JsonContexts, by invoking their IsCompatibleWithOptions method and passing itself as an argument. Each context then compares its corresponding settings (configured with [JsonSourceGenerationOptions] attribute) with ASP.NET Core's SerializerOptions as shown here:

bool IBuiltInJsonTypeInfoResolver.IsCompatibleWithOptions(JsonSerializerOptions options)
{
    JsonSerializerOptions generatedSerializerOptions = GeneratedSerializerOptions;
    if (generatedSerializerOptions != null 
        && options.Converters.Count == 0 
        && options.Encoder == null 
        && !JsonHelpers.RequiresSpecialNumberHandlingOnWrite(options.NumberHandling) 
        && options.ReferenceHandlingStrategy == ReferenceHandlingStrategy.None 
        && !options.IgnoreNullValues 
        && options.DefaultIgnoreCondition == generatedSerializerOptions.DefaultIgnoreCondition 
        && options.IgnoreReadOnlyFields == generatedSerializerOptions.IgnoreReadOnlyFields
        && options.IgnoreReadOnlyProperties == generatedSerializerOptions.IgnoreReadOnlyProperties 
        && options.IncludeFields == generatedSerializerOptions.IncludeFields 
        && options.PropertyNamingPolicy == generatedSerializerOptions.PropertyNamingPolicy)
    {
        return options.DictionaryKeyPolicy == null;
    }
    return false;
}

The reason why the Encoder property must be null is because otherwise the compatibility would evaluate to false, hence the source generated serializers do not (yet) support encoders. On the other hand, ASP.NET Core sets the JavaScriptEncoder.UnsafeRelaxedJsonEscaping encoder when SystemTextJsonOutputFormatter is added to the output formatters. This encoder would not encode some of the HTML-sensitive characters.

Appendix

Performance Measurements Benchmarks

Legend:

• MyDto is a type with 2 properties.
• MyDtoLarge is a type with 24 properties, but its serialized size matches MyDto.
• DefaultBuffeSize is set to 2MB with Buffer suffix, allowing fast-path serialization. Otherwise DefaultBuffeSize is left as default at 16384 bytes.
• Reflection mode is used with NoSG suffix. Otherwise, source generation mode is used.

Serializing a single 864 KB DTO:

Method	Mean	Error	StdDev	Allocated
MyDto	581.0 us	4.85 us	4.05 us	465 B
MyDtoLarge	610.2 us	8.30 us	7.76 us	465 B
MyDtoBuffer	535.1 us	3.77 us	3.34 us	33 B
MyDtoLargeBuffer	532.9 us	5.25 us	4.91 us	33 B
MyDtoNoSG	590.6 us	5.27 us	4.68 us	465 B
MyDtoLargeNoSG	586.1 us	5.74 us	5.37 us	465 B

In this case the additonal performance is only gained in the Buffer cases, where the DefaultBufferSize is large enough to enable the fast-path mode. Otherwise MyDto and MyDtoLarge performs similar to the non-source generated results.

Serializing a list of these DTOs (1 KB each), but in total larger to 16384 bytes:

Method	Mean	Error	StdDev	Allocated
MyDto	758.4 us	6.50 us	6.08 us	482 B
MyDtoLarge	1,217.2 us	9.90 us	9.26 us	500 B
MyDtoBuffer	618.5 us	4.28 us	3.80 us	2082 B
MyDtoLargeBuffer	920.3 us	16.23 us	16.67 us	2082 B
MyDtoNoSG	752.7 us	6.81 us	6.37 us	482 B
MyDtoLargeNoSG	1,223.0 us	11.73 us	10.97 us	500 B

This is a similar case as above considering the buffer size. Also note, that having significantly more properties to serialize (while in total containing the same amount of data) slows down serialization. This is show by the differences between the Large and regular DTO's mean execution time.

Single entity serialization (1 KB):

Method	Mean	Error	StdDev	Gen0	Allocated
MyDto	687.2 ns	5.44 ns	5.09 ns	0.0048	32 B
MyDtoLarge	984.3 ns	16.35 ns	15.29 ns	0.0038	32 B
MyDtoBuffer	682.0 ns	9.27 ns	7.74 ns	0.0048	32 B
MyDtoLargeBuffer	953.7 ns	3.07 ns	2.56 ns	0.0048	32 B
MyDtoNoSG	796.7 ns	5.05 ns	4.48 ns	0.0734	464 B
MyDtoLargeNoSG	1,286.2 ns	11.79 ns	10.45 ns	0.0725	464 B

Here the Buffer cases do not gain further performance as all source-generated results (MyDto, MyDtoLarge, MyDtoBuffer, MyDtoLargeBuffer) fit in the default buffer using the fast-path mode. However, we can see significant gains over the non-source generated results (MyDtoNoSG, MyDtoLargeNoSG).