Skip to content

Instantly share code, notes, and snippets.

@rzikm
Last active November 6, 2023 14:33
Show Gist options
  • Save rzikm/807c521b5bc4d8e84c4438f82f2fbf97 to your computer and use it in GitHub Desktop.
Save rzikm/807c521b5bc4d8e84c4438f82f2fbf97 to your computer and use it in GitHub Desktop.
QUIC Additional Options Proposal

Background and motivation

msquic has a lot of connection options and S.N.Q exposes just a few

List of msquic settings - https://github.com/microsoft/msquic/blob/main/docs/Settings.md

Settings I think should be added:

  • Disconnect Timeout - Importance discovered as part of dotnet/runtime#71927. People may want to be more or less aggressive with closing unresponsive streams
  • Keep Alive Interval - Kestrel HTTP/2 layer has a similar option. Should expose it for feature parity.
  • Flow Control Window - Kestrel HTTP/2 layer has a similar option. Should expose it for feature parity. Useful when sending large amounts of data over connection with latency. Allows you to trade potentially higher memory usage to avoid back pressure throttling the stream/connection
  • Stream Receive Window - Kestrel HTTP/2 layer has a similar option. Same reason as connection flow control window.
    • ncl: QUIC exposes limits for individual stream types, so we will expose separate properties for those
  • DatagramReceiveEnabled - Maybe. I think this will be needed for WebTransport datagram support. Does QuicStream have a story for supporting datagram in the future?
  • Handshake Idle Timeout - Kestrel has an option to supply a handshake timeout. Would be nice to flow that through to QUIC.

I don't think any of these are necessary for .NET 7.

API Proposal (edited by @rzikm)

See also discussion about naming below the proposal, we are open for suggestions for better names. Default values were copied from current MsQuic behavior.

namespace System.Net.Quic
{
    public abstract class QuicConnectionOptions
    {
        /// <summary>
        /// The interval at which keep alive packets are sent on the connection.
        /// Default <see cref="TimeSpan.Zero"/>, or <see cref="Timeout.InfiniteTimeSpan"> means no keep alive.
        /// </summary>
+       public TimeSpan KeepAliveInterval { get; set; } = TimeSpan.Zero;

        /// <summary>
        /// The initial flow-control window size for the connection.
        /// </summary>
+       public int InitialConnectionReceiveWindowSize { get; set; } = 16 * 1024 * 1024;

        /// <summary>
        /// The initial flow-control window size for locally initiated bidirectional streams.
        /// </summary>
+       public int InitialLocallyInitiatedBidirectionalStreamReceiveWindowSize { get; set; } = 64 * 1024;

        /// <summary>
        /// The initial flow-control window size for remotely initiated bidirectional streams.
        /// </summary>
+       public int InitialRemotelyInitiatedBidirectionalStreamReceiveWindowSize { get; set; } = 64 * 1024;

        /// <summary>
        /// The initial flow-control window size for (remotely initiated) unidirectional streams.
        /// </summary>
+       public int InitialUnidirectionalStreamReceiveWindowSize { get; set; } = 64 * 1024;

        /// <summary>
        /// The upper bound on time when the handshake must complete. If the handshake does not
        /// complete in this time, the connection is aborted.
        /// </summary>
+       public TimeSpan HandshakeTimeout { get; set; } = TimeSpan.FromSeconds(10);
    }
}

Properties InitialConnectionReceiveWindowSize, InitialLocallyInitiatedBidirectionalStreamReceiveWindowSize, InitialRemotelyInitiatedBidirectionalStreamReceiveWindowSize, and InitialUnidirectionalStreamReceiveWindowSize have rather long names but we did come up with a better alternative. They map directly to QUIC transport parameters initial_max_data, initial_max_stream_data_bidi_local, initial_max_stream_data_bidi_remote, and initial_max_stream_data_uni respectively. These transport parameters limit the amount of data the peer can send on the connection as a whole and on streams of individual types to prevent the peer from overwhelming the receiver with data. Thus, the Initial*ReceiveWindowSize part of the name seems appropriate. The rest of the name identifies the type of stream, where "Locally initiated bidirectional stream" and similar are terms used in the QUIC RFC and we should use them to avoid confusion.

For comparison, MsQuic uses names such as StreamRecvWindowBidiLocalDefault. And will expand the receive window if appropriate.

Parameter definition from RFC

initial_max_data (0x04)

The initial maximum data parameter is an integer value that contains the initial value for the maximum amount of data that can be sent on the connection. This is equivalent to sending a MAX_DATA (Section 19.9) for the connection immediately after completing the handshake.

initial_max_stream_data_bidi_local (0x05)

This parameter is an integer value specifying the initial flow control limit for locally initiated bidirectional streams. This limit applies to newly created bidirectional streams opened by the endpoint that sends the transport parameter. In client transport parameters, this applies to streams with an identifier with the least significant two bits set to 0x00; in server transport parameters, this applies to streams with the least significant two bits set to 0x01.

initial_max_stream_data_bidi_remote (0x06)

This parameter is an integer value specifying the initial flow control limit for peer-initiated bidirectional streams. This limit applies to newly created bidirectional streams opened by the endpoint that receives the transport parameter. In client transport parameters, this applies to streams with an identifier with the least significant two bits set to 0x01; in server transport parameters, this applies to streams with the least significant two bits set to 0x00.

initial_max_stream_data_uni (0x07)

This parameter is an integer value specifying the initial flow control limit for unidirectional streams. This limit applies to newly created unidirectional streams opened by the endpoint that receives the transport parameter. In client transport parameters, this applies to streams with an identifier with the least significant two bits set to 0x03; in server transport parameters, this applies to streams with the least significant two bits set to 0x02.

API Usage

var quicListenerOptions = new QuicListenerOptions
{
    ApplicationProtocols = _tlsConnectionCallbackOptions.ApplicationProtocols,
    ListenEndPoint = listenEndPoint,
    ListenBacklog = options.Backlog,
    ConnectionOptionsCallback = async (connection, helloInfo, cancellationToken) =>
    {
        var connectionOptions = new QuicServerConnectionOptions
        {
            // ... set various properties for server connection ...
        };
        return connectionOptions;
    }
};

Alternative Designs

To avoid long property names. We can introduce a wrapper type for the initial connection window sizes. However, the wrapper type must be a class, otherwise options.InitialReceiveWindowSizes.ConnectionTotal = x has no effect. We also don't see another use for the type.

namespace System.Net.Quic
{
+   public class QuicReceiveWindowSizes
    {
        /// <summary>
        /// The flow-control window size for the entire connection.
        /// </summary>
+       public int ConnectionTotal { get; set; } = 16 * 1024 * 1024;

        /// <summary>
        /// The flow-control window size for locally initiated bidirectional streams.
        /// </summary>
+       public int LocallyInitiatedBidirectionalStreams { get; set; } = 64 * 1024;

        /// <summary>
        /// The flow-control window size for remotely initiated bidirectional streams.
        /// </summary>
+       public int RemotelyInitiatedBidirectionalStreams { get; set; } = 64 * 1024;

        /// <summary>
        /// The flow-control window size for (remotely initiated) unidirectional streams.
        /// </summary>
+       public int UnidirectionalStreams { get; set; } = 64 * 1024;

    }

    public abstract class QuicConnectionOptions
    {
        // unchanged
+       public TimeSpan KeepAliveInterval { get; set; } = TimeSpan.Zero;
+       public TimeSpan HandshakeTimeout { get; set; } = TimeSpan.FromSeconds(10);

        /// <summary>
        /// The initial connection window sizes.
        /// Default <see cref="TimeSpan.Zero"/>, or <see cref="Timeout.InfiniteTimeSpan"> means no keep alive.
        /// </summary>
        // note: actuall implementation can create the default instance lazily to avoid alloc if user only calls set
+       public QuicReceiveWindowSizes InitialReceiveWindowSizes { get; set; } = new QuicReceiveWindowSizes();
    }
}

Risks

The options above are commonly exposed by other QUIC implementations and none of them are too specific as to vendor lock System.Net.Quic to MsQuic only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment