ASP.NET Core ETAg middleware

ETagMiddleware.cs

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.WebUtilities;
using Microsoft.Net.Http.Headers;
using System.IO;
using System.Security.Cryptography;
using System.Threading.Tasks;

public class ETagMiddleware
{
    private readonly RequestDelegate _next;

    public ETagMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var response = context.Response;
        var originalStream = response.Body;

        using (var ms = new MemoryStream())
        {
            response.Body = ms;

            await _next(context);

            if (IsEtagSupported(response))
            {
                string checksum = CalculateChecksum(ms);

                response.Headers[HeaderNames.ETag] = checksum;

                if (context.Request.Headers.TryGetValue(HeaderNames.IfNoneMatch, out var etag) && checksum == etag)
                {
                    response.StatusCode = StatusCodes.Status304NotModified;
                    return;
                }
            }

            ms.Position = 0;
            await ms.CopyToAsync(originalStream);
        }
    }

    private static bool IsEtagSupported(HttpResponse response)
    {
        if (response.StatusCode != StatusCodes.Status200OK)
            return false;

        // The 20kb length limit is not based in science. Feel free to change
        if (response.Body.Length > 20 * 1024)
            return false;

        if (response.Headers.ContainsKey(HeaderNames.ETag))
            return false;

        return true;
    }

    private static string CalculateChecksum(MemoryStream ms)
    {
        string checksum = "";

        using (var algo = SHA1.Create())
        {
            ms.Position = 0;
            byte[] bytes = algo.ComputeHash(ms);
            checksum = $"\"{WebEncoders.Base64UrlEncode(bytes)}\"";
        }

        return checksum;
    }
}

public static class ApplicationBuilderExtensions
{
    public static void UseETagger(this IApplicationBuilder app)
    {
        app.UseMiddleware<ETagMiddleware>();
    }
}

Startup.cs

// Add "app.UseETagger();" to "Configure" method in Startup.cs
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UseStaticFiles();
    
    // Add this after static files but before MVC in order to provide ETags to MVC Views and Razor Pages.
    app.UseETagger();

    app.UseMvc(routes =>
    {
        routes.MapRoute(
            name: "default",
            template: "{controller=Home}/{action=Index}/{id?}");
    });
}

@madskristensen Should you set the content-length to 0 if the response is a 304?

if (context.Request.Headers.TryGetValue(HeaderNames.IfNoneMatch, out var etag) && checksum == etag)
{
        response.StatusCode = StatusCodes.Status304NotModified;
        response.ContentLength = 0L;
        return;
}

I had 500 errors being thrown if the reponse body length didn't match the Content-Length.

Would this also work with Files returned using the "File()" method from controller methods?
@madskristensen

It will as long as the File method doesn't close the stream before it reaches the middleware. It's probably not very efficient to serve files through it, since it does a stream copy regardless of response size.

Also, if you are using the UseDeveloperExceptionPage middleware, be sure to add it after the ETagging middleware, as the Developer Exception middleware writes to the stream directly.

Mh this doesn't look like a proper solution for ETag. ETag is meant to remove bandwidth/stress. The issue is, this Middleware needs a completely prepared Response which means exhaustive tasks to fetch data were already made. Or am i getting it wrong?

@Memnarch But how can the Server know if the result he wants to return is already the one the client has cached? The ETag saves you the bandwith to actually transfer the data from the server to the client. The backend therefore saves resources because he can answer more efficiently, even though he had to process the content of the answer beforehand.
If you configured the cache to store and it is not yet expired, there will be no connection made to the server. If it is expired, but the ETag stays the same, the client can refresh the cache with the specified max age of the server and server does not have to actually send the data again.

Small edition to make sure that the response body is empty, so we can save bandwidth

if (context.Request.Headers.TryGetValue(HeaderNames.IfNoneMatch, out var etag) && checksum == etag) {
    response.StatusCode = StatusCodes.Status304NotModified;
    response.Headers[HeaderNames.ContentLength] = "0";
    response.ContentType = null;
    response.Body = null;
    return;
}

Hmm, any ideas how this could be adapted to work for PUT/PATCH? Create a new GET context based of the current one, run that through _next and generate the etag, then return 412 or _next(context)? Would that work?

Thank you, sir! This solution worked for me.
I will try subject it to some rigorous testing runs to see how it performs.

@zuckerthoben - it's the server who sends the very first etag (i.e. version field from optimistic concurrency control mechanism) related to requested resource, so the server has ability to check if the resource has changed in a more effective way than calculating hash from already prepared response.

Hmm, any ideas how this could be adapted to work for PUT/PATCH? Create a new GET context based of the current one, run that through _next and generate the etag, then return 412 or _next(context)? Would that work?

@mcintrye321 Did you find a solution to the If-Match on an update?

Can you include a license? Or could you give us permission to use? Would like to know if we can use/modify this code for our own software

@hedgelot Permission granted to use for anybody, following the Apache 2.0 license

To be able to use another middleware before this one the original stream should be set to response.Body again like this:

            if (IsEtagSupported(response))
            {
                string checksum = CalculateChecksum(ms);

                response.Headers[HeaderNames.ETag] = checksum;

                if (context.Request.Headers.TryGetValue(HeaderNames.IfNoneMatch, out var etag) && checksum == etag)
                {
                    response.StatusCode = StatusCodes.Status304NotModified;
                    // set the original stream again
                    response.Body = originalStream;
                    return;
                }
            }

            ms.Position = 0;
            await ms.CopyToAsync(originalStream);
            // set the original stream again
            response.Body = originalStream;

Because otherwise the stream whould be clsoed as in the context.Response.Body the memory stream remains. And this would cause another middleware to operate on the wrong stream and fail.

@zuckerthoben - it's the server who sends the very first etag (i.e. version field from optimistic concurrency control mechanism) related to requested resource, so the server has ability to check if the resource has changed in a more effective way than calculating hash from already prepared response.

Exactly. Saving the network bandwidth only is a partial solution that has no benefit for REST calls that have heavy computational logic on the server or trigger expensive database queries. A true cache solution on the server-side should be able to calculate the ETAG efficiently, without generating a response.

Do we really have to use the Response Body to calculate the ETag. I would recommend to avoid any logic in the Middleware based on the Response Body, Instead I would calculate the Etag on API or Page controller and pass it to Response and use this middleware to add Header only.

@Memnarch But how can the Server know if the result he wants to return is already the one the client has cached? The ETag saves you the bandwith to actually transfer the data from the server to the client. The backend therefore saves resources because he can answer more efficiently, even though he had to process the content of the answer beforehand. If you configured the cache to store and it is not yet expired, there will be no connection made to the server. If it is expired, but the ETag stays the same, the client can refresh the cache with the specified max age of the server and server does not have to actually send the data again.

Also, if you cache the result on server side, using the ETAG as key, then you also save resources on the server side. If the request is cachable.