Iris Go vs .NET Core Kestrel in terms of HTTP performance

iris-vs-kestrel.md

In Iris Go vs .NET Core Kestrel in terms of HTTP performance, an article by @kataras, he explains how his Iris framework beats AspNetCore and also tells his story on how some of the DotNet community members insulted him because of technological disagreements. In some cases, he is right and I am sure we can have a more constructive discourse. I appreciate his work and the technology he developed.

I tried his code and yes, Iris outperforms Kestrel when it returns a short constant string (value). In real world scenarios, we don’t do this usually. We do so many different things to prepare the response. To simulate a real-world example and a fair benchmark, I modified his example to return 1KB randomly generated string as response. On my machine (Corei7-7700K) Kestrel does the job 10+ times faster.

BTW, this result is more about runtime, memory management and compiler performance. Iris is still a lightweight framework comparing to AspNetCore and may perform better in some cases.

AspNetCore benchmark in C# (Startup.cs)

Bombarding http://localhost:5000/api/values/5 with 500000 requests using 125 connections
 500000 / 500000 [=============================================================================================================] 100.00% 5s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     99891.00   13123.35     111093
  Latency        1.25ms     1.00ms    39.00ms
  HTTP codes:
    1xx - 0, 2xx - 500000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   110.60MB/s

Iris benchmark in Go (main.go)

Bombarding http://localhost:5000/api/values/5 with 500000 requests using 125 connections
 500000 / 500000 [===========================================================================================================] 100.00% 1m6s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      7501.00     199.76       9393
  Latency       16.66ms     3.09ms    55.00ms
  HTTP codes:
    1xx - 0, 2xx - 500000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     8.31MB/s

main.go

package main

import (
  "github.com/kataras/iris"
  "github.com/kataras/iris/context"
  //"strings"
  "math/rand"
)

const letterBytes = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 "

func randomString(n int) string {
  b := make([]byte, n)
  for i := range b {
    b[i] = letterBytes[rand.Intn(len(letterBytes))]
  }
  return string(b)
}

func main() {
  app := iris.New()

  //txt := strings.Repeat("g", 10000)
  app.Get("/api/values/{id}", func(ctx context.Context) {
    txt := randomString(1024)
    ctx.WriteString(txt)
  })

  app.Run(iris.Addr(":5000"))
}

Startup.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.AspNetCore.Routing;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore;

namespace netcore
{
    public class Startup
    {
        public static void Main(string[] args)
        {
            WebHost.CreateDefaultBuilder(args)
            .UseStartup<Startup>()
            .Build()
            .Run();
        }

        public Startup()
        {
        }

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddRouting();
        }

        const string chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 ";

        static string RandomString(int length)
        {
            var random = new Random(DateTime.Now.Millisecond);
            var bytes = new Char[length];
            for (var i = 0; i < length; i++)
            {
                bytes[i] = chars[random.Next(chars.Length)];
            }
            return new string(bytes);
        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            var routeBuilder = new RouteBuilder(app);
            //var txt = new string('n', 10000);
            routeBuilder.MapGet("api/values/{id?}", context =>
            {
                var txt = RandomString(1024);
                return context.Response.WriteAsync(txt);
            });
            var routes = routeBuilder.Build();
            app.UseRouter(routes);
        }
    }
}

Definitely late to the game, but it seems like the Go example could benefit from having randomString written like:

func randomString(n int) string {
	b := make([]byte, n)
	r := rand.New(rand.NewSource(time.Now().Unix()))
	l := len(letterBytes)
	for i := range b {
		b[i] = letterBytes[r.Intn(l)]
	}
	return string(b)
}

This moves the performance in-line with the kestrel implementation. That being said, I'm very impressed with Kestrels performance, glad to see the asp.net team giving Go a run for it's money.

my result of your proposed code in updated Ubuntu 16.04 within virtual machine with 2-core processor and 1GB RAM with latest C# ASP.NET Core and Go Iris:
ASP.NET Core 2.1:
1000000 / 1000000 [=============================================] 100.00% 2m18s
Done!
Statistics Avg Stdev Max
Reqs/sec 7220.33 3255.30 20041.65
Latency 17.31ms 4.86ms 210.54ms

Iris 10.6:
1000000 / 1000000 [=============================================] 100.00% 1m44s
Done!
Statistics Avg Stdev Max
Reqs/sec 9550.23 1626.76 22610.84
Latency 13.10ms 8.00ms 184.01ms

How do you reach that throughput?!

I'm in a research about Go and .NET Core so I want the truth.
Your code make result very depend to generating random string. So for make it more real and fair, you can use better approach to creating a random string in the Go lang here
With this change new Go result is:
Bombarding http://localhost:5000/api/values/5 with 1000000 request(s) using 125 connection(s)
1000000 / 1000000 [=====================================================================] 100.00% 48s
Done!
Statistics Avg Stdev Max
Reqs/sec 20828.76 5129.75 44562.62
Latency 6.01ms 4.55ms 171.97ms