This document is a proposal for leveraging the learnings from the small-single-exe working group in ASP.NET Core in an immediately actionable way. All of these changes are aimed at improving one or more of the metrics that we're tracking for the single-effort without bad scenario takebacks.
There's still some design and scenario work to do here for ASP.NET Core - these aren't specs - the goal of this document to capture how we can apply learnings from this effort and accounting for what we stand to gain.
We've completed the prototyping exercise, our primary results:
Size Goal: 26mb (achieved goal of 30mb, baseline 92mb)
Throughput: no significant delta (achieved goal)
Startup: 256ms (not achieved goal of 100ms, baseline 357ms)
Working Set (bonus): 387mb (measured under load, baseline 400mb)
These data represent the No MVC scenario in our performance dashboard. This scenario most closely matches what's proposed here.
The summary of work we've identified to productionize the prototype include:
- Single-exe runtime host
- Diagnostics changes to work with new runtime host
- Removal of native runtime components such as the DAC and DBI
- Removal of assemblies from CoreFx's manual roots
- Additional investigations and trimming of CoreFx
- Redesign of certain ASP.NET features to allow removal of more code
- Scenario pivots in ASP.NET Core (route-to-code instead of MVC, removing JSON config files)
- Optimization work in ASP.NET Core startup for routing and logging
- SDK work to improve flexibility of linker task and targets
- Linker support for DI and other ASP.NET Core patterns
These changes represent size-on-disk of under 30mb, and a time-to-first-request of 220ms - an improvement of 100ms in the managed parts of the code - much of it by cutting features.
We should redefine the scenario we're targeting from the API template to the API template using route-to-code. We're working with a somewhat incomplete definition of who the customer is, but route-to-code is good programming style to pair with this regardless.
endpoints.MapGet("/weatherforecast", context =>
{
var rng = new Random();
var data = Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Date = DateTime.Now.AddDays(index),
TemperatureC = rng.Next(-20, 55),
Summary = Summaries[rng.Next(Summaries.Length)]
})
.ToArray();
context.Response.ContentType = "application/json";
return JsonSerializer.SerializeAsync(context.Response.Body, data);
});Contrasted with controllers, route-to-code will feel much more immediate and lightweight. This is a style of programming that's already widespread in other tech stacks and is becoming universal. We have plans in the works to try and make this style feel really good in .NET 5.
What's nice about this from a technical point-of-view is that route-to-code relies on very little dynamic infrastructure. The application is responsible directly configuring delegates and mappings to routes. There's no auto-discovery or indirection (beyond delegate calls) involved - it's linker-friendly by design. We should commit to keeping these invariants as we add features.
The default set of features that every new app gets today from Host.CreateDefaultBuilder include JSON configuration files as well as filewatcher support to refresh the configuration data when these files change.
This simply isn't a good fit for the single-exe workload - while we haven't totally nailed down the user cohort, it makes intuitive sense that you wouldn't create a single-binary that reads configuration from files on disk. We've also seen many user reports of problems related to file-watchers when running in linux server or especially container environments.
The final default state of Logging/Config for this customer should look like:
- Console logging
- EventSource logging
- Environment variable configuration
We definitely want users to have EventSource logging by default so they can use dotnet trace to analyze problems. It seems dangerous to remove from the defaults - users can do so if they choose.
Console logging can be in or out depending on the precise scenario definition. For general container and microservices scenarios console logging is almost ubiquitous. For IoT/embedded, it might not be supported at all.
Environment variable configuration is the obvious configuration source to include by default. It's low cost and environment variables are widely used.
We need to make a decision also here about EventLog logging for Windows. This is included by default by our default builder, but we need to make a scenario-based call about whether its needed.
From profiles that we've seen, initialization of routing's data-structures take ~15ms in our benchmarking environment with a trivial amount of data. This needs more investigation because it should be much faster than that. We expect to be able to recover most of this time.
Setup of the console logger is taking much longer than we expect or can easily justify as well - ~23ms. We know that the default initialization path uses the options binder to apply configuration to the logging options. We should make this faster by hand-coding what we need to in this area - it's on the critical path for startup of every application.
To function at all in this model, we need to make ASP.NET Core features (broadly) 'just work' with full trimming from the linker.
note: there are two different definitions for "linker-friendly" - either too much, or too little is trimmed. For now we're focused on cases where too much is trimmed, because it breaks the app.
We've identified plenty of these cases in ASP.NET Core already - I've created a separate document which you can find here that discusses what we could do.
The approach to making ASP.NET Core work with the linker could go down one of two paths:
- Build a dialect of attributes that are recognized by the linker, and annotate libraries
- Build specific pattern recognition using the linker's extensibility model
Some more design work is needed here to arrive at the right solution.
If we do all of the above (based on my estimations) - that leaves us at a total time-to-first-request of 220ms.
This doesn't reach our goal, but it's an improvement of 140ms vs our baseline and 100ms vs just the single-exe-host. So to repeat, that's 100ms gone from changing the scenario and making a few small investments.
Doing further work to improve startup will be driven by the requirements of the runtime. Our analysis shows that most of what we're seeing in performance traces is JIT or R2R overhead. The specific patterns used in managed code don't directly relate to our performance results ("feature X is expensive", "reflection is expensive"), the volume of code is what most closesly relates to our performance data.
- We can get faster by needing to JIT/Load less managed code.
- We can get faster by moving reflection/dynamism to build-time, which means we need to JIT/Load less code.
We have not yet done a run with Crossgen2 (nee CPAOT). Based on the performance data we collected, being able to crossgen with fragility could be very beneficial.
There's a difference in size between trimming + single-exe and our best results of about 17mb. What accounts for this size is the manual trimming of some native and managed components.
These generally fall into 4 categories:
- Native component that will be removed as part of productization (DAC, DBI)
- Managed component that's rooted manually by CoreFx
- Native dependency of a managed component
- Managed component that should rooted by ASP.NET
Most impactful by far, is the removal of native components.
The issue with this is that this list of items was developed by trial and error. If the assembly could be deleted without breaking our test configuration, it was put on this list.
Some set of this 17mb will need to remain because these APIs are:
- Hidden behind choke point APIs (can't be easily proven to be removable)
- Used in error-handling paths, or paths that aren't exercised by our test configuration
We'll need to work with CoreFx and accept some feature/API changes to ASP.NET to meet our goal.