Skip to content

Instantly share code, notes, and snippets.

@richlander
Last active July 23, 2019 16:30
Show Gist options
  • Save richlander/9dbb7cf0a9a53bfd161903ba4f20a1f6 to your computer and use it in GitHub Desktop.
Save richlander/9dbb7cf0a9a53bfd161903ba4f20a1f6 to your computer and use it in GitHub Desktop.
.NET Core 3.0 SDK Size Improvements

.NET Core 3.0 SDK Size Improvements

The .NET Core SDK is significantly smaller with .NET Core 3.0. The primary reason is that we changed the way we construct the SDK, by moving to purpose-built “packs” of various kinds (reference assemblies, frameworks, templates). In previous versions (including .NET Core 2.2), we constructed the SDK from NuGet packages, which included many artifacts that were not required and wasted a lot of space.

The following sections demonstrate the size improvements for Windows, Linux and macOS, including container delivery. They detail the process and commands that were used to determine the product sizes, enabling you to reproduce the same results in your own environment. To keep thing simple, zips and tar balls were downloaded from dotnet/core-sdk as opposed to the official installers.

Some readers will be shocked on how large the .NET Core 2.2 installer directory grows when the NuGetFallback archive is expanded to the NuGetFallBackFolder. We maximally compressed this archive to make the .NET Core SDK installer/zip smaller. IL compresses really well, which helps to explain the degree of expansion.

Summary of Improvements

.NET Core 3.0 SDK Size (size change in brackets)

Operating System Installer Size (change) On-disk Size (change)
Windows 164MB (-440KB; 0%) 441MB (-968MB; -68.7%)
Linux 115MB (-55MB; -32%) 332MB (-1068MB; -76.2%)
macOS 118MB (-51MB; -30%) 337MB (-1063MB; -75.9%)

The size improvements for Linux and macOS are dramatic. The improvement for Windows is smaller because we have added WPF and Windows Forms as part of .NET Core 3.0. It’s amazing that we added WPF and Windows Forms in 3.0 and the installer is still (a little bit) smaller.

You can see the same benefit with .NET Core SDK Docker images (here, limited to x64 Debian and Alpine).

Distro 2.2 Size (Compressed; Uncompressed) 3.0 Size (Compressed; Uncompressed)
Debian 598MB; 1.74GB 264MB; 706MB
Alpine 493MB; 1.48GB 148MB; 422MB

Note: .NET Core Runtime Docker images are much smaller than the SDK (both in 2.2 and 3.0).

.NET Core SDK for Windows

Note: Recent Windows 10 releases include curl and tar, which are used below. Alternative tools/methods will be needed for earlier Windows versions.

Download

C:\temp>curl -o dotnet22.zip https://dotnetcli.blob.core.windows.net/dotnet/Sdk/release/2.2.3xx/dotnet-sdk-latest-win-x64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  157M  100  157M    0     0  26.2M      0  0:00:06  0:00:06 --:--:-- 25.9M
C:\temp>curl -o dotnet30.zip https://dotnetcli.blob.core.windows.net/dotnet/Sdk/release/3.0.1xx/dotnet-sdk-latest-win-x64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  156M  100  156M    0     0  17.4M      0  0:00:09  0:00:09 --:--:-- 13.6M

List files sizes

C:\temp>dir dot*
 Volume in drive C is Windows
 Volume Serial Number is 384B-0B6E

 Directory of C:\temp

07/22/2019  09:27 AM       165,048,802 dotnet22.zip
07/22/2019  09:28 AM       164,558,259 dotnet30.zip

Unpack

C:\temp>mkdir dotnet22
C:\temp>mkdir dotnet30
C:\temp>tar -zxf dotnet22.zip -C dotnet22
C:\temp>tar -zxf dotnet30.zip -C dotnet30

List directory sizes

C:\temp>dir /s dotnet22
     Total Files Listed:
             945 File(s)    331,044,056 bytes
             548 Dir(s)  102,613,372,928 bytes free
C:\temp>dir /s dotnet30
     Total Files Listed:
            2718 File(s)    441,104,628 bytes
            1160 Dir(s)  102,616,453,120 bytes free

“First run” for .NET Core 2.2

C:\temp>set DOTNET_MULTILEVEL_LOOKUP=0
C:\temp>dotnet22\dotnet.exe new
Configuring...
--------------
A command is running to populate your local package cache to improve restore speed and enable offline access. This command takes up to one minute to complete and only runs once.
Decompressing 100% 5857 ms
Expanding 100% 23611 ms

Note: dotnet new produces a lot of extra console output that was elided for the sake of easy reading.

Note: The "first run" concept doesn't exist with .NET Core 3.0 (that's a big part of this overall improvement).

Note: See dotnet for more information on the DOTNET_MULTILEVEL_LOOKUP environment variable (only applies to Windows).

Re-list directory sizes

dir /s dotnet22
     Total Files Listed:
           11381 File(s)  1,409,344,902 bytes
           25679 Dir(s)  101,508,927,488 bytes free
C:\temp>dir /s dotnet30
     Total Files Listed:
            2718 File(s)    441,104,628 bytes
            1160 Dir(s)  101,507,854,336 bytes free

Note: dir /s produces a lot of extra console output that was elided for the sake of easy reading.

.NET Core SDK for Linux

Download

rich@trenzalore:~$ curl -o dotnet22.tar.gz https://dotnetcli.blob.core.windows.net/dotnet/Sdk/release/2.2.3xx/dotnet-sd
k-latest-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  162M  100  162M    0     0  13.9M      0  0:00:11  0:00:11 --:--:-- 15.9M
rich@trenzalore:~$ curl -o dotnet30.tar.gz https://dotnetcli.blob.core.windows.net/dotnet/Sdk/release/3.0.1xx/dotnet-sdk-latest-linux-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  109M  100  109M    0     0  20.2M      0  0:00:05  0:00:05 --:--:-- 22.0M

List file sizes

rich@trenzalore:~$ ls -l dot*
-rw-rw-rw- 1 rich rich 170192549 Jul 22 10:43 dotnet22.tar.gz
-rw-rw-rw- 1 rich rich 115335892 Jul 22 10:43 dotnet30.tar.gz

Unpack

rich@trenzalore:~$ mkdir dotnet22
rich@trenzalore:~$ tar -zxf dotnet22.tar.gz -C dotnet22
rich@trenzalore:~$ mkdir dotnet30
rich@trenzalore:~$ tar -zxf dotnet30.tar.gz -C dotnet30

List directory sizes

rich@trenzalore:~$ du -hs dotnet22 dotnet30
380M    dotnet22
332M    dotnet30

“First run” for .NET Core 2.2

rich@trenzalore:~$ dotnet22/dotnet new
Configuring...
--------------
A command is running to populate your local package cache to improve restore speed and enable offline access. This command takes up to one minute to complete and only runs once.
Decompressing 100% 6861 ms
Expanding 100% 71135 ms

Note: dotnet new produces a lot of extra console output that was elided for the sake of easy reading.

Note: The "first run" concept doesn't exist with .NET Core 3.0 (that's a big part of this overall improvement).

Re-list directory sizes

rich@trenzalore:~$ du -hs dotnet22 dotnet30
1.4G    dotnet22
332M    dotnet30

.NET Core SDK for macOS

Download

thundera:~ rich$ curl -o dotnet22.tar.gz https://dotnetcli.blob.core.windows.net/dotnet/Sdk/release/2.2.3xx/dotnet-sdk-latest-osx-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  161M  100  161M    0     0  21.1M      0  0:00:07  0:00:07 --:--:-- 21.6M
thundera:~ rich$ curl -o dotnet30.tar.gz https://dotnetcli.blob.core.windows.net/dotnet/Sdk/release/3.0.1xx/dotnet-sdk-latest-osx-x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  113M  100  113M    0     0  20.9M      0  0:00:05  0:00:05 --:--:-- 23.2M

List file sizes

thundera:~ rich$ ls -l dot*
-rw-r--r--  1 rich  staff  169797728 Jul 22 08:57 dotnet22.tar.gz
-rw-r--r--  1 rich  staff  118791630 Jul 22 08:58 dotnet30.tar.gz

Unpack

thundera:~ rich$ mkdir dotnet22
thundera:~ rich$ tar -zxf dotnet22.tar.gz -C dotnet22
thundera:~ rich$ mkdir dotnet30
thundera:~ rich$ tar -zxf dotnet30.tar.gz -C dotnet30

List directory sizes

thundera:~ rich$ du -hs dotnet22 dotnet30
339M dotnet22
337M dotnet30

“First run” for .NET Core 2.2

thundera:~ rich$ dotnet22/dotnet new
Configuring...
--------------
A command is running to populate your local package cache to improve restore speed and enable offline access. This command takes up to one minute to complete and only runs once.
Decompressing 100% 5010 ms
Expanding 100% 6243 ms

Note: dotnet new produces a lot of extra console output that was elided for the sake of easy reading.

Note: The "first run" concept doesn't exist with .NET Core 3.0 (that's a big part of this overall improvement).

Re-list directory sizes

thundera:~ rich$ du -hs dotnet22 dotnet30
1.4G dotnet22
337M dotnet30
thundera:~ rich$ 

.NET Core SDK for Docker

Pull images

docker pull mcr.microsoft.com/dotnet/core/sdk:2.2
docker pull mcr.microsoft.com/dotnet/core/sdk:2.2-alpine
docker pull mcr.microsoft.com/dotnet/core/sdk:3.0
docker pull mcr.microsoft.com/dotnet/core/sdk:3.0-alpine

Look at image sizes (uncompressed)

docker image ls mcr.microsoft.com/dotnet/core/sdk
REPOSITORY                          TAG                 IMAGE ID            CREATED             SIZE
mcr.microsoft.com/dotnet/core/sdk   2.2-alpine          010b0ec97feb        4 days ago          1.48GB
mcr.microsoft.com/dotnet/core/sdk   2.2                 2344653cce7d        4 days ago          1.74GB
mcr.microsoft.com/dotnet/core/sdk   3.0-alpine          475b34386410        4 days ago          424MB
mcr.microsoft.com/dotnet/core/sdk   3.0                 d8cf7ef43247        4 days ago          710MB

You can use docker image inspect to streamline the docker image output.

Get just the size for a single image (uncompressed)

docker image inspect --format="{{.Size}}" mcr.microsoft.com/dotnet/core/sdk:3.0-alpine
424262383

Get the size for multiple images, with a nice table (uncompressed)

docker images --format "table {{.Repository}}:{{.Tag}}\t {{.Size}}" mcr.microsoft.com/dotnet/core/sdk
REPOSITORY:TAG                                  SIZE
mcr.microsoft.com/dotnet/core/sdk:2.2-alpine    1.48GB
mcr.microsoft.com/dotnet/core/sdk:2.2           1.74GB
mcr.microsoft.com/dotnet/core/sdk:3.0-alpine    424MB
mcr.microsoft.com/dotnet/core/sdk:3.0           710MB

Get the size for an image (compressed)

The Microsoft Container Registry doesn't have a public API for determining image sizes. As a workaround, I uploaded the images to Docker Hub using the following round-about approach. You'll need to use your own DockerHub account instead of the one I'm using below.

docker login
docker pull mcr.microsoft.com/dotnet/core/sdk:3.0-alpine
docker tag mcr.microsoft.com/dotnet/core/sdk:3.0-alpine richlander/sdk:3.0-alpine
docker push richlander/sdk:3.0-alpine

I then repeated for the other images (minus the docker login, which is only required once). After that, I requested tag information from Docker Hub and used those sizes.

C:\Users\rich>curl -s -H "Authorization: JWT " "https://hub.docker.com/v2/repositories/richlander/sdk/tags/?page_size=100"
{"count": 4, "next": null, "previous": null, "results": [{"name": "2.2", "full_size": 598313550, "images": [{"size": 598313550, "architecture": "amd64", "variant": null, "features": null, "os": "linux", "os_version": null, "os_features": null}], "id": 63199109, "repository": 7443285, "creator": 1124295, "last_updater": 1124295, "last_updated": "2019-07-23T04:29:54.654360Z", "image_id": null, "v2": true}, {"name": "2.2-alpine", "full_size": 493602817, "images": [{"size": 493602817, "architecture": "amd64", "variant": null, "features": null, "os": "linux", "os_version": null, "os_features": null}], "id": 63198797, "repository": 7443285, "creator": 1124295, "last_updater": 1124295, "last_updated": "2019-07-23T04:24:20.929408Z", "image_id": null, "v2": true}, {"name": "3.0-alpine", "full_size": 148263985, "images": [{"size": 148263985, "architecture": "amd64", "variant": null, "features": null, "os": "linux", "os_version": null, "os_features": null}], "id": 63198713, "repository": 7443285, "creator": 1124295, "last_updater": 1124295, "last_updated": "2019-07-23T04:22:46.625796Z", "image_id": null, "v2": true}, {"name": "3.0", "full_size": 263928266, "images": [{"size": 263928266, "architecture": "amd64", "variant": null, "features": null, "os": "linux", "os_version": null, "os_features": null}], "id": 63198469, "repository": 7443285, "creator": 1124295, "last_updater": 1124295, "last_updated": "2019-07-23T04:18:55.138668Z", "image_id": null, "v2": true}]}

It is probably easiest to load the resulting JSON in an online JSON viewer.

Context: The Local Package Cache

The local package cache — the NuGetFallbackFolder — is no longer in use with .NET Core 3.0. This change is a major improvement to the initial experience using .NET Core and is the biggest win for reducing the amount of disk space .NET Core requires.

You’ve probably seen the following experience when you installed and .NET Core 2.2 or earlier versions for the first time:

Configuring...
--------------
A command is running to populate your local package cache to improve restore speed and enable offline access. This command takes up to one minute to complete and only runs once.
Decompressing 100% 5857 ms
Expanding 100% 23611 ms

With .NET Core 2.2, we copied a large number of NuGet packages into a local cache — the NuGetFallback folder — on your machine. Having the cache available made certain operations faster, limited the packages you needed to restore from NuGet.org and also provided an offline scenario (useful for laptops).

The SDK that you downloaded contained all of these packages in a maximally compressed file (with LZMA compression). The maximal compression significantly reduces the size of the SDK installer/zip, but is computationally expensive to uncompress (this is a typical tradeoff with comrpession). On slow hardware, it could take tens of seconds for this operation to complete. It was so bad on ARM hardware that we removed the local cache from those installers.

You can skip using the local cache by setting the DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true before installing the .NET Core SDK. This environment variable is no longer honored with .NET Core 3.0 since the basis for the setting has “disappeared”.

After you’ve fully adopted .NET Core 3.0 and uninstalled all older .NET Core versions, you can feel free to delete the NuGetFallbackFolder. The uninstaller does not remove this folder, so it will be left in place until explicitly deleted.

On Windows, you can find the NugetFallbackFolder at C:\Program Files\dotnet\sdk\NuGetFallbackFolder. It lives at similar locations (relative to the dotnet root) on macOS and Linux. This folder grows without bound. You can see that this folder is over 1GB on my desktop machine.

C:\>dir /s "C:\Program Files\dotnet\sdk\NuGetFallbackFolder"
     Total Files Listed:
           12517 File(s)  1,230,625,208 bytes

Closing

We learned a lot about building a whole new product — .NET Core — over the first four versions. We knew at the time that we didn’t have the best possible architecture in place for composing the product, but saw higher needs elsewhere, like establishing a compelling cross-platform and open source developer platform. With .NET Core 3.0, we’ve finally caught up on product composition and made the appropriate investments to resolve those challenges.

A smaller product helps many scenarios, with Docker containers and small devices being top of list. We’ve seen significant improvements in our own infrastructure, that uses Docker and/or ARM devices. We hope you see the same degree of improvements when you adopt .NET Core 3.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment