Skip to content

Instantly share code, notes, and snippets.

@layomia

layomia/ImmutableDictPerf.md

Last active May 31, 2019 18:00
Show Gist options
  • Save layomia/b1b07a5e5523ef67c3306d8c9fba8d86 to your computer and use it in GitHub Desktop.
Save layomia/b1b07a5e5523ef67c3306d8c9fba8d86 to your computer and use it in GitHub Desktop.
Download ZIP
Immutable Dict Perf
Raw
ImmutableDictPerf.md

Some performance notes.

TLDR

  • There's no significant performance impact on hot path (de)serialization i.e. primitives, objects, and enumerables.
  • There's a little (~4.6% avg.) regression for deserializing Dictionary, IDictionary, and IReadOnlyDictionary. This is due to some necessary if-checks introduced in shared code-paths between non-immutable dicts and immutable dicts. Splitting their code-paths entirely might mitigate this, but is beyond the scope for preview 6. This will be targeted in preview 7.
  • We might be able to improve the performance of (de)serializing ImmutableDicitonary, IImmutableDictionary, and ImmutableSortedDictionay by using IL emit to generate high performance CreateRange methods.

Hot-path deserialization and serialization use cases (Primitives, enumerables, objects)

These benchmarks live in the perf repo.

Before this change

[IndexViewModel]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 232.6 us 2.239 us 1.870 us 232.0 us 229.1 us 235.3 us 8.4746 0.9416 - 36.55 KB
DeserializeFromUtf8Bytes 230.5 us 4.789 us 4.480 us 230.5 us 224.6 us 241.7 us 5.4745 - - 23.65 KB
DeserializeFromStream 224.9 us 2.964 us 2.773 us 226.3 us 219.1 us 228.1 us 5.4005 - - 24.02 KB
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 337.4 us 3.357 us 2.803 us 337.0 us 332.1 us 342.2 us 5.6818 - - 27.95 KB
SerializeToUtf8Bytes 371.0 us 22.396 us 24.893 us 364.4 us 335.1 us 435.9 us 2.6596 - - 15.12 KB
SerializeToStream 363.0 us 29.353 us 32.626 us 348.4 us 330.1 us 446.6 us - - - 2.39 KB

[Location]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 8.986 us 0.2033 us 0.1902 us 8.995 us 8.718 us 9.462 us 0.1527 - - 704 B
DeserializeFromUtf8Bytes 8.693 us 0.1040 us 0.0922 us 8.693 us 8.551 us 8.871 us 0.1037 - - 472 B
DeserializeFromStream 9.431 us 0.1114 us 0.1042 us 9.455 us 9.262 us 9.637 us 0.1847 - - 856 B
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 8.457 us 0.5550 us 0.5939 us 8.239 us 7.765 us 10.029 us 0.1210 - - 608 B
SerializeToUtf8Bytes 8.041 us 0.5792 us 0.6438 us 7.791 us 7.467 us 9.539 us 0.0927 - - 400 B
SerializeToStream 7.825 us 0.0720 us 0.0638 us 7.840 us 7.730 us 7.913 us 0.0630 - - 352 B

[LoginViewModel]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 3.117 us 0.0579 us 0.0568 us 3.107 us 3.032 us 3.248 us 0.0624 - - 304 B
DeserializeFromUtf8Bytes 2.960 us 0.0483 us 0.0452 us 2.952 us 2.905 us 3.049 us 0.0351 - - 192 B
DeserializeFromStream 3.614 us 0.0625 us 0.0554 us 3.618 us 3.516 us 3.687 us 0.1298 - - 576 B
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 3.022 us 0.0409 us 0.0383 us 3.013 us 2.962 us 3.106 us 0.0839 - - 360 B
SerializeToUtf8Bytes 2.967 us 0.0438 us 0.0409 us 2.951 us 2.909 us 3.047 us 0.0587 - - 280 B
SerializeToStream 3.229 us 0.0956 us 0.1023 us 3.197 us 3.119 us 3.504 us 0.0714 - - 352 B

[MyEventsListerViewModel]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 2.247 ms 0.0439 ms 0.0411 ms 2.227 ms 2.206 ms 2.356 ms 35.3982 8.8496 - 161.9 KB
DeserializeFromUtf8Bytes 2.227 ms 0.0183 ms 0.0163 ms 2.228 ms 2.204 ms 2.263 ms 18.0180 - - 82.3 KB
DeserializeFromStream 2.412 ms 0.0410 ms 0.0342 ms 2.403 ms 2.355 ms 2.489 ms 19.2308 - - 82.68 KB
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 2.272 ms 0.0345 ms 0.0323 ms 2.266 ms 2.209 ms 2.312 ms 71.4286 44.6429 44.6429 370.61 KB
SerializeToUtf8Bytes 2.235 ms 0.0247 ms 0.0231 ms 2.237 ms 2.192 ms 2.264 ms 62.5000 8.9286 - 291 KB
SerializeToStream 2.269 ms 0.0523 ms 0.0537 ms 2.248 ms 2.195 ms 2.396 ms 44.6429 - - 211.62 KB

After this change

[IndexViewModel]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 234.2 us 2.904 us 2.717 us 234.0 us 229.8 us 240.1 us 8.3643 0.9294 - 36.59 KB
DeserializeFromUtf8Bytes 227.5 us 2.527 us 2.240 us 227.6 us 223.2 us 230.9 us 5.3763 - - 23.68 KB
DeserializeFromStream 236.7 us 2.255 us 2.110 us 237.2 us 232.0 us 239.6 us 5.6926 - - 24.06 KB
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 338.1 us 2.541 us 2.377 us 338.2 us 334.5 us 342.3 us 6.6489 - - 27.98 KB
SerializeToUtf8Bytes 332.3 us 1.488 us 1.319 us 332.4 us 328.8 us 334.4 us 2.6596 - - 15.15 KB
SerializeToStream 338.8 us 3.786 us 3.356 us 339.1 us 332.9 us 343.6 us - - - 2.43 KB

[Location]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 9.367 us 0.1075 us 0.1005 us 9.368 us 9.166 us 9.521 us 0.1473 - - 704 B
DeserializeFromUtf8Bytes 9.036 us 0.0908 us 0.0850 us 9.012 us 8.923 us 9.218 us 0.1113 - - 472 B
DeserializeFromStream 9.783 us 0.1341 us 0.1189 us 9.788 us 9.610 us 10.036 us 0.1969 - - 864 B
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 7.622 us 0.0856 us 0.0801 us 7.625 us 7.502 us 7.810 us 0.1236 - - 608 B
SerializeToUtf8Bytes 7.712 us 0.1600 us 0.1843 us 7.664 us 7.457 us 8.120 us 0.0903 - - 400 B
SerializeToStream 7.688 us 0.0395 us 0.0350 us 7.689 us 7.642 us 7.755 us 0.0615 - - 360 B

[LoginViewModel]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 3.190 us 0.0296 us 0.0262 us 3.187 us 3.120 us 3.232 us 0.0634 - - 304 B
DeserializeFromUtf8Bytes 3.075 us 0.0394 us 0.0369 us 3.069 us 2.982 us 3.132 us 0.0365 - - 192 B
DeserializeFromStream 3.619 us 0.0430 us 0.0381 us 3.628 us 3.555 us 3.694 us 0.1314 - - 584 B
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 3.012 us 0.0321 us 0.0300 us 3.009 us 2.962 us 3.061 us 0.0858 - - 360 B
SerializeToUtf8Bytes 3.002 us 0.0749 us 0.0832 us 2.985 us 2.897 us 3.192 us 0.0582 - - 280 B
SerializeToStream 3.346 us 0.1224 us 0.1409 us 3.314 us 3.161 us 3.581 us 0.0763 - - 360 B

[MyEventsListerViewModel]

Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
DeserializeFromString 2.312 ms 0.0355 ms 0.0349 ms 2.307 ms 2.263 ms 2.365 ms 36.6972 9.1743 - 161.9 KB
DeserializeFromUtf8Bytes 2.344 ms 0.0440 ms 0.0412 ms 2.346 ms 2.263 ms 2.407 ms 18.5185 - - 82.34 KB
DeserializeFromStream 2.521 ms 0.0320 ms 0.0299 ms 2.509 ms 2.484 ms 2.583 ms 20.2020 - - 82.72 KB
Method Mean Error StdDev Median Min Max Gen 0/1k Op Gen 1/1k Op Gen 2/1k Op Allocated Memory/Op
SerializeToString 2.255 ms 0.0179 ms 0.0167 ms 2.254 ms 2.229 ms 2.281 ms 71.4286 44.6429 44.6429 370.64 KB
SerializeToUtf8Bytes 2.209 ms 0.0206 ms 0.0193 ms 2.214 ms 2.182 ms 2.239 ms 62.5000 8.9286 - 291.07 KB
SerializeToStream 2.246 ms 0.0507 ms 0.0449 ms 2.245 ms 2.180 ms 2.350 ms 44.6429 - - 211.66 KB

Deserialization and serialization of Dictionary, IDictionary, IReadOnlyDictionary

We care about the perf changes here because this PR modifies their code paths. The benchmarks are being added to the performance repo: dotnet/performance#530.

Before this change

Method Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
DeserializeDict 698.0 ns 13.884 ns 22.811 ns 685.8 ns 0.0992 - - 416 B
DeserializeIDict 689.9 ns 6.155 ns 4.805 ns 690.4 ns 0.0992 - - 416 B
DeserializeIReadOnlyDict 689.1 ns 6.550 ns 5.470 ns 689.1 ns 0.0992 - - 416 B
SerializeDict 824.9 ns 18.945 ns 18.607 ns 821.0 ns 0.0954 - - 400 B
SerializeIDict 790.2 ns 15.665 ns 19.812 ns 784.8 ns 0.0954 - - 400 B
SerializeIReadOnlyDict 826.1 ns 19.380 ns 28.406 ns 820.8 ns 0.0954 - - 400 B

After this change

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
DeserializeDict 731.5 ns 14.684 ns 21.524 ns 0.0992 - - 416 B
DeserializeIDict 719.8 ns 14.314 ns 20.066 ns 0.0992 - - 416 B
DeserializeIReadOnlyDict 721.6 ns 14.251 ns 24.582 ns 0.0992 - - 416 B
SerializeDict 817.5 ns 16.344 ns 23.957 ns 0.0954 - - 400 B
SerializeIDict 802.7 ns 9.360 ns 8.755 ns 0.0954 - - 400 B
SerializeIReadOnlyDict 839.9 ns 16.863 ns 26.253 ns 0.0954 - - 400 B

[Added by this PR] Deserialization and serialization of ImmutableDictionary, IImmutableDictionary, ImmutableSortedDictionary

The benchmarks are being added to the performance repo: dotnet/performance#530.

Method Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
DeserializeImmutableDict 1.947 us 0.0700 us 0.1974 us 1.876 us 0.6332 - - 2649 B
DeserializeIImmutableDict 1.949 us 0.0404 us 0.1147 us 1.895 us 0.6332 - - 2649 B
DeserializeImmutableSortedDict 2.532 us 0.0504 us 0.1283 us 2.527 us 0.7591 - - 3177 B
SerializeImmutableDict_ToBytes 1.606 us 0.0081 us 0.0071 us 1.606 us 0.1793 - - 752 B
SerializeIImmutableDict_ToBytes 1.685 us 0.0338 us 0.0473 us 1.677 us 0.1793 - - 752 B
SerializeImmutableSortedDict_ToBytes 1.501 us 0.0299 us 0.0265 us 1.507 us 0.1564 - - 656 B
SerializeImmutableDict_ToString 1.721 us 0.0271 us 0.0290 us 1.711 us 0.1869 - - 784 B
SerializeIImmutableDict_ToString 1.709 us 0.0234 us 0.0182 us 1.707 us 0.1869 - - 784 B
SerializeImmutableSortedDict_ToString 1.357 us 0.0112 us 0.0094 us 1.358 us 0.1640 - - 688 B

Json.NET perf

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
DeserializeDict 948.3 ns 5.016 ns 4.447 ns 0.7010 - - 2.87 KB
DeserializeIDict 959.0 ns 6.703 ns 5.942 ns 0.7000 - - 2.87 KB
DeserializeIReadOnlyDict 1,010.8 ns 2.997 ns 2.804 ns 0.7172 - - 2.94 KB
SerializeDict_ToString 593.5 ns 1.524 ns 1.351 ns 0.3443 - - 1.41 KB
SerializeIDict_ToString 607.2 ns 2.617 ns 2.448 ns 0.3443 - - 1.41 KB
SerializeIReadOnlyDict_ToString 598.5 ns 2.760 ns 2.581 ns 0.3443 - - 1.41 KB
DeserializeImmutableDict 1,426.5 ns 6.333 ns 5.924 ns 0.7629 - - 3.12 KB
DeserializeIImmutableDict 1,839.7 ns 17.861 ns 14.915 ns 0.8793 - - 3.59 KB
DeserializeImmutableSortedDict 1,726.2 ns 7.363 ns 6.527 ns 0.8793 - - 3.59 KB
SerializeImmutableDict_ToString 1,196.9 ns 6.363 ns 5.640 ns 0.3967 - - 1.63 KB
SerializeIImmutableDict_ToString 1,192.0 ns 3.545 ns 3.316 ns 0.3967 - - 1.63 KB
SerializeImmutableSortedDict_ToString 948.9 ns 4.334 ns 3.842 ns 0.3748 - - 1.53 KB 
[JsonExporterAttribute.Full]
[MemoryDiagnoser]
public class JsonDotNet_DictionaryPerf
{
    private const string _jsonString = @"{""Hello"":""World"",""Hello2"":""World2""}";

    private Dictionary<string, string> _dict = new Dictionary<string, string>() { { "Hello", "World" }, { "Hello2", "World2" } };
    private static IDictionary<string, string> _iDict = new Dictionary<string, string>() { { "Hello", "World" }, { "Hello2", "World2" } };
    private IReadOnlyDictionary<string, string> _iReadOnlyDict = new Dictionary<string, string>() { { "Hello", "World" }, { "Hello2", "World2" } };

    private static ImmutableDictionary<string, string> _immutableDict;
    private static IImmutableDictionary<string, string> _iimmutableDict;
    private static ImmutableSortedDictionary<string, string> _immutableSortedDict;

    [GlobalSetup]
    public void Setup()
    {
        Dictionary<string, string> _dict = new Dictionary<string, string>() { { "Hello", "World" }, { "Hello2", "World2" } };

        _immutableDict = ImmutableDictionary.CreateRange(_dict);
        _iimmutableDict = ImmutableDictionary.CreateRange(_dict);
        _immutableSortedDict = ImmutableSortedDictionary.CreateRange(_dict);
    }

    [Benchmark]
    public Dictionary<string, string> DeserializeDict()
    {
        return JsonConvert.DeserializeObject<Dictionary<string, string>>(_jsonString);
    }

    [Benchmark]
    public IDictionary<string, string> DeserializeIDict()
    {
        return JsonConvert.DeserializeObject<IDictionary<string, string>>(_jsonString);
    }

    [Benchmark]
    public IReadOnlyDictionary<string, string> DeserializeIReadOnlyDict()
    {
        return JsonConvert.DeserializeObject<IReadOnlyDictionary<string, string>>(_jsonString);
    }

    [Benchmark]
    public string SerializeDict_ToString()
    {
        return JsonConvert.SerializeObject(_dict);
    }

    [Benchmark]
    public string SerializeIDict_ToString()
    {
        return JsonConvert.SerializeObject(_iDict);
    }

    [Benchmark]
    public string SerializeIReadOnlyDict_ToString()
    {
        return JsonConvert.SerializeObject(_iReadOnlyDict);
    }

    [Benchmark]
    public ImmutableDictionary<string, string> DeserializeImmutableDict()
    {
        return JsonConvert.DeserializeObject<ImmutableDictionary<string, string>>(_jsonString);
    }

    [Benchmark]
    public IImmutableDictionary<string, string> DeserializeIImmutableDict()
    {
        return JsonConvert.DeserializeObject<IImmutableDictionary<string, string>>(_jsonString);
    }

    [Benchmark]
    public ImmutableSortedDictionary<string, string> DeserializeImmutableSortedDict()
    {
        return JsonConvert.DeserializeObject<ImmutableSortedDictionary<string, string>>(_jsonString);
    }

    [Benchmark]
    public string SerializeImmutableDict_ToString()
    {
        return JsonConvert.SerializeObject(_immutableDict);
    }

    [Benchmark]
    public string SerializeIImmutableDict_ToString()
    {
        return JsonConvert.SerializeObject(_iimmutableDict);
    }

    [Benchmark]
    public string SerializeImmutableSortedDict_ToString()
    {
        return JsonConvert.SerializeObject(_immutableSortedDict);
    }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment