Skip to content

Instantly share code, notes, and snippets.

@PhilMarsh
Last active April 1, 2024 15:27
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save PhilMarsh/1c7c695631cd0cd1a3924371fe3764c9 to your computer and use it in GitHub Desktop.
Save PhilMarsh/1c7c695631cd0cd1a3924371fe3764c9 to your computer and use it in GitHub Desktop.
Investigate effect of size on Python dict construction times. Expanding on https://gist.github.com/hofrob/ad143aaa84c096f42489c2520a3875f9
import random
import string
import time
values = []
for i in range(50):
values.append("".join(random.choices(string.ascii_letters + string.digits, k=10)))
n = 500000
print(f"Construct {n=} dicts with these random values: {values=}")
print("---")
begin_dict = time.time_ns()
for i in range(n):
dict(
a0=values[0],
a1=values[1],
a2=values[2],
a3=values[3],
a4=values[4],
a5=values[5],
a6=values[6],
a7=values[7],
a8=values[8],
a9=values[9],
a10=values[10],
a11=values[11],
a12=values[12],
a13=values[13],
a14=values[14],
a15=values[15],
a16=values[16],
a17=values[17],
a18=values[18],
a19=values[19],
a20=values[20],
a21=values[21],
a22=values[22],
a23=values[23],
a24=values[24],
a25=values[25],
a26=values[26],
a27=values[27],
a28=values[28],
a29=values[29],
a30=values[30],
a31=values[31],
a32=values[32],
a33=values[33],
a34=values[34],
a35=values[35],
a36=values[36],
a37=values[37],
a38=values[38],
a39=values[39],
a40=values[40],
a41=values[41],
a42=values[42],
a43=values[43],
a44=values[44],
a45=values[45],
a46=values[46],
a47=values[47],
a48=values[48],
a49=values[49],
)
constructor_timing = time.time_ns() - begin_dict
print(f"{'dict constructor:':>20} {constructor_timing / 10 ** 6:.1f}ms")
begin_dict = time.time_ns()
for i in range(n):
{
"a0": values[0],
"a1": values[1],
"a2": values[2],
"a3": values[3],
"a4": values[4],
"a5": values[5],
"a6": values[6],
"a7": values[7],
"a8": values[8],
"a9": values[9],
"a10": values[10],
"a11": values[11],
"a12": values[12],
"a13": values[13],
"a14": values[14],
"a15": values[15],
"a16": values[16],
"a17": values[17],
"a18": values[18],
"a19": values[19],
"a20": values[20],
"a21": values[21],
"a22": values[22],
"a23": values[23],
"a24": values[24],
"a25": values[25],
"a26": values[26],
"a27": values[27],
"a28": values[28],
"a29": values[29],
"a30": values[30],
"a31": values[31],
"a32": values[32],
"a33": values[33],
"a34": values[34],
"a35": values[35],
"a36": values[36],
"a37": values[37],
"a38": values[38],
"a39": values[39],
"a40": values[40],
"a41": values[41],
"a42": values[42],
"a43": values[43],
"a44": values[44],
"a45": values[45],
"a46": values[46],
"a47": values[47],
"a48": values[48],
"a49": values[49],
}
literal_timing = time.time_ns() - begin_dict
print(f"{'literal dict:':>20} {literal_timing / 10 ** 6:.1f}ms")
percentage_difference = (constructor_timing - literal_timing) / constructor_timing * 100
print("---")
print(f"A literal dict is {percentage_difference:.2f}% faster")

Run on:

$ python -VV
Python 3.9.16 (main, Jan 18 2023, 14:05:22) 
[Clang 14.0.0 (clang-1400.0.29.202)]
Apple M2 Pro
16GB
Sonoma 14.2.1 (23C71)

For each smaller size, delete the extra construction lines and update the range() end.

50

$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['Q07b2eSYeY', 'cTQMAYVJ7a', '2hHGd5Niib', 'Nfwd8Ij4mf', '0tPSdtwg2b', 'kqhdPJNWC6', 'CMrPLtKqG7', 'EncR7XR31Y', 'jSOxEeQQ1f', 'WrZbpHDT5A', 'fny0XRlzoz', 'cijpaiY0X5', 'm7cRo4DUMB', '7Tts6EjyUi', 'aAWfmQqvd1', 'L6gEnDxjMT', 'Azdge4PbC7', '9SVUKETpPh', '0nxZ8kAx74', '2byIqL5izY', 'XFt09pSKO3', 'SCbOOXL8un', 'VYV0sTSV9n', '5RSxqgbFau', '0x7OpmDoEN', 'xIz9v7wHYT', 'zn07kWyie2', 'zM7XcH58C3', 'LnpxROB5BB', '6WNKpNgiRy', 'LEbhv6cYAc', 'tAi6uHhaZI', 'qZWMuxDxyR', 'oJAWYzxKxe', 'f6maUlQdlx', '1b9uEjH3eM', 'D30Z0j8jKT', 'VKm4VmUW69', '6UZL7zUrJr', 'L2yFwzMBxe', '9GqLcqCqKv', 'j2oZgJbO2A', '3v8uAEyO7b', 'k2F1TMbHYA', 'obDXvNT83P', 'XYx5nsKD8u', 'lyvFvlAuLz', 'HqOZfmdZKt', 'bKyJUNUPSf', 'gP4vcUYui5']
---
   dict constructor: 1109.8ms
       literal dict: 953.7ms
---
A literal dict is 14.06% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['DtdzJt2aeU', 'qrYvxFBf1J', 'ougpkK0Pet', 'zFHmRG67lT', 'Y6amsgssTY', 'ZlkNICLp73', '95IBdD7RDH', '9DOn12HMG5', 'zScqBwHYSe', '8vioCsB3P5', 'umAjLObZDF', 'bBGYdyX1XS', 'BjBzXnIUMV', 'K9ejBbYHqk', 'VMuE0nKjMm', 'ZUBGFsKESx', '6GwCoRuYtO', 'NYAGdA8N5p', '2RKJbNNZoJ', 'SRV1zlAgG9', 'YKLhxD7OKm', 'FuN3qQC5kd', 'PFtcNOat6z', 'jRks8Ggw5Y', 'byHLYCHRCo', 'RwGVRCsXgY', 'GCcCTXXdhA', 'FGJET59bxK', 'H5ymAcsrop', 'mYdxIhpzPL', '2dLwIA3Fwu', 'yvz80OhhG3', 'NkFwAH9JMr', 'QkWeaivkKK', 'xML0lcEZCi', 'xW4Yu7mPuA', 'n1o7KMrRTl', 'HmRancOmw6', 'dSymoA6rfm', 'MWTlrZgrsc', 'NSxX1dkUXl', 'ZUoW4iHvth', 'GtaOOUYz4m', '5gs2qAPOsv', '34b4gttaGL', 'HjPKdSc3js', '6bqvdJ3iMg', 'vR9iI7jj1Z', '7zYKLtq8El', 'MP1ZeYhjx4']
---
   dict constructor: 1105.3ms
       literal dict: 933.1ms
---
A literal dict is 15.58% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['Mnl01rtWmy', 'o9JKN07orq', 'wVU8CyoWWF', 'Pysck7cy5y', '2KNWZrPqOX', 'E5Ss7Db6Yi', '7772D1ALOo', 'ZPPuWUBGRt', 'qkFkbRKc4S', 'z22aSTmm1o', '6cfPPoBwUJ', 'Xp0bNb7QX7', 'MN2dLQOL4c', 'iqJ0d3fB6y', 'DwvIQGS96k', '0Hn64dlusU', 'bS1lZNaawb', 'avP97ZzV1S', 'R7vUm1S9ZV', 'yRSQuDmKHw', '98C0y5vQDo', 'zdO5Ys0FgL', 'cKKU0PGrvC', '2N3yqzOhOQ', 'IeqM6s9EEC', 'fTdcJNIiXr', 'DUcZi2SrC3', 'jxOtwoo7AP', 'Jzmym9hizX', 'ArEvpa3L7R', '18WqOqkAOh', 'zVssNI0RM9', 'm07cz6E9GT', 'KHT83uWuUe', 'GrxAZM2J2m', 'cuUlb03Apa', 'JuOixj5bDi', 'yKhsQy6GTg', 'aZb9hUpAhJ', 'vz0YIzLeZ6', 'XpkFO888kM', '6yrFmbVIpi', 'dhMOoqw4sh', 'oIprD2LR3a', 'gmmB6r8egy', 'RQDGWxD2N0', 'vW1bMf0uN3', 'PtvaDzhopv', 'esMmEoyTt7', 'bUAzdW9onH']
---
   dict constructor: 1164.8ms
       literal dict: 951.2ms
---
A literal dict is 18.34% faster

20

$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['0aKGOjPI3b', 'ltya1lEZGT', 'Uip0Qn8S8m', '04EIZuChcx', 'a577ZFPzab', 'BFf6jUopfF', '62iKoxmxpB', 'Yt9V3IJSja', 'r54CCF1M5b', 'ovDsdng3HF', 'H6DwTaNxDF', '9qanHDAANn', 'kmXrril7s0', 'XlNeOpEdnn', 'e2LqzWQiYX', 'nDeRgiA5uA', 'imYhjDF1lW', 'DgZcdMQZmt', 'BpJCcP1mR0', 'fHi1nFVpys']
---
   dict constructor: 583.8ms
       literal dict: 464.4ms
---
A literal dict is 20.46% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['B5UxMtmOek', 'h1bJl8PGon', 'MowDNuVw3S', 'qbtb2KjcLh', 'PVt79Q3lQi', 'Ht6pRcLtsS', 'w3XUrqbwN8', 'VXIwFwksYX', 'hFxLAmJWju', '9sQYwmnjpX', 'cMBZtqPfrx', 'Big6zRXYUH', 'GWQkW8dana', 'AKGUriXAmF', 'wiYcpuINBP', 'jL10QL37dI', 'Wb4npVzd1T', 'nZs82OVoTB', '8nVFeDSBCn', 'TAVJhNVcK4']
---
   dict constructor: 494.2ms
       literal dict: 423.1ms
---
A literal dict is 14.38% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['0vpI7vNWod', 'OsloodXWjm', '0pJHjG849h', 'V44KQGwoIa', 'vZTmHpELF7', 'jih00ymiUp', 'ewGb92mjuO', 'u8IO05yPsf', 'ZYfofoB2Im', '3a3Xm9DKSe', 'g4gUpUM8O6', '90QTgFXRmm', '7jba1IiP7v', '8v25rbpmtW', 'nmEhd7OA35', 'BNbTg7XC27', 'GfFHvYPClw', 'queSI9DDx1', '0st2VcsC0b', '0URUb2ZH1r']
---
   dict constructor: 483.8ms
       literal dict: 443.5ms
---
A literal dict is 8.33% faster

10

$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['JA1Jo995xY', '5Q77Da7DDf', 'pFLaW1MHDO', 'HmMFN6yil8', 'ujSKdKXQWJ', 'hR3DPM606J', '17ulebMgWF', 'r0MOWNw36j', 'vanz85CIWx', 'Kg2vwesoOC']
---
   dict constructor: 226.8ms
       literal dict: 205.3ms
---
A literal dict is 9.45% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['Gr8AVJB2CW', '6WtnoRo2en', '9BgMDdIzmu', 'DQoWbeeNn3', 'j2GFvjlU8p', 'xkP5oE3WqL', 'z3TRlG4iq3', 'XdiGg7WUbU', 'aK8SbCDn6Y', 'tgQTIzbiOb']
---
   dict constructor: 244.9ms
       literal dict: 210.0ms
---
A literal dict is 14.22% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['py1kSwgD2K', '683GVgjRvC', 'PeWNhmnPih', '1GO9Y25MHI', 'UXxxYAWcSF', 'yxs7nxxkS7', 'vdpo9cLbus', 'FeRXJvxycP', 'NDxy1p1ozV', 'ZL7wDxVc6k']
---
   dict constructor: 236.6ms
       literal dict: 210.0ms
---
A literal dict is 11.23% faster

5

$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['T4rJnngcnp', 'R9111wRC9r', '1y4qx6D3Ve', 'dbqq4gsoXy', '0BrHhFBBhU']
---
   dict constructor: 137.3ms
       literal dict: 114.9ms
---
A literal dict is 16.35% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['M0X9rBPar3', 'o9sfo9M7sD', 'KghzNd1S0u', 'XK7EhdYVh5', 'u3yiDQewzR']
---
   dict constructor: 136.6ms
       literal dict: 113.5ms
---
A literal dict is 16.92% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['sO8Y2M5veL', 'VTs2ePBWqG', '4bbzQ4zW4A', 'hAXpaMcIqL', 'cKu0Jezvqd']
---
   dict constructor: 133.0ms
       literal dict: 113.8ms
---
A literal dict is 14.37% faster

2

$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['dua8KscKuQ', 'Cpod2UASyl']
---
   dict constructor: 82.7ms
       literal dict: 60.5ms
---
A literal dict is 26.89% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['pVHFGEDPuB', 'xfKqzjrNzk']
---
   dict constructor: 76.6ms
       literal dict: 56.6ms
---
A literal dict is 26.10% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['8IZPjupobh', 'teTVxXUca9']
---
   dict constructor: 83.4ms
       literal dict: 58.5ms
---
A literal dict is 29.87% faster

1

$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['6loLLlD8jx']
---
   dict constructor: 69.3ms
       literal dict: 42.1ms
---
A literal dict is 39.25% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['0CMtpPSQTt']
---
   dict constructor: 64.8ms
       literal dict: 40.8ms
---
A literal dict is 37.01% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=['3JTlV2dp0j']
---
   dict constructor: 63.4ms
       literal dict: 41.4ms
---
A literal dict is 34.68% faster

0

$ python dict_perf.py
Construct n=500000 dicts with these random values: values=[]
---
   dict constructor: 48.8ms
       literal dict: 20.5ms
---
A literal dict is 57.95% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=[]
---
   dict constructor: 52.0ms
       literal dict: 20.3ms
---
A literal dict is 60.92% faster
$ python dict_perf.py
Construct n=500000 dicts with these random values: values=[]
---
   dict constructor: 49.3ms
       literal dict: 20.6ms
---
A literal dict is 58.11% faster

summary

values run constructor (ms) literal (ms) diff
0 1 48.8 20.5 57.95%
0 2 52.0 20.3 60.92%
0 3 49.3 20.6 58.11%
1 1 69.3 42.1 39.25%
1 2 64.8 40.8 37.01%
1 3 63.4 41.4 34.68%
2 1 82.7 60.5 26.89%
2 2 76.6 56.6 26.10%
2 3 83.4 58.5 29.87%
5 1 137.3 114.9 16.35%
5 2 136.6 113.5 16.92%
5 3 133.0 113.8 14.37%
10 1 226.8 205.3 9.45%
10 2 244.9 210.0 14.22%
10 3 236.6 210.0 11.23%
20 1 583.8 464.4 20.46%
20 2 494.2 423.1 14.38%
20 3 483.8 443.5 8.33%
50 1 1109.8 953.7 14.06%
50 2 1105.3 933.1 15.58%
50 3 1164.8 951.2 18.34%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment