Latency Numbers Every Programmer Should Know

latency.txt

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD
Read 1 MB sequentially from memory     250,000   ns      250 us
Round trip within same datacenter      500,000   ns      500 us
Read 1 MB sequentially from SSD*     1,000,000   ns    1,000 us    1 ms  ~1GB/sec SSD, 4X memory
Disk seek                           10,000,000   ns   10,000 us   10 ms  20x datacenter roundtrip
Read 1 MB sequentially from disk    20,000,000   ns   20,000 us   20 ms  80x memory, 20X SSD
Send packet CA->Netherlands->CA    150,000,000   ns  150,000 us  150 ms

Notes
-----
1 ns = 10^-9 seconds
1 us = 10^-6 seconds = 1,000 ns
1 ms = 10^-3 seconds = 1,000 us = 1,000,000 ns

Credit
------
By Jeff Dean:               http://research.google.com/people/jeff/
Originally by Peter Norvig: http://norvig.com/21-days.html#answers

Contributions
-------------
'Humanized' comparison:  https://gist.github.com/hellerbarde/2843375
Visual comparison chart: http://i.imgur.com/k0t1e.png

need a solar system type visualization for this, so we can really appreciate the change of scale.

I agree, would be fun to see. :-)

useful information & thanks

Looks nice kudos !
One comment about the Branch mispredict: if the cpu architecture is based on P4 or Bulldozer that would result in 20-30+ cycles on a mispredict that would translate to a much bigger number (and they do mispredict) :)

For SSD's would be something like:
Disk seek: 100 000 ns

Latency numbers between large cities: https://wondernetwork.com/pings/

@preinheimer Asia & Australasia have it bad.

From the same author: http://videolectures.net/wsdm09_dean_cblirs/

"Latency numbers every programmer should know" - yet naturally, it has no information about humans!

http://biae.clemson.edu/bpc/bp/lab/110/reaction.htm

maybe you want to incorporate some of this: https://gist.github.com/2843375

Curious to see numbers for SSD read time

I think the reference you want to cite is here: http://norvig.com/21-days.html#answers

This remind me of this Grace Hopper's video about Nanoseconds. Really worthy.
http://www.youtube.com/watch?v=JEpsKnWZrJ8

I find comparisons much more useful than raw numbers: https://gist.github.com/2844130

I'm surprised that mechanical disk reads are only 80x the speed of main memory reads.

my version : https://gist.github.com/2842457 includes SSD number, would love some more

Does L1 and L2 cache latency depends on processor type? and what about L3 cache.

Ofc it does ... those are averages I think.

Would be nice to right-align the numbers so people can more easily compare orders of magnitude.

Good idea. Fixed.

And expanded even a bit more: https://gist.github.com/2845836 (SSD numbers, relative comparisons, more links)

TLB misses would be nice to list too, so people see the value of large pages...

Context switches (for various OSes), ...

Also, regarding packet sends, that must be latency from send initiation to send completion -- I assume.

If you're going to list mutex lock/unlock, how about memory barriers?

Thanks! This is quite useful, particularly for flogging at others.

Quick pie chart of data with scales in time (1 sec -> 9.5 years) for fun.

Spreadsheet with chart

"Read 1 MB sequentially from disk - 20,000,000 ns". Is this with or without disk seek time?

I made a fusion table for this at:
https://www.google.com/fusiontables/DataSource?snapid=S523155yioc

Maybe be helpful for graphing, etc. Thanks for putting this together

Cool. Thanks.
Thanks everyone for all the great improvements.

Here is a chart version. It's a bit hard to read, but I hope it conveys the perspective.
http://i.imgur.com/k0t1e.png

It would also be very interesting to add memory allocation timings to that : )

How long does it take before this shows up in XKCD?

You guys are talking about is the powers of ten http://vimeo.com/819138

If it does show up on xkcd it will be next to a gigantic "How much time it takes for a human to react to any results", hopefully with the intent to show people that any USE of this knowledge should be tempered with an understanding of what it will be used for--possibly showing how getting a bit from the cache is pretty much identical to getting a bit from china when it comes to a single fetch of information to show a human being.

@BillKress yes, this is specifically for Programmers, to make sure they have an understanding about the bottlenecks involved in programming. If you know these numbers, you know that you need to cut down on disk access before cutting down on in-memory shuffling.
If you don't properly follow these numbers and what they stand for, you will make programs that don't scale well. That is why they are important on their own and (in this context) should not be dwarfed by human reaction times.

@BillKress If we were only concerned with showing information to a single human being at a time we could just as well shut down our development machines and go out into the sun and play. This is about scalability.

this is getting out of hand, how do i unsubscribe from this gist?

Saw this via @smashingmag . While you guys debate the fit for purpose, here is another visualization of your quick reference latency data with Prezi ow.ly/bnB7q

Does anybody know how to stop receiving notifications from a gist's activity?

Here's a tool to visualize these numbers over time: http://www.eecs.berkeley.edu/~rcs/research/interactive_latency.html

I just created flash cards for this: https://ankiweb.net/shared/info/3116110484 They can be downloaded using the Anki application: http://ankisrs.net

I'm also missing something like "Send 1MB bytes over 1 Gbps network (within datacenter over TCP)". Or does that vary so much that it would be impossible to specify?

If L1 access is a second, then:

L1 cache reference : 0:00:01
Branch mispredict : 0:00:10
L2 cache reference : 0:00:14
Mutex lock/unlock : 0:00:50
Main memory reference : 0:03:20
Compress 1K bytes with Zippy : 1:40:00
Send 1K bytes over 1 Gbps network : 5:33:20
Read 4K randomly from SSD : 3 days, 11:20:00
Read 1 MB sequentially from memory : 5 days, 18:53:20
Round trip within same datacenter : 11 days, 13:46:40
Read 1 MB sequentially from SSD : 23 days, 3:33:20
Disk seek : 231 days, 11:33:20
Read 1 MB sequentially from disk : 462 days, 23:06:40
Send packet CA->Netherlands->CA : 3472 days, 5:20:00

You can add LTO4 tape seek/access time, ~ 55 sec, or 55.000.000.000 ns

I'm missing things like sending 1K via Unix pipe/ socket / tcp to another process.
Has anybody numbers about that?

@metakeule its easily measurable.

Related page from "Systems Performance" with similar second scaling mentioned by @kofemann: https://twitter.com/rzezeski/status/398306728263315456/photo/1

L1D hit on a modern Intel CPU (Nehalem+) is at least 4 cycles. For a typical server/desktop at 2.5Ghz it is at least 1.6ns.
Fastest L2 hit latency is 11 cycles(Sandy Bridge+) which is 2.75x not 14x.
May be the numbers by Norwig were true at some time, but at least caches latency numbers are pretty constant since Nehalem which was 6 years ago.

Please note that Peter Norvig first published this expanded version (at that location - http://norvig.com/21-days.html#answers) ~JUL2010 (see wayback machine). Also, note that it was "Approximate timing for various operations on a typical PC".

One light-nanosecond is roughly a foot, which is considerably less than the distance to my monitor right now. It's kind of surprising to realize just how much a CPU can get done in the time it takes light to traverse the average viewing distance...

@jboner, I would like to cite some numbers in a formal publication. Who is the author? Jeff Dean? Which url should I cite? Thanks.

I'd like to see the number for "Append 1 MB to file on disk".

The "Send 1K bytes over 1 Gbps network" doesn't feel right, if you were comparing the 1MB sequential read of memory, SSD, Disk, the Gbps network for 1MB would be faster than disk (x1024), that doesn't feel right.

A great solar system type visualisation: http://joshworth.com/dev/pixelspace/pixelspace_solarsystem.html

I turned this into a set of flashcards on Quizlet: https://quizlet.com/_1iqyko

Can you update the the Notes section with the following
1 ns = 10^-9 seconds
1 ms = 10^-3 seconds

Thanks.

@misgeatgit Updated

Zippy is nowadays called snappy. Might be worth updating. Tx for the gist.

Several of the recent comments are spam. The links lead to sites in India which have absolutely nothing to do with latency.

Are there any numbers about latency between NUMA nodes?

Sequential SSD speed is actually more like 500 MB/s, not 1000 MB/s for SATA drives (http://www.tomshardware.com/reviews/ssd-recommendation-benchmark,3269.html).

You really should cite the folks at Berkeley. Their site is interactive, has been up for 20 years, and it is where you "sourced" your visualization. http://www.eecs.berkeley.edu/~rcs/research/interactive_latency.html

Question~ do these numbers not vary from one set of hardware to the next? How can these be accurate for all different types of RAM, CPU, motherboard, hard drive, etc?

(I am primarily a front-end JS dev, I know little-to-nothing about this side of programming, where one must consider numbers involving RAM and CPU. Forgive me if I'm missing something obvious.)

The link to the animated presentation is broken, here's the correct one: http://prezi.com/pdkvgys-r0y6/latency-numbers-for-programmers-web-development

Love this one.

Mentioned gist : https://gist.github.com/2843375 is private or was removed.
can someone restore it?
Thanks!

It would be nice to be able to compare this to computation times -- How long to do an add, xor, multiply, or branch operation?

Last year, I came up with this concept for an infographic illustrating these latency numbers with time analogies (if 1 CPU cycle = 1 second). Here was the result: http://imgur.com/8LIwV4C

Most of these number were valid in 2000-2001, right now some of these numbers are wrong by an order of magnitude. ( especially reading from main memory, as DRAM bandwidth doubles every 3 years )

µs, not us

I realize this was published some time ago, but the following URLs are no longer reachable/valid:

• https://gist.github.com/2843375
• http://prezi.com/pdkvgys-r0y6/latency-numbers-for-programmers-web-development/latency.txt

However, the second URL should now be: https://prezi.com/pdkvgys-r0y6/latency-numbers-for-programmers-web-development/

Oh, and @mpron - nice!

Thank you @jboner

Note: I created my own "fork" of this.

Thank you @GLMeece

Google it

Median human reaction time (to some stimulus showing up on a screen): 270 ms
(value probably increases with age)
https://www.humanbenchmark.com/tests/reactiontime/statistics

Awesome info. Thanks!

https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html

Could you please add printf & fprintf to this list

Heh, imagine this transposed into human distances.

1ns = 1 step, or 2 feet.

L1 cache reference = reaching 1 foot across your desk to pick something up
Datacentre roundtrip = 94 mile hike.
Internet roundtrip (California to Netherlands) = Walk around the entire earth. Wait! You're not done. Then walk from London, to Havana. Oh, and then to Jacksonville, Florida. Then you're done.

The last link is giving a 404

The numbers "Read 1 MB sequentially from memory" mean a memory bandwidth of 4 GB/s. That is a very old number. Can you update it? The time should be roughly 1/5th - one core can do about 20 GB/s today, all cores of a 4 or 8 core about 40 GB/s together. I remember seeing 18-19 GB/s in memtest86 for single core on my Ryzen 1800X and there are several benchmarks floating around where all cores do about 40 GB/s. It is very hard to find anything on the web about single core memory bandwidth...

Good information, thanks.

http://ram.userbenchmark.com/
Ram has gotten slightly faster. It is 70 ns now.

Edit: I was wrong. https://developers.redhat.com/blog/2016/03/01/reducing-memory-access-times-with-caches/

there is an updated version of the latency table?

Nice gist. Thanks @jboner.

Links are dead..

https://gist.github.com/2843375
http://prezi.com/pdkvgys-r0y6/latency-numbers-for-programmers-web-development/latency.txt

@jboner lets remove them

https://prezi.com/pdkvgys-r0y6/latency-numbers-for-programmers-web-development/

This prezi presentation is reversed: the larger numbers are inside the smaller ones, instead of the logical opposite.

Humanized version can be found: https://gist.github.com/hellerbarde/2843375

Thanks. Updated.

Where is the xkcd version?

This one is nice https://gist.github.com/hellerbarde/2843375#gistcomment-1896153

http://ithare.com/infographics-operation-costs-in-cpu-clock-cycles/

Just use logarithms directly: https://gist.github.com/negrinho/8a8b45a8958a8653054aa2b349b4cb05

Thanks

Is there any resources when one can test himself with a tasks involving these numbers?
E.g. calculate how much time will it take to read 5Mb from DB in another datacenter and get it back?
That would be a great test of applying those numbers in some real use cases.

I think given increased use of GPUs / TPUs it might be interesting numbers to add here now. Like: 1MB over PCIexpress to GPU memory, Computing 100 prime numbers per core of CPU compared to CPU, reading 1 MB from GPU memory to GPU etc.

Markdown version https://gist.github.com/sergekukharev/ccdd49d23a5078f108175dc71ad3c06c

useful information & thanks

Some data of the Berkeley interactive version (https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html ) is estimated, eg: 4 µs in 2019 to read 1 MB sequentially from memory; it seems too fast.

this is a great idea.
how about the time to complete a DNS request - UDP packet request and response with a DNS server having, say, 1ms response time, with the DNS server being 5ms packet time-of-flight away?

https://computers-are-fast.github.io

What effect on latency has the use multiple native threads on doing operations possible due to proper mutex locking? Assumed you have:

• an operation 1024 ns operation in 1st level cache
• 2 x lock unlock mutex (50 ns)
• move it from/to main memory (200 ns)

Now, I wonder about malloc latency, can you tell about it? It is definitely missing because I can compute data without any lock as owning the data.

interesting when you see in a glance. but would't it be good to use one unit in the comparison e.g. memory page 4k?

Nanoseconds

It's an excellent explanation. I had to search the video because the account was closed. Here's the result I got: https://www.youtube.com/watch?v=9eyFDBPk4Yw

Send 1K bytes over 1 Gbps network 10,000 ns 10 us
This doesn't look right to me. 1 Gbps = 125, 000 KB/s, the time should be 1 / 125,000 = 8 * 10^-6 seconds which is 8000ns

Send 1K bytes over 1 Gbps network 10,000 ns 10 us
This doesn't look right to me. 1 Gbps = 125, 000 KB/s, the time should be 1 / 125,000 = 8 * 10^-6 seconds which is 8000ns

For a direct host-to-host connection with 1000BaseT interfaces, a wire latency of 8µs is correct.

However, if the hosts are connected using SGMII, the Serial Gigabit Media Independent Interface, data is 8b10b encoded, meaning 10 bits are sent for every 8 bits of data, leading to a latency of 10µs.

Jeff may also have been referring to the fact that in a large cluster you'll have a few switches between the hosts, so even where 1000BaseT is in use, the added switching latency (even for switches operating in cut-through mode) for, say, 2 switches can approach 2µs.

In any event, the main thing to take away from these numbers are the orders of magnitude differences between latency for various methods of I/O.

Fancy unicode version:

Latency Comparison Numbers (~2012)
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│L1 cache reference			          0.5 ns								│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Branch mispredict			          5   ns								│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│L2 cache reference			          7   ns				14× L1 cache			│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Mutex lock/unlock			         25   ns								│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Main memory reference			        100   ns				20× L2 cache, 200× L1 cache	│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Compress 1K bytes with Zippy		      3,000   ns         3 μs							│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Send 1K bytes over 1 Gbps network	     10,000   ns        10 μs							│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Read 4K randomly from SSD*		    150,000   ns       150 μs			~1GB/sec SSD			│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Read 1 MB sequentially from memory	    250,000   ns       250 μs							│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Round trip within same datacenter	    500,000   ns       500 μs							│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Read 1 MB sequentially from SSD*	  1,000,000   ns     1,000 μs	    1 ms  	~1GB/sec SSD, 4× memory		│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Disk seek				 10,000,000   ns    10,000 μs	   10 ms  	20× datacenter roundtrip	│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Read 1 MB sequentially from disk	 20,000,000   ns    20,000 μs	   20 ms  	80× memory, 20× SSD		│
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Send packet CA→Netherlands→CA		150,000,000   ns   150,000 μs	  150 ms 					│
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Are these numbers still relevant in 2020? Or this needs an update?