This is a short post that explains how to write a high-performance matrix multiplication program on modern processors. In this tutorial I will use a single core of the Skylake-client CPU with AVX2, but the principles in this post also apply to other processors with different instruction sets (such as AVX512).
Matrix multiplication is a mathematical operation that defines the product of
| from cpython cimport PyObject | |
| from libc.string cimport strcmp | |
| cdef extern from "Python.h": | |
| ctypedef struct PyTypeObject: | |
| pass | |
| cdef extern from "structmember.h": | |
| ctypedef struct PyMemberDef: | |
| char *name |
| Latency Comparison Numbers (~2012) | |
| ---------------------------------- | |
| L1 cache reference 0.5 ns | |
| Branch mispredict 5 ns | |
| L2 cache reference 7 ns 14x L1 cache | |
| Mutex lock/unlock 25 ns | |
| Main memory reference 100 ns 20x L2 cache, 200x L1 cache | |
| Compress 1K bytes with Zippy 3,000 ns 3 us | |
| Send 1K bytes over 1 Gbps network 10,000 ns 10 us | |
| Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD |
| /*###################################################################### | |
| Example 6 : MPI_Isend MPI_Irecv | |
| Description: | |
| Examples 5 and 6 demonstrate the difference between blocking | |
| and non-blocking point-to-point communication. | |
| Example 5: MPI_Send/MPI_Recv (blocking) | |
| Example 6: MPI_Isend/MPI_Irecv (non-blocking) | |
A good commit message looks like this:
Header line: explaining the commit in one line
Body of commit message is a few lines of text, explaining things
in more detail, possibly giving some background about the issue
being fixed, etc etc.
The body of the commit message can be several paragraphs, and
please do proper word-wrap and keep columns shorter than about