wildgarden

Effective Engineer - Notes

• By Edmond Lau
• Highly Recommended 👍
• http://www.theeffectiveengineer.com/

What's an Effective Engineer?

• They are the people who get things done. Effective Engineers produce results.

Adopt the Right Mindsets

wildgarden

(gdb) bt
#0  0x00007ffff7df2e60 in open64 () from /lib64/ld-linux-x86-64.so.2
#1  0x00007ffff7de0918 in open_verify.constprop () from /lib64/ld-linux-x86-64.so.2
#2  0x00007ffff7de1046 in open_path () from /lib64/ld-linux-x86-64.so.2
#3  0x00007ffff7de26b6 in _dl_map_object () from /lib64/ld-linux-x86-64.so.2
#4  0x00007ffff7de6d80 in openaux () from /lib64/ld-linux-x86-64.so.2
#5  0x00007ffff7de93a4 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#6  0x00007ffff7de7005 in _dl_map_object_deps () from /lib64/ld-linux-x86-64.so.2
#7  0x00007ffff7ddcfc9 in dl_main () from /lib64/ld-linux-x86-64.so.2
#8  0x00007ffff7df14ef in _dl_sysdep_start () from /lib64/ld-linux-x86-64.so.2

wildgarden

Original case ( without intercepting ):

/------------------------------\
| application                  |
\------------------------------/
    |
    |
/------------------------------\
| libc                         |
|                              |

wildgarden

Goal: make file operations faster!
Assumption:
One bottleneck is the block I/O layer in the kernel.
We suspect, that pmem file operations can be faster by avoiding said layer ( might turn out to be false, we'll see ).

One can implement a file system either using the block IO layer, or above it - the only way
to implement it above it is to not use the kernel at all.

File operations on a regular rainy Monday morning:
/------------\

wildgarden

F1. FS 128k streaming writes

Benchmark: fio write

Command: fio --name=seqwrite --rw=write --bs=128k --size=4g --end_fsync=1 --loops=4 # aggrb tput

Rationale: Measure the performance of a single threaded streaming write of a reasonably large file. The aim is to measure how well the file system and platform can sustain a write workload, which will depend on how well it can group and dispatch writes. It's difficult to benchmark buffered file system writes in both a short duration and in a repeatable way, as performance greatly depends on if and when the pagecache begins to flush dirty data. As a workaround, an fsync() at the end of the benchmark is called to ensure that flushing will always occur, and the benchmark also repeats four times. While this provides a much more reliable measurement, it is somewhat worst-case (applications don't always fsync), providing closer to a minimum rate – rather than a maximum rate – that you should expect.

F2. FS cached 4k random reads

wildgarden

<?php

/**
 * SplClassLoader implementation that implements the technical interoperability
 * standards for PHP 5.3 namespaces and class names.
 *
 * http://groups.google.com/group/php-standards/web/final-proposal
 *
 *     // Example which loads classes for the Doctrine Common package in the
 *     // Doctrine\Common namespace.