simonhf

Performance experiment showing overhead of locking versus shared nothing architecture

How the experiement was performed

• Rent a packet.com m1.xlarge.x86 box which is an Intel Xeon E5-2650 V4 (2x) with 24 Cores @ 2.2 Ghz at $1.70/hr.
• On the box run the SHF performance test in various combinations using 100 million key values.
• The SHF performance test can be compiled with or without spin locks, in order to test the spin lock over head.
• Obviously when compiled without spin locks, concurrent access is not possible.
• However, when compiled without spin locks, multiple single CPU, non-concurrent instances can run in parallel AKA 'shared nothing'.
• E.g. comparing 12 CPUs running SHF with locks versus 12 CPUs running individual shared nothing, single CPU SHF instances, reveals the overhead of not using shared nothing.

simonhf

Example comparison of C Pointers versus Array Indexing

Why this example comparison?

Often using C array indexing has benefits over using pointers:

• Can be the same speed or faster code. Compiler does a good job.
• Code easier to comprehend.
• Log / instrumentation easier to comprehend.
• Special debug log / instrumentation easier to write and comprehend.

simonhf

use strict;

# Your Task
# You’re part of the team that explores Mars by sending remotely controlled vehicles to the surface of the planet. Develop an interface that translates the commands sent from earth to instructions that are understood by the rover.

# Requirements
# You are given the initial starting point (x,y) of a rover and the direction (N,S,E,W) it is facing.
# The rover receives a character array of commands as a string, for example "fffrfbrflb".
# Implement commands that move the rover forward/backward (f,b).
# Implement commands that turn the rover left/right (l,r).

simonhf

Experiments with libarchive read blocking: Part 1

Disclaimer: Don't know much about libarchive... yet!

Step 1: Describe the issue

• When reading a streamed archive using archive_read_open() [1] and archive_read_extract() [2] then a callback is called one or more times to read chunks of the archive.
• This creates an issue if (a) your program needs to wait for the next chunk to arrive, and/or (b) you want to process multiple archive streams in the same thread.
• Effectively archive_read_open() [1] and archive_read_extract() [2] block until all the necessary number of archive stream chunks have been read via the callback.

simonhf

Experiments with gcc auto instrumentration of C/C++ source code for run-time function call tree tracing

What are we trying to do?

Imagine a simple C project. Here is a very small one as an example:

#include <stdio.h>

extern int bar(int a);

simonhf

Experiments with libarchive read blocking: Part 2

• Note: This is a continuation from Part 1 [1].

[1] Experiments with libarchive read blocking: Part 1

How to comprehend libarchive internally?

There are too many files and lines of code (LOC) to start reading file by file

simonhf

Experiments with libarchive read blocking: Part 3

• Note: This is a continuation from Part 2 [1].

[1] Experiments with libarchive read blocking: Part 2

What if we used coroutines to swap between different libarchive instances in the same thread?

• Possible advantage: We might not need to modify libarchive itself!

simonhf

Experiments with libarchive read blocking: Part 4

• Note: This is a continuation from Part 3 [1].

[1] Experiments with libarchive read blocking: Part 3

How to figure out how much stack space the libarchive example3.c code needs?

• Modify cwrap.pl to report the stack pointer on every log line.
• Modify ./doit.sh to run example3.c with two archives, and show the SHA256 sums of the extracted files.

simonhf

Experiments with gcc auto instrumentration of C/C++ source code for run-time function call tree tracing continued...

What are we trying to do?

• In the original experiments [1] we managed to auto instrument C but not C++.
• So now it's back to the drawing board to figure out how to do the same for C++.

[1] Original auto instrumentation experiments

Why can't we use the same technique used for C for C++ ?

simonhf

Which missing feature in gcc would make tracing function calls in userspace C/C++ applications easier

Background

Let's say we want to compile a larger C/C++ application with 80,000 unique functions so that at run-time the function call-tree is output for code comprehension and debugging purposes.

There are a couple of options to do this today which spring to mind:

Option 1: Manually markup the source code