brendangregg

Using my perf-tools just to wrap ftrace:

# ./perf-tools/bin/kprobe 'p:tcp_v4_connect skc_dport=+2(%si):u16' 
Tracing kprobe tcp_v4_connect. Ctrl-C to end.
          telnet-9723  [001] d... 62326244.175951: tcp_v4_connect: (tcp_v4_connect+0x0/0x480) skc_dport=1600
          telnet-9725  [001] d... 62326246.502760: tcp_v4_connect: (tcp_v4_connect+0x0/0x480) skc_dport=1700
          telnet-9726  [001] d... 62326247.861937: tcp_v4_connect: (tcp_v4_connect+0x0/0x480) skc_dport=100
          telnet-9727  [001] d... 62326249.220740: tcp_v4_connect: (tcp_v4_connect+0x0/0x480) skc_dport=e803
          
Now a crappy ntohs() to process the dport string:

brendangregg

# perf script | c++filt | more
node 36945 [004]  5233.662132: sdt_node:http__server__request: (da8b0c)
	          9a8b0c node::DTRACE_HTTP_SERVER_REQUEST(v8::FunctionCallbackInfo<v8::Value> const&)+0xffffffffffc0050c (/mnt/node-v4.4.1/out/Release/node)
	          452992 v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&))+0xffffffffffc00092 (/mnt/node-v4.4.1/out/Releas
e/node)
	          47c981 v8::internal::MaybeHandle<v8::internal::Object> v8::internal::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::(anonymous namespace
)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)1>&)+0xffffffffffc001e1 (/mnt/node-v4.4.1/out/Release/node)
	          47cdde v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*)+0xffffffffffc0003e (/mnt/node-v4.4.1/out/Release/node)
	    3da7b98068fb Stub:CEntryStub+0x5b (/tmp/perf-36945.map)
	    3da7b999a619 LazyCompile:~parserOnIncoming _http_server.js:454+0x2f9 (/tmp/perf-36945.

brendangregg

/*
 * memprofiletest
 *
 * USAGE: ./memprofiletest size_kb
 *
 * eg, ./memprofiletest 10240		# make larger than LLC cache for misses
 */
 
#include <stdio.h>
#include <stdlib.h>

brendangregg

var http = require('http');
var conns = 0;

http.createServer(function (req, res) {
	res.writeHead(200, {'Content-Type': 'text/plain'});
	res.end('G\'Day\n');
	conns++;
}).listen(8199, '0.0.0.0');

function status(interval) {

brendangregg

There are some quick benchmarks for the "Decade of Wasted Cores" patches on Linux 4.1. I had to add "extern int sched_max_numa_distance;" to arch/x86/kernel/smpboot.c for Linux 4.1 to compile. Brief analysis during the benchmarks using time(1) and mpstat(1) to check runtimes, usr/sys time, and per-CPU balance; iostat(1) to check for disk bottlenecks.

Summary: no significant difference seen in these tests.

c3.8xlarge (32 CPU) PV 1-node NUMA

The patch shouldn't make a difference to this 1-node system, but I felt it worth checking, especially since most of our systems are 1-node.

brendangregg

#!/usr/bin/python
# [...]
from __future__ import print_function
from bcc import BPF
from time import strftime

# load BPF program
bpf_text = """
#include <uapi/linux/ptrace.h>
int printret(struct pt_regs *ctx) {

brendangregg

#!/usr/bin/python
#
# chaintest   Summarize off-CPU time by kernel stack + 2 waker stacks
#             WORK IN PROGRESS. For Linux, uses BCC, eBPF.
#
# USAGE: chaintest [-h] [-u] [-p PID] [-i INTERVAL] [-T] [duration]
#
# PLEASE DO NOT RUN THIS IN PRODUCTION! This is a work in progress, intended to
# explore chain graphs on Linux, using eBPF capabilities from a particular
# kernel version (4.3ish). This tool will eventually get much better.

brendangregg

#!/bin/bash
#
# usdt - trace user statically defined tracepoints. User-level dynamic tracing.
#        Written using Linux ftrace. Experimental.
#
# WARNING: This is a proof of concept for USDT tracing from Linux ftrace, and
# is not safe to use in production environments. In particular, the -i option
# sets memory semaphores by piping the output of printf through dd and then
# to process memory via /proc/PID/mem. Yes, this program pipes the output of
# the shell directly over top of live process memory. If you don't understand

brendangregg

#!/bin/bash
#
# xenctx_prof - xenctx based unikernel profiler. Takes a single stack sample.
#
# WARNING: current version is a PROOF OF CONCEPT and is vastly inefficient.
# Check for future work.
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; under version 2 of the License.

brendangregg

--- flamegraph.pl	2015-12-08 10:30:59.000000000 -0800
+++ flamegraph-ocaml.pl	2016-01-26 17:20:46.000000000 -0800
@@ -342,6 +342,18 @@
 		}
 		# fall-through to color palettes
 	}
+	if (defined $type and $type eq "ocaml") {
+		if ($name =~ /::/) {		# C++
+			$type = "yellow";
+		} elsif ($name =~ /^caml/) {	# OCaml