class: center, middle
-
Originally developed by Sun Microsystems, now Oracle
-
Works with most UNIXes. Comes pre-installed on Solaris and Mac OS X
-
CDDL license, not compatible with the GPL
DTrace has been described as a tool that "allows you to ask arbitrary questions about what the system is doing, and get answers."
-
Only works with Linux
-
GPL license
-
Current project members include Red Hat, IBM, Hitachi, and Oracle.
-
Provide a similar feature set
-
Are loaded into the kernel as modules to compiled in
-
Must be run as root (or with special groups)
-
Use Kprobes to implant probes and register event handlers(?)
-
A probe is an automated breakpoint that is implanted dynamically in executing (kernel-space) modules without the need to modify their underlying source.
-
They are particularly advocated in production environments where the use of interactive debuggers is undesirable.
-
Probe event handlers run as extensions to the system breakpoint interrupt handler and are expected to have little or no dependence on system facilities.
-
Because of this design point, probes are able to be implanted in the most hostile environments without adversely skewing system performance.
https://sourceware.org/systemtap/kprobes/
# hello-world.d
BEGIN {
trace("hello world\n");
exit(0);
}$ dtrace -s hello-world.d
# hello-world.stp
probe begin {
print("hello world\n")
exit()
}$ stap hello-world.stp
$ dtrace -n 'BEGIN { trace("hello world\n"); exit(0) }'
$ stap -e 'probe begin { print("hello world\n"); exit() }'
| DTrace | SystemTap |
|---|---|
| BEGIN | begin |
| END | end |
| syscall:::entry | syscall.\* |
| syscall:::return | syscall.\*.return |
| syscall::read:entry | syscall.read |
| profile:::tick-10s | timer.s(10) |
| DTrace | SystemTap |
|---|---|
| execname | execname() |
| uid | uid() |
| pid | pid() |
| timestamp | gettimeofday_ns() |
| arg0..N | (custom variable: see `stap -L PROBE`) |
| $target | target() |
| DTrace | SystemTap |
|---|---|
| stack() | print_backtrace() |
| quantize() | @hist_log() |
| lquantize() | @hist_linear() |
| exit(status) | exit() |
$ dtrace -ln syscall:::entry
$ stap -l 'syscall.*'
syscall.accept
syscall.access
syscall.acct
syscall.add_key
syscall.adjtimex
syscall.alarm
class: center, middle
$ dtrace -n 'syscall::read:return { @bytes = quantize(arg1); }'
$ stap -e 'global bytes; probe syscall.read.return { bytes <<< $return; } probe end { print(@hist_log(bytes)); }'
$ dtrace -n 'syscall:::entry { @[execname] = count(); }'
$ stap -e 'global ops; probe syscall.* { ops[execname()] <<< 1; }'
$ dtrace -n 'proc:::exec-success { trace(curpsinfo->pr_psargs); }'
$ stap -e 'probe process.begin { printf("%s\n", cmdline_str()); }'
-
$ git clone git://sourceware.org/git/systemtap.git $ cd /systemtap/testsuite/systemtap.examples/
The current iteration of SystemTap allows for a multitude of options when probing kernel-space events for a wide range of kernels. However, SystemTap's ability to probe user-space events is dependent on kernel support (the Utrace mechanism) that is unavailable in many kernels. Thus, only some kernel versions support user-space probing. At present, the developmental efforts of the SystemTap community are geared towards improving SystemTap's user-space probing capabilities.
https://sourceware.org/systemtap/wiki/utrace/arch/HowTo
But utrace is dead? http://stackoverflow.com/questions/12134041/is-utrace-project-dead
-
Linux Trace Toolkit - next generation: http://lttng.org/
-
OProfile: http://oprofile.sourceforge.net/about/
-
http://www.postgresql.org/docs/9.2/static/dynamic-trace.html
-
/usr/share/systemtap/tapsets/