Skip to content

Instantly share code, notes, and snippets.

@trevnorris
Last active December 24, 2023 05:25
Star You must be signed in to star a gist
Save trevnorris/9616784 to your computer and use it in GitHub Desktop.
Quick steps of how to create a flame graph using perf

The prep-script.sh will setup the latest Node and install the latest perf version on your Linux box.

When you want to generate the flame graph, run the following (folder locations taken from install script):

sudo sysctl kernel.kptr_restrict=0
# May also have to do the following:
# (additional reading http://unix.stackexchange.com/questions/14227/do-i-need-root-admin-permissions-to-run-userspace-perf-tool-perf-events-ar )
sudo sysctl kernel.perf_event_paranoid=0

perf record -i -g -e cycles:u -- ~/sources/node/node --perf-basic-prof script.js

perf script | egrep -v "( __libc_start| LazyCompile | v8::internal::| Builtin:| Stub:| LoadIC:|\[unknown\]| LoadPolymorphicIC:)" | sed 's/ LazyCompile:[*~]\?/ /' | ~/sources/FlameGraph/stackcollapse-perf.pl > out.perf-folded

~/sources/FlameGraph/flamegraph.pl out.perf-folded > node-flame.svg

The data munging is to help expose the most important bits. There is still some play that I'm working with, but right now it seems to be working.

Alternatives:

perf script | egrep -v "( __libc_start|node::Start\(| LazyCompile | Builtin:| Stub:| LoadIC:|\[unknown\]| LoadPolymorphicIC:)" | ../FlameGraph/stackcollapse-perf.pl | grep "uv_run" > out.perf-folded

perf script | egrep -v "( __libc_start|node::Start\(| LazyCompile | v8::internal::| Builtin:| Stub:| LoadIC:|\[unknown\]| LoadPolymorphicIC:)" | ../FlameGraph/stackcollapse-perf.pl | grep "uv_run" > out.perf-folded
#!/bin/bash
sh /etc/lsb-release
wget -O - http://llvm.org/apt/llvm-snapshot.gpg.key|sudo apt-key add -
apt-add-repository "deb http://llvm.org/apt/${DISTRIB_CODENAME}/ llvm-toolchain-${DISTRIB_CODENAME} main"
apt-get update
apt-get -y upgrade
apt-get -y install make build-essential linux-tools-common elfutils libelf-dev flex bison libunwind8 libunwind8-dev libaudit-dev libdw-dev binutils-dev libnuma-dev libslang2-dev asciidoc llvm-3.6 clang-3.6 lldb-3.6 libllvm3.6 subversion libc6-dev-i386 libgtk2.0-dev libperl-dev python-dev git tmux nginx gdb apache2-utils
ln -s /usr/bin/clang-3.6 /usr/bin/clang
ln -s /usr/bin/clang++-3.6 /usr/bin/clang++
ln -s /usr/bin/lldb-3.6 /usr/bin/lldb
echo 'export CC=clang' > ~/.bash_aliases
echo 'export CXX=clang++' >> ~/.bash_aliases
echo 'export GYP_DEFINES="clang=1"' >> ~/.bash_aliases
. ~/.bash_aliases
chmod 777 /usr/share/nginx/html
ln -s /usr/share/nginx/html www
mkdir sources
cd sources
wget https://github.com/joyent/node/archive/v0.11.13.tar.gz
tar xvf v0.11.13.tar.gz
rm v0.11.13.tar.gz
mv node-0.11.13 node
cd node
./configure --gdb
make
make install
cd ~/sources
wget https://github.com/brendangregg/FlameGraph/archive/master.tar.gz
tar xvf master.tar.gz
rm master.tar.gz
mv FlameGraph-master FlameGraph
#cd ~/sources
#wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.15.6.tar.gz
#tar xvf linux-3.15.6.tar.gz
#rm linux-3.15.6.tar.gz
#cd linux-3.15.6/tools/perf/
#make -f Makefile.perf install prefix=/usr
@dignifiedquire
Copy link

I used this to get a html version (which is much more usable in the browser) of the graph using stackvis:

$ npm i -g stackvis
$ perf script | egrep -v "( __libc_start| LazyCompile | v8::internal::| Builtin:| Stub:| LoadIC:|\[unknown\]| LoadPolymorphicIC:)" | sed 's/ LazyCompile:[*~]\?/ /' > intermediate.out
$ stackvis perf < intermediate.out > flamegraph.htm

@arunoda
Copy link

arunoda commented Aug 2, 2014

What's the use of nginx? to watch the dumped traces?
If so, using python -m SimpleHTTPServer

@bjouhier
Copy link

bjouhier commented Aug 4, 2014

@trevnorris
Thanks, that's very helpful. I found two glitches on ubuntu:

  • the lines that add the exports to .bash_aliases broke the build. I had to remove these exports.
  • I got the following XML error in some of the generated SVGs:
<g class="func_g" onmouseover="s('RegExp:(^|[\r\n])([^"'\/]*)\/(?:\*[\s\S]*?\*\/|\/.*) (11 samples, 0.12%)')" onmouseout="c()">
-------------------------------------------------------^

I hacked around it that by inserting the following at line 460 of flamegraph.pl:

$info =~ s/\[\^\"\'/[^/g;

I don't know if the problem comes from flamegraph.pl or from one of the preprocessing steps.

@balupton
Copy link

balupton commented Aug 5, 2014

Would be awesome to get a docker image that does this.

@aseemk
Copy link

aseemk commented Sep 3, 2014

This is awesome. Any way to achieve this on Mac OS X?

@trevnorris
Copy link
Author

@dignifiedquire Very nice. Thanks for sharing.

@arunoda The reason for nginx is because I found it was the easiest way to view the flamegraphs myself and share them with others (simple as a single URL). Instead of needing to scp/ftp the file down to my box to look at it.

@bjouhier Thanks for the info about the broken XML. Sorry about the .bash_aliases break. I used this script on fresh servers that hadn't been touched yet.

@ryanstevens
Copy link

@trevnorris I just tried to do run perf after installing all the dependancies via the setup script. I've experienced the following error both in Ubuntu and Debian.

After running this step

$ perf record -i -g -e cycles:u -- ~/sources/node/node --perf-basic-prof script.js

I receive this

$ Perf session creation failed.

@thlorenz
Copy link

@aseemk you can do that with the flamegraph app I created.
Includes instructions on how to get things working on Linux and Mac.

Also removes the LazyCompile for you when you load a map file.

@jlukanta
Copy link

jlukanta commented Jan 8, 2015

I encountered the following errors:

E: Unable to locate package llvm-3.6
E: Couldn't find any package by regex 'llvm-3.6'
E: Unable to locate package clang-3.6
E: Couldn't find any package by regex 'clang-3.6'
E: Unable to locate package lldb-3.6
E: Couldn't find any package by regex 'lldb-3.6'
E: Unable to locate package libllvm3.6
E: Couldn't find any package by regex 'libllvm3.6'

It seems that llvm/clang/lldb/libllvm 3.6 packages were removed. To fix this issue, I simply switched to 3.5 packages. Hopefully this will help some people.

@Redsandro
Copy link

$ perf script > perf.data.script
[kernel.kallsyms] with build id 9a87d34680d3efd31b5d57f56906d8c2fc38fc5e not found, continuing without symbols
Failed to open /tmp/perf-29438.map, continuing without symbols

What did I miss?

@cantremember
Copy link

probably the node --perf-basic-prof part

@Redsandro
Copy link

I'm interested in the function calls. There's a lot of module._something, garbage collection and _ZN2v88internal5Scope11LookupLocalEPKNS0_12AstRawStringE stuff in the graph together with function calls. Can I remove all the other stuff and only have the function calls?

@idx3d
Copy link

idx3d commented Nov 30, 2016

^^ +1 for the question above. Thx.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment