Skip to content

Instantly share code, notes, and snippets.

@sonots
Last active April 24, 2024 13:54
Show Gist options
  • Save sonots/5abc0bccec2010ac69ff74788b265086 to your computer and use it in GitHub Desktop.
Save sonots/5abc0bccec2010ac69ff74788b265086 to your computer and use it in GitHub Desktop.
How to use NVIDIA profiler

Usually, located at /usr/local/cuda/bin

Non-Visual Profiler

$ nvprof python train_mnist.py

I prefer to use --print-gpu-trace.

$ nvprof --print-gpu-trace python train_mnist.py

Visual Profiler

On GPU machine, run

$ nvprof -o prof.nvvp python train_mnist.py

Copy prof.nvvp into your local machine

$ scp your_gpu_machine:/path/to/prof.nvvp .

Then, run nvvp (nvidia visual profiler) on your local machine:

$ nvvp prof.nvvp

It works more comfortably than X11 forwarding or something.

@HenryYihengXu
Copy link

Hello, do you know how to see GPU utilization with nvvp? I only see the duration of a function. But I want to also see the percentage of GPU utilization during a function.

@keke8273
Copy link

keke8273 commented Mar 4, 2020

i have to use the --profile-child-processes option to get the profile to work on windows.

@hudannag
Copy link

I get no outputs using nvprof. No matter if I give arguments or not.
Doesn't work with .exe file build with VS2019, nor using nvprof python program.py.

Can you please help?

@hitvoice
Copy link

On Mac, nvvp prof.nvvp doesn't work.
An absolute path (instead of a relative path like "prof.nvvp") has to be provided.

@danpovey
Copy link

My recommendation to anyone who wants to install nvvp on an up-to-date Mac (catalina) is not to even try.
Firstly you have to disable gatekeeper:
sudo spctl --master-disable
but then after you download nvvp it won't run because it wants an outdated java runtime (JRE) that's hard to get or install.
I still haven't figured it out.
Currently I'm trying to figure out how to do this, but it looks like it's very very painful.

@1chimaruGin
Copy link

Thanks man! Great help

@ccjjs
Copy link

ccjjs commented Jun 12, 2021

PDD XBJ GOD, YYDS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment