ncu-profiling.md

Enable NVIDIA Nsight Compute Profiling for Your Customers

The Problem

Your GPU customers can't profile their CUDA/PyTorch workloads because NCU profiling is disabled by default:

$ ncu python script.py
==ERROR== ERR_NVGPUCTRPERM: Permission denied

The Fix

Run this once on each GPU host:

echo "options nvidia NVreg_RestrictProfilingToAdminUsers=0" | sudo tee /etc/modprobe.d/nvprof.conf
sudo reboot

That's it. All containers on that host can now use NCU.

Docker probably needs one more flag to be added

Reference

Official NVIDIA guidance: NVIDIA Developer Docs