If you have sudo privileges then this is relatively easy, and can be kept secured to those users with sudo privs. Avoids needing to reconfigure users/environments, and allows you to debug any process on the machine regardless of which user account it is running as.
If you use Visual Studio to make an initial attempt to debug you will find that a ~/.vs-debugger folder has been created in the home directory of the user account you were attempting to use. This command will help you locate the vsdbg binary which was installed. You can install VsDbg manually but I find leveraging the automated process is easier. If you are using VSCode this becomes a manual process, and an exercize left for the reader, but I would still use VS2019 IDE to prep the target just to keep things consistent between tools.
find ~ | grep vsdbg
For my installation the binary is located at ~/.vs-debugger/vs2019/vsdbg and this path will most likely change over time.
First, rename the binary to something convenient:
mv ~/.vs-debugger/vs2019/vsdbg ~/.vs-debugger/vs2019/vsdbg-bin
Second, create a script to replace the binary:
touch ~/.vs-debugger/vs2019/vsdbg
chmod 770 ~/.vs-debugger/vs2019/vsdbg
nano ~/.vs-debugger/vs2019/vsdbg
The script content might look something like this, note the full path to vsdbg-bin, the use of $@ ensures all command-line args passed to your script are forwarded to VsDbg.
#!/bin/bash
sudo ~/.vs-debugger/vs2019/vsdbg-bin $@
Now retry your debug session from Visual Studio, if you did things correctly you should be able to attach to any remote process on the target machine using SSH->VsDbg.