Install a window manager, XFCE is my preference for VNC setups.
$ sudo apt install xubuntu-desktop tightvncserver
$ sudo reboot
Next, configure the VNC session behavior. vncpasswd interactively prompts for a password, its different than the user password, and hashed into ~/.vnc/passwd.
$ mkdir ~/.vnc
$ vncpasswd
$ touch ~/.vnc/xstartup
$ chmod +x ~/.vnc/xstartup
Edit ~/.vnc/xstartup and set it to
#!/bin/bash
xrdb $HOME/.Xresources
startxfce4 &
Next, start the VNC server. If you want to be smart you can make a systemd unit or something but this works for what I needed
$ vncserver -geometry 1920x1080 -localhost
Each vncserver session binds to 5900+n, where n is the display number (starts at 1). It only binds to localhost, so forward it over SSH.
$ ssh user@host -L 5901:localhost:5901
Then, you can connect to it. When prompted, provide the password set using vncpasswd
$ vncviewer localhost:5901
To kill the VNC server (replace 1 with the display number):
$ vncserver -kill :1
You can run a program over SSH in the VNC session by setting the $DISPLAY env var.
$ export DISPLAY=:1.0
$ xcalc