You don't want Xorg to use your NVIDIA gpu. Need to blacklist nouveau and nvidia-drm.
In the created file /etc/modprobe.d/blacklist-nouveau.conf:
blacklist nouveau
options nouveau modeset=0
In the file /etc/modprobe.d/blacklist-nvidia-drm.conf:
blacklist nvidia-drm