I've found a couple places that need the change.
In ~/.Xresources, add something like Xcursor.size: 16
. This fixed many issues for me, but chromium still had a large cursor when I hovered over it. A quick change to gsettings fixed it: gsettings set org.gnome.desktop.interface cursor-size 16
.