If you installed java with the Oracle installer, get rid of it:
sudo rm -rf /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin
sudo rm -rf /Library/PreferencePanes/JavaControlPanel.prefPane
sudo rm -rf /Library/Application\ Support/Oracle/Java/
sudo rm -rf /Library/Java/JavaVirtualMachines
If you installed java with brew, remove previously brew installed version of java
brew cask uninstall java