So I had problems install Docker on an RPI 4, using the usual routes caused hassle and issues, so I went on a mission to fix it!
Here are the steps:
It turns out the installed iptables is kind of broken, so it needs to be re-installed correctly.
sudo apt purge iptables
sudo apt install iptables -y
Once thats complete, we need to reboot the machine to load all the kernal modules.