In Linux, when you add an IP to an interface, the kernel creates two routes for you:
table local: local x.x.x.y dev foo proto kernel scope host src x.x.x.y
table main: x.x.x.a/bb dev foo proto kernel scope link src x.x.x.y
Now, if you are setting up an HA pair or cluster, you will often have a VIP -- a "virtual" or "floating" IP -- which is moved between boxes during failovers. And if you happen to be running clients on these nodes as well which connect to that VIP, something very odd happens when you move the IP.
So... linux has routing rules, tables, and a cache. When a connection is made, the cache is consulted for a matching route tuple (src, dst, tos, fwmark, iif) and if it exists, the connection stores a pointer to it so each packet can rapidly be routed. If the cache entry expires or otherwise goes away, a new route is cloned by following the policy rules to look in the tables.
Now... when an IP you're connected to/from goes away... something very odd happens. The stack realizes that it can