swarm join will use the address you provide only during initial connection.
Once connected, the manager will give the worker the list of all managers (including itself) along with their
address (as defined in advertise address).
If a manager goes down, its workers will randomly reconnect to another manager based on that list. When a new manager joins, its advertise address is broadcasted to all workers.
Advertise address is also used for Raft. Raft configuration has the list of all managers' advertised addresses. When a manager joins raft, it gets the list of all other managers and connect to them to sync data.