The goal is to provide a safe shutdown of a management server when running in cluster mode, so that the server stops processing new commands and waits for ongoing commands before being "ready to stop". Then the MS should be in a clean state for a shutdown. A little requirement is that the proxy in front of the management servers should have its route to the server going down turned off to avoid routing reconnection to the stopping server.
-
add a API call to request a graceful shutdown of a MS, which will not be received by a different MS (Since the proxy will not direct this call to the corresponding MS)
-
add a new flag for the MS state with value
STOPPING -
inform the clusterManager that it is going down
-
check ongoing jobs that have a return path to the management server and wait for them to come
-
disconnect from agents without pending jobs, and disconnect from the other as soon as the job answer has arrived.
-
stop all local threads executors (stop picking up new async job)