When running a REX job, the flow is roughly as follows:
- The user triggers a job, under the hood this trigger a "parent task"
- The parent task goes in batches of 100 and creates a sub-task for each host in the job. The batches are processed sequentially. Inside the batches the sub-tasks are also created sequentially.
- After each batch from step 2 is prepared, the prepared sub-tasks are delegated to the smart proxy.
- The smart proxy unwraps the batch and spawns a single ssh process for each, all of these run completely independently from each other.
- The smart proxy reports back to Foreman as the per-host jobs finish with one request per host.
- When the reports come to Foreman, Foreman dispatches them to the relevant per-host sub-tasks from 2. These sub-tasks process the updates and exit.