ConcurrentMaven3Reloaded

h1. Concurrent Maven 3 reloaded

h2. High-level Requirements

To achieve robust concurrency, such an implementation would need:
- Transactional install to build-local repo of all artifacts. These will be written to local build upon completion.
- Log output must be handled properly

h2. Eexcution modes

For all the figures below, the number in the box indicates build order. The same number occuring multiple places
means concurrent execution.

h3. Linear mode build (classic) - Mode 0

|| || validate \\ || compile \\ || test \\ || package \\ || integration-test \\ || install \\ || deploy \\ ||
| module1 \\ | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
| module2 \\ | 8 | 9 | 10 | 11 \\ | 12 | 13 | 14 |
| module3 \\ | 15 | 16 | 17 | 18 | 19 | 20 | 21 |
| module4 \\ | 22 | 23 | 24 | 25 | 26 | 27 | 28 |

h3. Weave mode build - Mode 1

|| || validate \\ || compile \\ || test \\ || package \\ || integration-test \\ || install \\ || deploy \\ ||
| module1 \\ | 1 | 5 | 9 \\ | 13 | 17 | 21 | 25 |
| module2 \\ | 2 | 6 | 10 | 14 | 18 | 22 | 26 |
| module3 \\ | 3 | 7 | 11 | 15 | 19 | 23 | 27 |
| module4 \\ | 4 | 8 | 12 | 16 | 20 | 24 | 28 |
Weave-mode build is considered a precondition to achieving any of the real concurrency

targets.

h3. Concurrent Simple Weave mode build - Mode 2

In this build mode, a lifecyclephase either accepts concurrent execution or not.

|| || validate \\ || compile \\ || test \\ || package \\ || integration-test \\ || install \\ || deploy \\ ||
| module1 \\ | 1 | 5 | 9 | 10 | 11 | 12 | 13 |
| module2 \\ | 2 | 6 | 9 | 10 | 11 | 12 | 13 |
| module3 \\ | 3 | 7 | 9 | 10 | 11 | 12 | 13 |
| module4 \\ | 4 | 8 | 9 | 10 | 11 | 12 | 13 |

The disadvantage of this model is that any single phase taking a long time will effectively become single-threaded (integration in a single module tests taking a long time)

h3. Concurrent Phase-Crossing Weave mode - Mode 3

|| || validate \\ || compile \\ || test \\ || package \\ || integration-test \\ || install \\ || deploy \\ ||
| module1 \\ | 1 | 5 | 9 | 9 | 9 | 10 | 11 |
| module2 \\ | 2 | 6 | 9 | 9 | 9 | 10 | 11 |
| module3 \\ | 3 | 7 | 9 | 9 | 9 | 10 | 11 |
| module4 \\ | 4 | 8 | 9 | 9 | 9 | 10 | 11 |

Mode 3 is basically the same as mode 2, but allows running jobs in subsequent lifecycles in parallel.

h3. Concurrent Combined Mode   - Mode 4

Combines mode 0 and mode 2 (I *hope* I've gotten the numbers right.). To read this graph properly you have to think "at completion of task 1, I can start
all the jobs numbered 2" and so on.

|| || validate \\ || compile \\ || test \\ || package \\ || integration-test \\ || install \\ || deploy \\ ||
| module1 \\ | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
| module2 \\ | 2 | 5 | 8 | 9 | 10 | 11 | 12 |
| module3 \\ | 3 | 6 | 8 | 9 | 10 | 11 | 12 |
| module4 \\ | 4 | 7 | 8 | 9 | 10 | 11 | 12 |


Mode4 should not degrade on long-running integration tests, since it attacks both the linear classic build and the weaved build.
Another advantage of mode 4 is that it actually *has* output to provide to the user all of the time, and output could be presented in
classic linear fashion. (The build will seem to start slowly and seem to pick up speed around 8)


h2. Log handling

It is important that log output be presented to the user in a consistent manner, even though the build may be running in an "odd" manner.
Furthermore it is not acceptable to have huge periods of no build output in normal-cases.

Some options are:

* Exactly as-is today (Presents a problem For all modes but mode 4, which should handle this well)
* Per-phase/per plugin consistency (all output from a phase/plugin comes as one block)
* May require some changes to log output format when running multithreaded.

Some serious multiplexing of log output will be necessary in all cases.

h2. Issues to be ironed out

A multithreaded build *must* not support all the regular modes of maven operation,
and these various gotchas will have to be checked out:

* Correctly implementing reactor failure behavior (--fail-fast,
\--fail-at-end, \--fail-never) with blacklisting
* What happens if we specify multiple lifecycles?  "mvn compile test"
* What happens if we just specify the raw goals? "mvn
myPlugin:goal"
* What if we mix and match? "mvn compile myPlugin:goal test"
* What if we put clean last?  Would we clean projects while later
projects depend on them? "mvn compile clean"
* What if the reactor is building a plugin that is used later in the
reactor?
* How would users resume "weave" mode?  (Today we allow users to
\--resume-from a particular project.)  Would "weave" users resume from a
particular project+phase?  Would resuming even be reasonable?  If you
changed a class, you'd need to recompile it and THEN retest it...