CamiloGarciaLaRotta/maven-plugin-testing-rant

## maven-plugin-testing-rant
Recall Maven strictly follows the lifecycles: validate, compile, test, package, verify, install, deploy.
Attempting to switch the order of any of the lifecycles can only be done through hacky approaches
(for example, the way we override the default maven-compiler by setting the compile lifecycle phase
of the compiler to "None" and we push the compilation to the validation phase)

Here's the resume of my adventures testing Maven plugins:

Maven-verifier: can execute arbitrary lifecycles on a dummy project. Hence can assert on its artifacts, on stdout/err and logfiles
issue: the dummy project requires the plugin under test to already be available in the local artifact repository.
but due to the lifecycle order, the integration tests (verify phase) are run before the artifact is stored in the local repo (install)
hence we would need to run twice the test: once to install the new version, the next to actually test.
This is aweful, we override our local working copy of the plugin with a potentially broken one.
I was unable to find a way to inject the plugin-under-test (not the one in .m2 repository) into the dummy project.

Maven-invoker-plugin: invoke maven on a dummy project trhough a pom file (its is currently used by BuildMedic to recompile after fix injection)
issue: adds a complete new technological weight as it doesnt support Junit, only Groovy.
it does not solve the issue of injecting the plugin artifact to the testing environment

Maven-plugin-testing-harness: abstract class for testing plugins with a collection of stubs.
We can manipulate the project as an object, so we get access to all the pom file attributes and to a certain extent, to the project's resources
Although it is not meant for integration tests, its usage is complex and the documentation is often deprecated.

Junit: no access to Maven project entities as objects.
creating test cases becomes very archaic for even the simplest Maven related tasks.
Would only be able to test methods that don't leverage maven, e.g. URL manipulation, string manipulation, file R/W

Conclusion: I will use JUnit for the very little number of methods that are unrelated to Maven
I will have to dive into Maven-plugin-testing-harness to figure out how to unit test Maven related methods.
Hopefully the union of both test sets will yield a reasonable coverage.
It is a shame that integration tests cant be easily done,
I believe 1 e2e provides more bang for your buck in terms of confidence that the plugin works as intended
	Recall Maven strictly follows the lifecycles: validate, compile, test, package, verify, install, deploy.
	Attempting to switch the order of any of the lifecycles can only be done through hacky approaches
	(for example, the way we override the default maven-compiler by setting the compile lifecycle phase
	of the compiler to "None" and we push the compilation to the validation phase)

	Here's the resume of my adventures testing Maven plugins:

	Maven-verifier: can execute arbitrary lifecycles on a dummy project. Hence can assert on its artifacts, on stdout/err and logfiles
	issue: the dummy project requires the plugin under test to already be available in the local artifact repository.
	but due to the lifecycle order, the integration tests (verify phase) are run before the artifact is stored in the local repo (install)
	hence we would need to run twice the test: once to install the new version, the next to actually test.
	This is aweful, we override our local working copy of the plugin with a potentially broken one.
	I was unable to find a way to inject the plugin-under-test (not the one in .m2 repository) into the dummy project.

	Maven-invoker-plugin: invoke maven on a dummy project trhough a pom file (its is currently used by BuildMedic to recompile after fix injection)
	issue: adds a complete new technological weight as it doesnt support Junit, only Groovy.
	it does not solve the issue of injecting the plugin artifact to the testing environment

	Maven-plugin-testing-harness: abstract class for testing plugins with a collection of stubs.
	We can manipulate the project as an object, so we get access to all the pom file attributes and to a certain extent, to the project's resources
	Although it is not meant for integration tests, its usage is complex and the documentation is often deprecated.

	Junit: no access to Maven project entities as objects.
	creating test cases becomes very archaic for even the simplest Maven related tasks.
	Would only be able to test methods that don't leverage maven, e.g. URL manipulation, string manipulation, file R/W

	Conclusion: I will use JUnit for the very little number of methods that are unrelated to Maven
	I will have to dive into Maven-plugin-testing-harness to figure out how to unit test Maven related methods.
	Hopefully the union of both test sets will yield a reasonable coverage.
	It is a shame that integration tests cant be easily done,
	I believe 1 e2e provides more bang for your buck in terms of confidence that the plugin works as intended