This page explains how to reproduce the Gradle vs Maven performance numbers yourself. For that, you need to install the Gradle profiler, a tool which will automate benchmarking by running several builds in a row, with the appropriate options (it also runs Maven builds).
Our performance comparison uses 4 test projects:
- a large, monolithic app
- single project
- 50000 source files and 50000 test source files
- a medium sized, multi-project build
- 100 subprojects
- each subproject has 100 source files and 100 test source files
- a large, multi-project build
- 500 subprojects
- each subproject has 100 source files and 100 test source files
- Apache Commons Lang
The first 3 can be found in the Gradle repository. To generate a test project, checkout the Gradle sources, then execute:
./gradlew largeMonolithicJavaProject
for the large monolithic build./gradlew mediumJavaMultiProject
for the medium multiproject build./gradlew largeJavaMultiProject
for the large multiproject build
The test projects will then be found in subprojects/performance/build/<test project name>
.
The last project is a real-world project (Apache Commons Lang that needs to be setup separately:
git clone https://github.com/gradle/performance-comparisons.git commons-lang --branch commons-lang --depth 1
cd commons-lang
This will clone the repository which is a copy of the original Commons Lang repository, but containing our converted to Gradle build and the scenario file.
Make sure you have installed the Gradle Profiler and that it's available on your PATH.
Change directory to a test project and run:
gradle-profiler --gradle-version 4.0 --benchmark --scenario-file performance.scenarios --project-dir .
This will execute all scenarios by default, which can take a significant amount of time. If you want to run one or more specific scenarios, just add them explicitly to the command line:
gradle-profiler --gradle-version 4.0 --benchmark --scenario-file performance.scenarios --project-dir . <scenario1> <scenario2> ...
First make sure you have properly set the MAVEN_HOME
and MAVEN_OPTS
environment variables:
export MAVEN_HOME=/path/to/apache-maven-3.5.0
export MAVEN_OPTS="-Xms2g -Xmx2g"
Then execute:
gradle-profiler --benchmark --scenario-file performance.scenarios --project-dir . --maven --warmups 2 --iterations 2
Note: For the large monolithic case, increase Maven memory to 3g instead of 2g, otherwise it won't run. Gradle will use 2g for this scenario.
You will notice that we only use 2 warmups and runs for Maven. There are multiple reasons for this:
- Maven doesn't have a daemon, so doesn't benefit from subsequent runs of the same build
- Maven is significantly slower. A test that would take minutes to complete with Gradle would take several hours with Maven
You will notice that for Apache Maven, we run clean compile
instead of compile. It may, at first glance, look like
an unfair comparison, but it's for an important reason: Maven's incremental compiler is completely broken and will miss changes. An example of serious bug can be found here.
Now let's talk about incremental compilation.
Yes, maven does not have it by default and if you want it you will need to use 3rd party module - http://takari.io/book/40-lifecycle.html with jdt compiler. And yes, gradle does have it by default without the need of external components.
Let's look at the example. Imagine situation when I change only 5 classes (e.g. add new property) in the 5 different modules:
For gradle this means detect changes and recompile only 5 classes. For maven without takari-lifecycle this means detect changes and recompile 500 files since default maven configuration recompiles entire module if it detects even simple change.
For maven execution takes about 11 seconds
mvn -T 4 compile
.Now I execute following commands
after this I modify same 5 files from different modules and run compilation with maven.
There is no difference at all for such scenario. I mean recompiling all classes of 5 modules with maven takes 11 seconds and detecting the change and recompiling only 5 files from different modules takes same 11-12 seconds. By the way, when you configure takari-lifecycle with incremental build for maven it will still be around same 12 seconds :-) of compilation time.
I think that it is still possible to come up with some scenario where gradle will be slightly faster. But that would be synthetic scenario. For me personally modification of 5 different modules at once is covering most of real world scenarios any developer will have. Also do not forget that with maven recompiling entire project with 100 modules and 10000 Java source files takes less than 35 seconds is also very fast.
Gradle is more actively developed compared to maven. But still it has bugs. You have link to maven bug. But running
is:issue is:open incremental label:a:bug
query today (3/12/2021) I see a number of open defects with interesting names likeSo in reality gradle users will still be doing clean build to avoid issues.