This page explains how to reproduce the Gradle vs Maven performance numbers yourself. For that, you need to install the Gradle profiler, a tool which will automate benchmarking by running several builds in a row, with the appropriate options (it also runs Maven builds).
Our performance comparison uses 4 test projects:
- a large, monolithic app
- single project
- 50000 source files and 50000 test source files
- a medium sized, multi-project build
- 100 subprojects
- each subproject has 100 source files and 100 test source files
- a large, multi-project build
- 500 subprojects
- each subproject has 100 source files and 100 test source files
- Apache Commons Lang
The first 3 can be found in the Gradle repository. To generate a test project, checkout the Gradle sources, then execute:
./gradlew largeMonolithicJavaProject
for the large monolithic build./gradlew mediumJavaMultiProject
for the medium multiproject build./gradlew largeJavaMultiProject
for the large multiproject build
The test projects will then be found in subprojects/performance/build/<test project name>
.
The last project is a real-world project (Apache Commons Lang that needs to be setup separately:
git clone https://github.com/gradle/performance-comparisons.git commons-lang --branch commons-lang --depth 1
cd commons-lang
This will clone the repository which is a copy of the original Commons Lang repository, but containing our converted to Gradle build and the scenario file.
Make sure you have installed the Gradle Profiler and that it's available on your PATH.
Change directory to a test project and run:
gradle-profiler --gradle-version 4.0 --benchmark --scenario-file performance.scenarios --project-dir .
This will execute all scenarios by default, which can take a significant amount of time. If you want to run one or more specific scenarios, just add them explicitly to the command line:
gradle-profiler --gradle-version 4.0 --benchmark --scenario-file performance.scenarios --project-dir . <scenario1> <scenario2> ...
First make sure you have properly set the MAVEN_HOME
and MAVEN_OPTS
environment variables:
export MAVEN_HOME=/path/to/apache-maven-3.5.0
export MAVEN_OPTS="-Xms2g -Xmx2g"
Then execute:
gradle-profiler --benchmark --scenario-file performance.scenarios --project-dir . --maven --warmups 2 --iterations 2
Note: For the large monolithic case, increase Maven memory to 3g instead of 2g, otherwise it won't run. Gradle will use 2g for this scenario.
You will notice that we only use 2 warmups and runs for Maven. There are multiple reasons for this:
- Maven doesn't have a daemon, so doesn't benefit from subsequent runs of the same build
- Maven is significantly slower. A test that would take minutes to complete with Gradle would take several hours with Maven
You will notice that for Apache Maven, we run clean compile
instead of compile. It may, at first glance, look like
an unfair comparison, but it's for an important reason: Maven's incremental compiler is completely broken and will miss changes. An example of serious bug can be found here.
What is the parallelism setting for maven? (The -T option)