Please join the Configuration team to improve our test coverage by making the entire Gradle test suite compatible with the configuration cache feature.
We will create Github issues for each platform. We hope each team will look into fixing the tests in the platforms they own.
The configuration cache feature imposes certain constraints on code that executes during the execution phase (see CC requirements). In order to ensure our test suites abide by those rules, as one of the PR validation steps in Github, the tests are run with configuration cache enabled, via the configCacheIntegTest task, which enables the ConfigurationCacheGradleExecuter.
A good number of our tests violate those constraints, and as such, as a strategy for incrementally migrating them towards being CC-compatible, they have been marked as known (and expected to) fail with the configuration cache enabled, by being annotated with @ToBeFixedForConfigurationCache.
Over time, as the configuration cache implementation evolved, many of the tests originally marked as known to fail under CC have been migrated to be CC-ready, so the annotation was removed. However, when we introduced load-after-store by default (in 8.0), we discovered that a large number of tests that at first seemed compatible with the configuration cache started to fail (see FAQ below for an explanation). Again, in order to incrementally migrate them to be fully CC-compatible, as a temporary workaround, instead of marking each offending test with @ToBeFixedForConfigurationCache, we disabled load-after-store at the subproject level for any projects that contained such tests (examples). So, they still run when configuration cache is enabled, but the load-after-store feature is off when running all tests in that subproject. That is not ideal because Gradle runs with load-after-store enabled, so we are potentially masking actual bugs that way.
Our goal in this initiative is to ensure that the workaround is no longer required. Given that the number of projects and set of tests that need to be migrated is non-trivial, we propose to do that as a team effort, where each team will address issues in subprojects and platforms they own.
The Configuration team will welcome any questions teams may have about addressing test issues in the #bt-configuration-cache channel.
Configuration cache's load-after-store approach was released in 8.0. load-after-store changes the way Gradle works when configuration cache is enabled.
Before load-after-store:
- On a cold cache, Gradle with CC enabled would behave more or less like when CC was off, but with a cache store operation at the end of the build (after the execution phase).
- On a warm cache, Gradle would not run any code meant to run during configuration, instead loading the configuration state from the CC cache in disk, before starting the execution phase
After load-after-store:
- On a cold cache, Gradle will run the configuration phase, store the state to the CC cache, discard the in-memory configuration state, and then load the configuration state from the CC cache in disk, before starting the execution phase
- On a warm cache, the same as before
As a result, with load-after-store, the behavior between a cold and a warm cache is much closer than before, which is a major benefit: we go from three modes (no-CC, CC on a cache miss, CC on a cache hit) to two (no-CC and CC).
Most typically, they were only passing with the CC-enabled executer because they never triggered configuration cache loads (most integration tests run Gradle only once). With load-after-store, the behavior of a cold and a warm cache run are much closer, so now we expose issues which before would only surface on a warm cache.
That is required so configuration cache is enabled during testing.
That is the default, however we have an internal system property to disable load-after-store in tests on a project basis. Check the corresponding build.gradle file. If it has a section like the one below, then remove it or comment it out (setting it to "false" does not work).
// Remove as part of fixing .../gradle/configuration-cache/issues/585
tasks.configCacheIntegTest {
systemProperties["...test-disable-load-after-store"] = "true"
}
See Configuration cache requirements.
Example of a common failure:
#### Cannot reference a Gradle script object from a Groovy closure as these are not supported with the configuration cache.
In code such as:
task verify {
doLast {
assert configurations.customCompileClasspath.state.toString() == "UNRESOLVED"
assert configurations.customRuntimeClasspath.state.toString() == "UNRESOLVED"
}
}
Which could be fixed by changing it to something like this:
task verify {
def customCompileClasspathState = provider {
configurations.customCompileClasspath.state.toString()
}
def customRuntimeClasspathState = provider {
configurations.customRuntimeClasspath.state.toString()
}
doLast {
assert customCompileClasspathState.get() == "UNRESOLVED"
assert customRuntimeClasspathState.get() == "UNRESOLVED"
}
}
IOW, by accessing at configuration time any API that is not supported by CC at execution time.
Tests that expect for failures tend to be fragile in a sense they often assume more than users care about the fine details of how Gradle reports invalid build configuration, essentially expecting some error output verbatim.
The test relies on configuring data that is not properly serialized - it is either missing or incorrectly implemented. There are two possibilities here:
- Serialization in Configuration Cache itself is broken (produces exceptions, or restores objects wrongly with missing data, or with the wrong concrete type. In this case, a bug report against the Configuration team is a good outcome (please use
@configuration-cacheandin-configuration-cacheas issue labels). - Some built-in plugin defines a task that contains objects that are incompatible with Configuration Cache. In this case, the ideal outcome is a PR that ensures only CC-friendly objects need to be referred to by the task.
The test relies on an API that is incompatible with the Configuration Cache and has no CC-friendly counterpart. For these, the best outcome we can hope for is a github issue with a clear description of the use case.