Skip to content

Instantly share code, notes, and snippets.

@suztomo
Created June 22, 2020 16:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save suztomo/aaebc34b3933eec3573e940aed517fe2 to your computer and use it in GitHub Desktop.
Save suztomo/aaebc34b3933eec3573e940aed517fe2 to your computer and use it in GitHub Desktop.
Linkage Checker 1.4.2 detects new linkage errors (compared to 1.4.0)
suztomo@suztomo:~/beam$ ./sdks/java/build-tools/beam-linkage-check.sh
HEAD is now at 4dfc8a1351 Merge pull request #11405: [BEAM-9747] Deprecate RedisIO.readAll() and add RedisIO.readKeyPatterns as a replacement
Mon 22 Jun 2020 03:59:53 PM UTC: Running linkage check (baseline) for beam-sdks-java-core in master
Mon 22 Jun 2020 03:59:53 PM UTC: to create a baseline (existing errors before change) build/linkagecheck/baseline-beam-sdks-java-core.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Wrote the linkage errors as exclusion file: build/linkagecheck/baseline-beam-sdks-java-core.xml
NOTE: This task published artifacts into your local Maven repository. You may want to remove them manually.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 28s
966 actionable tasks: 171 executed, 795 up-to-date
Mon 22 Jun 2020 04:00:22 PM UTC: Running linkage check (baseline) for beam-sdks-java-io-google-cloud-platform in master
Mon 22 Jun 2020 04:00:22 PM UTC: to create a baseline (existing errors before change) build/linkagecheck/baseline-beam-sdks-java-io-google-cloud-platform.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Wrote the linkage errors as exclusion file: build/linkagecheck/baseline-beam-sdks-java-io-google-cloud-platform.xml
NOTE: This task published artifacts into your local Maven repository. You may want to remove them manually.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1m 12s
966 actionable tasks: 171 executed, 795 up-to-date
Mon 22 Jun 2020 04:01:35 PM UTC: Running linkage check (baseline) for beam-runners-google-cloud-dataflow-java in master
Mon 22 Jun 2020 04:01:35 PM UTC: to create a baseline (existing errors before change) build/linkagecheck/baseline-beam-runners-google-cloud-dataflow-java.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Wrote the linkage errors as exclusion file: build/linkagecheck/baseline-beam-runners-google-cloud-dataflow-java.xml
NOTE: This task published artifacts into your local Maven repository. You may want to remove them manually.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1m 23s
966 actionable tasks: 171 executed, 795 up-to-date
Mon 22 Jun 2020 04:02:59 PM UTC: Running linkage check (baseline) for beam-sdks-java-io-hadoop-format in master
Mon 22 Jun 2020 04:02:59 PM UTC: to create a baseline (existing errors before change) build/linkagecheck/baseline-beam-sdks-java-io-hadoop-format.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Wrote the linkage errors as exclusion file: build/linkagecheck/baseline-beam-sdks-java-io-hadoop-format.xml
NOTE: This task published artifacts into your local Maven repository. You may want to remove them manually.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 58s
966 actionable tasks: 171 executed, 795 up-to-date
Previous HEAD position was 4dfc8a1351 Merge pull request #11405: [BEAM-9747] Deprecate RedisIO.readAll() and add RedisIO.readKeyPatterns as a replacement
Switched to branch 'BEAM-10298'
Your branch is up to date with 'suztomo/BEAM-10298'.
Mon 22 Jun 2020 04:03:58 PM UTC: Running linkage check (validate) for beam-sdks-java-core in BEAM-10298
Mon 22 Jun 2020 04:03:58 PM UTC: using baseline build/linkagecheck/baseline-beam-sdks-java-core.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
NOTE: This task published artifacts into your local Maven repository. You may want to remove them manually.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 28s
966 actionable tasks: 171 executed, 795 up-to-date
Mon 22 Jun 2020 04:04:27 PM UTC: Done: 0
Mon 22 Jun 2020 04:04:27 PM UTC: Running linkage check (validate) for beam-sdks-java-io-google-cloud-platform in BEAM-10298
Mon 22 Jun 2020 04:04:27 PM UTC: using baseline build/linkagecheck/baseline-beam-sdks-java-io-google-cloud-platform.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
NOTE: This task published artifacts into your local Maven repository. You may want to remove them manually.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1m 10s
966 actionable tasks: 171 executed, 795 up-to-date
Mon 22 Jun 2020 04:05:38 PM UTC: Done: 0
Mon 22 Jun 2020 04:05:38 PM UTC: Running linkage check (validate) for beam-runners-google-cloud-dataflow-java in BEAM-10298
Mon 22 Jun 2020 04:05:38 PM UTC: using baseline build/linkagecheck/baseline-beam-runners-google-cloud-dataflow-java.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
NOTE: This task published artifacts into your local Maven repository. You may want to remove them manually.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1m 11s
966 actionable tasks: 171 executed, 795 up-to-date
Mon 22 Jun 2020 04:06:50 PM UTC: Done: 0
Mon 22 Jun 2020 04:06:50 PM UTC: Running linkage check (validate) for beam-sdks-java-io-hadoop-format in BEAM-10298
Mon 22 Jun 2020 04:06:50 PM UTC: using baseline build/linkagecheck/baseline-beam-sdks-java-io-hadoop-format.xml
Configuration on demand is an incubating feature.
> Task :checkJavaLinkage
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
(org.apache.commons:commons-compress:1.4.1) org.apache.commons.compress.archivers.zip.ExtraFieldUtils's method class$(String arg1) is not found;
referenced by 1 class file
org.vafer.jdeb.shaded.compress.compress.archivers.zip.ExtraFieldUtils (org.vafer:jdeb:0.8)
org.apache.commons:commons-compress:1.4.1 is at:
org.apache.beam:beam-sdks-java-io-hadoop-format:jar:2.23.0-SNAPSHOT / org.apache.hadoop:hadoop-common:2.8.5 (provided) / org.apache.commons:commons-compress:1.4.1 (compile)
and 127 other dependency paths.
org.vafer:jdeb:0.8 is at:
org.apache.beam:beam-sdks-java-io-hadoop-format:jar:2.23.0-SNAPSHOT / org.apache.hadoop:hadoop-common:2.8.5 (provided) / org.apache.zookeeper:zookeeper:3.4.6 (compile) / org.vafer:jdeb:0.8 (compile, optional)
and 745 other dependency paths.
com.google.inject:guice:jar:3.0 was not resolved. Dependency path: org.apache.beam:beam-sdks-java-io-hadoop-format:jar:2.23.0-SNAPSHOT (compile) > org.apache.beam:beam-sdks-java-io-hadoop-common:jar:2.23.0-SNAPSHOT (compile) > org.apache.hadoop:hadoop-client:jar:2.8.5 (provided) > org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.8.5 (compile) > org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.8.5 (compile) > org.apache.hadoop:hadoop-yarn-common:jar:2.8.5 (compile) > com.google.inject:guice:jar:3.0 (compile)
Exception in thread "main" com.google.cloud.tools.opensource.classpath.LinkageCheckResultException: Found 1 linkage error
at com.google.cloud.tools.opensource.classpath.LinkageCheckerMain.main(LinkageCheckerMain.java:135)
> Task :checkJavaLinkage FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkJavaLinkage'.
> Process 'command '/usr/local/buildtools/java/jdk8-google-v7-64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 13s
966 actionable tasks: 171 executed, 795 up-to-date
Mon 22 Jun 2020 04:08:03 PM UTC: Done: 1
There's new linkage errors. See above for details.
suztomo@suztomo:~/beam$ git diff origin/master
diff --git a/build.gradle b/build.gradle
index 489a9d92cc..b15cdfa3c7 100644
--- a/build.gradle
+++ b/build.gradle
@@ -329,7 +329,7 @@ if (project.hasProperty('javaLinkageArtifactIds')) {
configurations { linkageCheckerJava }
dependencies {
- linkageCheckerJava "com.google.cloud.tools:dependencies:1.4.0"
+ linkageCheckerJava "com.google.cloud.tools:dependencies:1.4.2"
}
// We need to evaluate all the projects first so that we can find depend on all the
diff --git a/sdks/java/build-tools/beam-linkage-check.sh b/sdks/java/build-tools/beam-linkage-check.sh
index fb4d053f31..0ed30c96f2 100755
--- a/sdks/java/build-tools/beam-linkage-check.sh
+++ b/sdks/java/build-tools/beam-linkage-check.sh
@@ -72,10 +72,6 @@ function runLinkageCheck () {
COMMIT=$1
BRANCH=$2
MODE=$3 # "baseline" or "validate"
- # An empty invocation so that the subsequent checkJavaLinkage does not
- # contain garbage
- echo "`date`:" "Installing artifacts of ${BRANCH}(${COMMIT}) to Maven local repository."
- ./gradlew -Ppublishing -PjavaLinkageArtifactIds=beam-sdks-java-core -PjavaLinkageWriteBaseline=/dev/null :checkJavaLinkage > /dev/null 2>&1
for ARTIFACT in $ARTIFACTS; do
echo "`date`:" "Running linkage check (${MODE}) for ${ARTIFACT} in ${BRANCH}"
suztomo@suztomo:~/beam$
@suztomo
Copy link
Author

suztomo commented Jun 22, 2020

https://search.maven.org/artifact/org.vafer/jdeb/0.8/maven-plugin has outdated commons-compress

    <dependencies>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-compress</artifactId>
            <version>1.0</version>
        </dependency>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment