Skip to content

Instantly share code, notes, and snippets.

Avatar
💭
I may be slow to respond.

Akira Ajisaka aajisaka

💭
I may be slow to respond.
View GitHub Profile
@aajisaka
aajisaka / test-patch.log
Created Dec 1, 2020
test-patch with Yetus 0.13.0-RC4
View test-patch.log
➜ hadoop git:(trunk) ~/git/yetus/precommit/src/main/shell/test-patch.sh --personality="./dev-support/bin/hadoop.sh" --debug YARN-10507
[Tue Dec 1 20:40:54 JST 2020 DEBUG]: Importing /Users/aajisaka/git/yetus/precommit/src/main/shell/robots.d/azurepipelines.sh
[Tue Dec 1 20:40:54 JST 2020 DEBUG]: Importing /Users/aajisaka/git/yetus/precommit/src/main/shell/robots.d/buildkite.sh
[Tue Dec 1 20:40:54 JST 2020 DEBUG]: Importing /Users/aajisaka/git/yetus/precommit/src/main/shell/robots.d/circleci.sh
[Tue Dec 1 20:40:54 JST 2020 DEBUG]: Importing /Users/aajisaka/git/yetus/precommit/src/main/shell/robots.d/cirrusci.sh
[Tue Dec 1 20:40:54 JST 2020 DEBUG]: Importing /Users/aajisaka/git/yetus/precommit/src/main/shell/robots.d/githubactions.sh
[Tue Dec 1 20:40:54 JST 2020 DEBUG]: Importing /Users/aajisaka/git/yetus/precommit/src/main/shell/robots.d/gitlabci.sh
[Tue Dec 1 20:40:54 JST 2020 DEBUG]: Importing /Users/aajisaka/git/yetus/precommit/src/main/shell/robots.d/jenkins.sh
[Tue Dec 1 20:40:54 JST 2020 DEBUG]:
@aajisaka
aajisaka / TestDNS
Created Oct 27, 2020
Java 15 Hadoop test failures
View TestDNS
[INFO] Running org.apache.hadoop.net.TestDNS
[ERROR] Tests run: 12, Failures: 0, Errors: 5, Skipped: 0, Time elapsed: 0.403 s <<< FAILURE! - in org.apache.hadoop.net.TestDNS
[ERROR] testNullDnsServer(org.apache.hadoop.net.TestDNS) Time elapsed: 0.134 s <<< ERROR!
java.io.UncheckedIOException: java.net.SocketException: Unsupported address type
at java.base/sun.nio.ch.DatagramSocketAdaptor.connect(DatagramSocketAdaptor.java:120)
at java.base/java.net.DatagramSocket.connect(DatagramSocket.java:341)
at jdk.naming.dns/com.sun.jndi.dns.DnsClient.doUdpQuery(DnsClient.java:408)
at jdk.naming.dns/com.sun.jndi.dns.DnsClient.query(DnsClient.java:214)
at jdk.naming.dns/com.sun.jndi.dns.Resolver.query(Resolver.java:81)
at jdk.naming.dns/com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:434)
@aajisaka
aajisaka / stacktrace
Created Sep 9, 2020
KMS debug log (HADOOP-17255)
View stacktrace
2020-09-09 20:06:29,276 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedActionException as: <principal> (auth:KERBEROS)
java.io.IOException: Rename unsuccessful : 'hdfs://<cluster>/user/kms/kms.keystore_NEW' to 'hdfs://<cluster>/user/kms/kms.keystore_NEW_ORPHANED_1599649589221'
at org.apache.hadoop.crypto.key.JavaKeyStoreProvider.renameOrFail(JavaKeyStoreProvider.java:644)
at org.apache.hadoop.crypto.key.JavaKeyStoreProvider.flush(JavaKeyStoreProvider.java:541)
at org.apache.hadoop.crypto.key.KeyProviderExtension.flush(KeyProviderExtension.java:127)
at org.apache.hadoop.crypto.key.KeyProviderExtension.flush(KeyProviderExtension.java:127)
at org.apache.hadoop.crypto.key.KeyProviderExtension.flush(KeyProviderExtension.java:127)
at org.apache.hadoop.crypto.key.kms.server.KeyAuthorizationKeyProvider.flush(KeyAuthorizationKeyProvider.java:377)
at org.apache.hadoop.crypto.key.kms.server.KMS$1.run(KMS.java:157)
at org.apache.hadoop.cryp
View gist:0697a1a62cfd23a9ce7a473ca145c42c
2020-08-26 16:50:32,963 INFO org.apache.hadoop.hdfs.server.federation.store.impl.MembershipStoreImpl: NN registration state has changed: <dr01>-><ns01>:nn2:<nn02>-ACTIVE -> <dr02>-><ns01>:nn2:<nn02>-UNAVAILABLE
2020-08-26 16:50:32,980 WARN org.apache.hadoop.ipc.Client: Exception encountered while connecting to the server
java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:271)
at java.base/java.io.DataInputStream.readInt(DataInputStream.java:392)
at org.apache.hadoop.ipc.Client$IpcStreams.readResponse(Client.java:1881)
View maven.log
➜ hadoop git:(trunk) mvn clean process-sources javadoc:javadoc-no-fork -pl hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.4.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for org.jsonschema2pojo:jsonschema2pojo-maven-plugin is missing. @ line 402, column 15
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
@aajisaka
aajisaka / error.log
Created Aug 3, 2020
hadoop-hdfs-rbf javadoc java 11
View error.log
➜ hadoop git:(trunk) ✗ mvn process-sources javadoc:javadoc-no-fork -pl hadoop-hdfs-project/hadoop-hdfs-rbf
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.15
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 15
@aajisaka
aajisaka / error.log
Created Aug 3, 2020
hadoop-hdfs-client javadoc errors
View error.log
➜ hadoop git:(trunk) ✗ mvn process-sources javadoc:javadoc-no-fork -pl hadoop-hdfs-project/hadoop-hdfs-client
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.15
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 15
@aajisaka
aajisaka / hadoop-hdfs
Last active Aug 3, 2020
javadoc errors
View hadoop-hdfs
➜ hadoop git:(trunk) ✗ mvn process-sources javadoc:javadoc-no-fork -pl hadoop-hdfs-project/hadoop-hdfs
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.15
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 15
View error.log
$ mvn process-sources javadoc:javadoc-no-fork -pl hadoop-common-project/hadoop-common
(snip)
[INFO]
27 errors
100 warnings
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 42.681 s
[INFO] Finished at: 2020-08-03T11:09:22+09:00
@aajisaka
aajisaka / TestCombineFileInputFormat
Last active Jul 13, 2020
TestUserGroupMappingPlacementRule
View TestCombineFileInputFormat
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.mapred.TestCombineFileInputFormat
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.341 s - in org.apache.hadoop.mapred.TestCombineFileInputFormat
[INFO] Running org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
[ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 30.916 s <<< FAILURE! - in org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
[ERROR] testMissingBlocks(org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat) Time elapsed: 0.59 s <<< ERROR!
java.lang.ClassCastException: org.apache.hadoop.hdfs.DistributedFileSystem cannot be cast to org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat$MissingBlockFileSystem
at org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat.testMissingBlocks(TestCombineFileInputFormat.java:1655)