Skip to content

Instantly share code, notes, and snippets.

@wlach
Created September 18, 2019 19:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save wlach/c2ecebc1bb03fd37c517daa64424ee7d to your computer and use it in GitHub Desktop.
Save wlach/c2ecebc1bb03fd37c517daa64424ee7d to your computer and use it in GitHub Desktop.
Hammersmith:ingestion-beam wlach$ ./bin/mvn compile exec:java -Dexec.args="\
--inputType=heka \
--input=1567279450_36_ip-172-31-23-39.gz \
--outputFileFormat=text \
--outputType=stdout \
> "
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] ingestion [pom]
[INFO] ingestion-core [jar]
[INFO] ingestion-beam [jar]
[INFO]
[INFO] ------------------< com.mozilla.telemetry:ingestion >-------------------
[INFO] Building ingestion 0.1-SNAPSHOT [1/3]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (default) @ ingestion ---
[INFO] argLine set to -javaagent:/var/maven/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/var/maven/project/target/jacoco.exec -Xmx1024m -XX:MaxPermSize=256m -Djdk.net.URLClassPath.disableClassPathURLCheck=true
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ ingestion ---
[INFO] skipping execute as per configuration
[INFO]
[INFO] ----------------< com.mozilla.telemetry:ingestion-core >----------------
[INFO] Building ingestion-core 0.1-SNAPSHOT [2/3]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (default) @ ingestion-core ---
[INFO] argLine set to -javaagent:/var/maven/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/var/maven/project/ingestion-core/target/jacoco.exec -Xmx1024m -XX:MaxPermSize=256m -Djdk.net.URLClassPath.disableClassPathURLCheck=true
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ ingestion-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /var/maven/project/ingestion-core/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ ingestion-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4 source files to /var/maven/project/ingestion-core/target/classes
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ ingestion-core ---
[INFO] skipping execute as per configuration
[INFO]
[INFO] ----------------< com.mozilla.telemetry:ingestion-beam >----------------
[INFO] Building ingestion-beam 0.1-SNAPSHOT [3/3]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (default) @ ingestion-beam ---
[INFO] argLine set to -javaagent:/var/maven/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/var/maven/project/ingestion-beam/target/jacoco.exec -Xmx1024m -XX:MaxPermSize=256m -Djdk.net.URLClassPath.disableClassPathURLCheck=true
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ ingestion-beam ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /var/maven/project/ingestion-beam/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ ingestion-beam ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 63 source files to /var/maven/project/ingestion-beam/target/classes
[INFO] /var/maven/project/ingestion-beam/src/main/java/com/mozilla/telemetry/avro/BinaryRecordFormatter.java: Some input files use or override a deprecated API.
[INFO] /var/ma ven/project/ingestion-beam/src/main/java/com/mozilla/telemetry/avro/BinaryRecordFormatter.java: Recompile with -Xlint:deprecation for details.
[INFO] /var/maven/project/ingestion-beam/src/main/java/com/mozilla/telemetry/transforms/PubsubMessageToTableRow.java: /var/maven/project/ingestion-beam/src/main/java/com/mozilla/telemetry/transforms/PubsubMessageToTableRow.java uses unchecked or unsafe operations.
[INFO] /var/maven/project/ingestion-beam/src/main/java/com/mozilla/telemetry/transforms/PubsubMessageToTableRow.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ ingestion-beam ---
[WARNING]
org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder$PopulateDisplayDataException: Error while populating display data for component 'org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Write': null
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:791)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.delegate (DisplayData.java:747)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Write$PubsubBoundedWriter.populateDisplayData (PubsubIO.java:1122)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:782)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:740)
at org.apache.beam.sdk.transforms.ParDo.populateDisplayData (ParDo.java:910)
at org.apache.beam.sdk.transforms.ParDo.access$100 (ParDo.java:388)
at org.apache.beam.sdk.transforms.ParDo$MultiOutput.populateDisplayData (ParDo.java:871)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:782)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.access$100 (DisplayData.java:710)
at org.apache.beam.sdk.transforms.display.DisplayData.from (DisplayData.java:78)
at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.translateAppliedPTransform (PTransformTranslation.java:395)
at org.apache.beam.repackaged.direct_java.runners.core.construction.ParDoTranslation$ParDoTranslator.translate (ParDoTranslation.java:119)
at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.toProto (PTransformTranslation.java:155)
at org.apache.beam.repackaged.direct_java.runners.core.construction.ParDoTranslation.getParDoPayload (ParDoTranslation.java:650)
at org.apache.beam.repackaged.direct_java.runners.core.construction.ParDoTranslation.isSplittable (ParDoTranslation.java:665)
at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformMatchers$6.matches (PTransformMatchers.java:269)
at org.apache.beam.sdk.Pipeline$2.visitPrimitiveTransform (Pipeline.java:282)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:665)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600 (TransformHierarchy.java:317)
at org.apache.beam.sdk.runners.TransformHierarchy.visit (TransformHierarchy.java:251)
at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:460)
at org.apache.beam.sdk.Pipeline.replace (Pipeline.java:260)
at org.apache.beam.sdk.Pipeline.replaceAll (Pipeline.java:210)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:170)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:67)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:315)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:301)
at com.mozilla.telemetry.Sink.run (Sink.java:69)
at com.mozilla.telemetry.Sink.run (Sink.java:44)
at com.mozilla.telemetry.Sink.main (Sink.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.NullPointerException
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$PubsubTopic.fromPath (PubsubIO.java:367)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$TopicTranslator.apply (PubsubIO.java:309)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$TopicTranslator.apply (PubsubIO.java:305)
at org.apache.beam.sdk.options.ValueProvider$NestedValueProvider.get (ValueProvider.java:129)
at org.apache.beam.sdk.transforms.display.DisplayData.item (DisplayData.java:867)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO.populateCommonDisplayData (PubsubIO.java:166)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO.access$2000 (PubsubIO.java:90)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Write.populateDisplayData (PubsubIO.java:1033)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:782)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.delegate (DisplayData.java:747)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Write$PubsubBoundedWriter.populateDisplayData (PubsubIO.java:1122)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:782)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:740)
at org.apache.beam.sdk.transforms.ParDo.populateDisplayData (ParDo.java:910)
at org.apache.beam.sdk.transforms.ParDo.access$100 (ParDo.java:388)
at org.apache.beam.sdk.transforms.ParDo$MultiOutput.populateDisplayData (ParDo.java:871)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include (DisplayData.java:782)
at org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.access$100 (DisplayData.java:710)
at org.apache.beam.sdk.transforms.display.DisplayData.from (DisplayData.java:78)
at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.translateAppliedPTransform (PTransformTranslation.java:395)
at org.apache.beam.repackaged.direct_java.runners.core.construction.ParDoTranslation$ParDoTranslator.translate (ParDoTranslation.java:119)
at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.toProto (PTransformTranslation.java:155)
at org.apache.beam.repackaged.direct_java.runners.core.construction.ParDoTranslation.getParDoPayload (ParDoTranslation.java:650)
at org.apache.beam.repackaged.direct_java.runners.core.construction.ParDoTranslation.isSplittable (ParDoTranslation.java:665)
at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformMatchers$6.matches (PTransformMatchers.java:269)
at org.apache.beam.sdk.Pipeline$2.visitPrimitiveTransform (Pipeline.java:282)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:665)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:657)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600 (TransformHierarchy.java:317)
at org.apache.beam.sdk.runners.TransformHierarchy.visit (TransformHierarchy.java:251)
at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:460)
at org.apache.beam.sdk.Pipeline.replace (Pipeline.java:260)
at org.apache.beam.sdk.Pipeline.replaceAll (Pipeline.java:210)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:170)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:67)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:315)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:301)
at com.mozilla.telemetry.Sink.run (Sink.java:69)
at com.mozilla.telemetry.Sink.run (Sink.java:44)
at com.mozilla.telemetry.Sink.main (Sink.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for ingestion 0.1-SNAPSHOT:
[INFO]
[INFO] ingestion .......................................... SUCCESS [ 10.949 s]
[INFO] ingestion-core ..................................... SUCCESS [ 10.904 s]
[INFO] ingestion-beam ..................................... FAILURE [01:47 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:10 min
[INFO] Finished at: 2019-09-18T19:07:28Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project ingestion-beam: An exception occured while executing the Java class. Error while populating display data for component 'org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Write': null: NullPointerException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :ingestion-beam
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment