Skip to content

Instantly share code, notes, and snippets.

@wlach
Created September 12, 2019 21:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save wlach/30988b55c1b99b15e09371d4975866f0 to your computer and use it in GitHub Desktop.
Save wlach/30988b55c1b99b15e09371d4975866f0 to your computer and use it in GitHub Desktop.
Hammersmith:ingestion-beam wlach$ ./bin/mvn compile exec:java -Dexec.args="\
--inputType=heka \
--input=ingestion-beam/tmp/cheezburger.bad.heka \
--outputFileFormat=text \
--outputType=stdout \
--errorOutputType=stderr"
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] ingestion [pom]
[INFO] ingestion-core [jar]
[INFO] ingestion-beam [jar]
[INFO]
[INFO] ------------------< com.mozilla.telemetry:ingestion >-------------------
[INFO] Building ingestion 0.1-SNAPSHOT [1/3]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (default) @ ingestion ---
[INFO] argLine set to -javaagent:/var/maven/project/?/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/var/maven/project/target/jacoco.exec -Xmx1024m -XX:MaxPermSize=256m -Djdk.net.URLClassPath.disableClassPathURLCheck=true
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ ingestion ---
[INFO] skipping execute as per configuration
[INFO]
[INFO] ----------------< com.mozilla.telemetry:ingestion-core >----------------
[INFO] Building ingestion-core 0.1-SNAPSHOT [2/3]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (default) @ ingestion-core ---
[INFO] argLine set to -javaagent:/var/maven/project/?/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/var/maven/project/ingestion-core/target/jacoco.exec -Xmx1024m -XX:MaxPermSize=256m -Djdk.net.URLClassPath.disableClassPathURLCheck=true
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ ingestion-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /var/maven/project/ingestion-core/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ ingestion-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ ingestion-core ---
[INFO] skipping execute as per configuration
[INFO]
[INFO] ----------------< com.mozilla.telemetry:ingestion-beam >----------------
[INFO] Building ingestion-beam 0.1-SNAPSHOT [3/3]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.4:prepare-agent (default) @ ingestion-beam ---
[INFO] argLine set to -javaagent:/var/maven/project/?/.m2/repository/org/jacoco/org.jacoco.agent/0.8.4/org.jacoco.agent-0.8.4-runtime.jar=destfile=/var/maven/project/ingestion-beam/target/jacoco.exec -Xmx1024m -XX:MaxPermSize=256m -Djdk.net.URLClassPath.disableClassPathURLCheck=true
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ ingestion-beam ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /var/maven/project/ingestion-beam/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ ingestion-beam ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ ingestion-beam ---
Sep 12, 2019 9:08:12 PM org.apache.beam.sdk.io.FileIO$MatchAll$MatchFn process
INFO: Matched 1 files for pattern ingestion-beam/tmp/cheezburger.bad.heka
[WARNING]
org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException: cannot encode a null byte[]
at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish (DirectRunner.java:348)
at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish (DirectRunner.java:318)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:213)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:67)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:299)
at com.mozilla.telemetry.Sink.run (Sink.java:64)
at com.mozilla.telemetry.Sink.run (Sink.java:42)
at com.mozilla.telemetry.Sink.main (Sink.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException: cannot encode a null byte[]
at org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.add (ImmutabilityCheckingBundleFactory.java:114)
at org.apache.beam.runners.direct.ParDoEvaluator$BundleOutputManager.output (ParDoEvaluator.java:264)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.outputWindowedValue (SimpleDoFnRunner.java:248)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.access$700 (SimpleDoFnRunner.java:74)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner$DoFnProcessContext.output (SimpleDoFnRunner.java:560)
at org.apache.beam.sdk.transforms.DoFnOutputReceivers$WindowedContextOutputReceiver.output (DoFnOutputReceivers.java:71)
at com.mozilla.telemetry.transforms.MapElementsWithErrors$DoFnWithErrors.processElementOrError (MapElementsWithErrors.java:97)
Caused by: org.apache.beam.sdk.coders.CoderException: cannot encode a null byte[]
at org.apache.beam.sdk.coders.ByteArrayCoder.encode (ByteArrayCoder.java:63)
at org.apache.beam.sdk.coders.ByteArrayCoder.encode (ByteArrayCoder.java:56)
at org.apache.beam.sdk.coders.ByteArrayCoder.encode (ByteArrayCoder.java:41)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubMessageWithAttributesCoder.encode (PubsubMessageWithAttributesCoder.java:55)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubMessageWithAttributesCoder.encode (PubsubMessageWithAttributesCoder.java:33)
at org.apache.beam.sdk.util.CoderUtils.encodeToSafeStream (CoderUtils.java:82)
at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray (CoderUtils.java:66)
at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray (CoderUtils.java:51)
at org.apache.beam.sdk.util.CoderUtils.clone (CoderUtils.java:141)
at org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector.<init> (MutationDetectors.java:113)
at org.apache.beam.sdk.util.MutationDetectors.forValueWithCoder (MutationDetectors.java:44)
at org.apache.beam.runners.direct.ImmutabilityCheckingBundleFactory$ImmutabilityEnforcingBundle.add (ImmutabilityCheckingBundleFactory.java:112)
at org.apache.beam.runners.direct.ParDoEvaluator$BundleOutputManager.output (ParDoEvaluator.java:264)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.outputWindowedValue (SimpleDoFnRunner.java:248)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.access$700 (SimpleDoFnRunner.java:74)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner$DoFnProcessContext.output (SimpleDoFnRunner.java:560)
at org.apache.beam.sdk.transforms.DoFnOutputReceivers$WindowedContextOutputReceiver.output (DoFnOutputReceivers.java:71)
at com.mozilla.telemetry.transforms.MapElementsWithErrors$DoFnWithErrors.processElementOrError (MapElementsWithErrors.java:97)
at com.mozilla.telemetry.transforms.MapElementsWithErrors$DoFnWithErrors$DoFnInvoker.invokeProcessElement (Unknown Source)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.invokeProcessElement (SimpleDoFnRunner.java:214)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.processElement (SimpleDoFnRunner.java:179)
at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows (SimplePushbackSideInputDoFnRunner.java:78)
at org.apache.beam.runners.direct.ParDoEvaluator.processElement (ParDoEvaluator.java:211)
at org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement (DoFnLifecycleManagerRemovingTransformEvaluator.java:54)
at org.apache.beam.runners.direct.DirectTransformExecutor.processElements (DirectTransformExecutor.java:160)
at org.apache.beam.runners.direct.DirectTransformExecutor.run (DirectTransformExecutor.java:124)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for ingestion 0.1-SNAPSHOT:
[INFO]
[INFO] ingestion .......................................... SUCCESS [ 5.319 s]
[INFO] ingestion-core ..................................... SUCCESS [ 4.325 s]
[INFO] ingestion-beam ..................................... FAILURE [ 22.529 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 32.888 s
[INFO] Finished at: 2019-09-12T21:08:13Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project ingestion-beam: An exception occured while executing the Java class. java.lang.RuntimeException: org.apache.beam.sdk.coders.CoderException: cannot encode a null byte[] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :ingestion-beam
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment