Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aromanenko-dev/a42a0a011f9fbd10b8443bf2ca38c68a to your computer and use it in GitHub Desktop.
Save aromanenko-dev/a42a0a011f9fbd10b8443bf2ca38c68a to your computer and use it in GitHub Desktop.
MacBook-Pro-de-Alexey:nexmark aromanenko$ mvn -Pdirect-runner exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main "-Dexec.args=--sinkType=PUBSUB --runner=DirectRunner --query=1 --streaming=false --manageResources=false --monitorJobs=true --enforceEncodability=false --enforceImmutability=false --pubsubRootUrl=http://localhost:8085 --pubsubSubscription=projects/nexmark/subscriptions/source --pubsubTopic=projects/nexmark/topics/sink --resourceNameMode=VERBATIM --sourceType=PUBSUB --pubSubMode=COMBINED"
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.13
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 13
[INFO] os.detected.classifier: osx-x86_64
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Beam :: SDKs :: Java :: Nexmark 2.5.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.5.0:java (default-cli) @ beam-sdks-java-nexmark ---
2018-04-09T10:01:14.750Z Running query:1; sourceType:PUBSUB; sinkType:PUBSUB
2018-04-09T10:01:15.064Z Generating 100000 events in batch mode
2018-04-09T10:01:15.110Z Writing events to Pubsub projects/nexmark/topics/sink
==========================================================================================
Run started 2018-04-09T10:01:14.713Z and ran for PT0.723S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:1; sourceType:PUBSUB; sinkType:PUBSUB
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline)
0000 *** not run ***
==========================================================================================
[WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.IllegalArgumentException: unable to serialize DoFnAndMainOutput{doFn=org.apache.beam.sdk.nexmark.NexmarkLauncher$2@16aeca13, mainOutputTag=Tag<output>}
at org.apache.beam.sdk.util.SerializableUtils.serializeToByteArray (SerializableUtils.java:57)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.translateDoFn (ParDoTranslation.java:440)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation$1.translateDoFn (ParDoTranslation.java:148)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.payloadForParDoLike (ParDoTranslation.java:656)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.translateParDo (ParDoTranslation.java:144)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation$ParDoPayloadTranslator.translate (ParDoTranslation.java:108)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.PTransformTranslation.toProto (PTransformTranslation.java:202)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.getParDoPayload (ParDoTranslation.java:515)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.isSplittable (ParDoTranslation.java:525)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.PTransformMatchers$4.matches (PTransformMatchers.java:194)
at org.apache.beam.sdk.Pipeline$2.visitPrimitiveTransform (Pipeline.java:278)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:668)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:660)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:660)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600 (TransformHierarchy.java:311)
at org.apache.beam.sdk.runners.TransformHierarchy.visit (TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:458)
at org.apache.beam.sdk.Pipeline.replace (Pipeline.java:256)
at org.apache.beam.sdk.Pipeline.replaceAll (Pipeline.java:209)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:173)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:62)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:297)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.lambda$createSource$0 (NexmarkLauncher.java:952)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.invokeBuilderForPublishOnlyPipeline (NexmarkLauncher.java:453)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.createSource (NexmarkLauncher.java:943)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run (NexmarkLauncher.java:1098)
at org.apache.beam.sdk.nexmark.Main.runAll (Main.java:69)
at org.apache.beam.sdk.nexmark.Main.main (Main.java:301)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
at java.lang.Thread.run (Thread.java:748)
Caused by: java.io.NotSerializableException: org.apache.beam.sdk.nexmark.NexmarkLauncher
at java.io.ObjectOutputStream.writeObject0 (ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.defaultWriteFields (ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData (ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject (ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0 (ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields (ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData (ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject (ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0 (ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject (ObjectOutputStream.java:348)
at org.apache.beam.sdk.util.SerializableUtils.serializeToByteArray (SerializableUtils.java:53)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.translateDoFn (ParDoTranslation.java:440)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation$1.translateDoFn (ParDoTranslation.java:148)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.payloadForParDoLike (ParDoTranslation.java:656)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.translateParDo (ParDoTranslation.java:144)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation$ParDoPayloadTranslator.translate (ParDoTranslation.java:108)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.PTransformTranslation.toProto (PTransformTranslation.java:202)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.getParDoPayload (ParDoTranslation.java:515)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.ParDoTranslation.isSplittable (ParDoTranslation.java:525)
at org.apache.beam.runners.direct.repackaged.runners.core.construction.PTransformMatchers$4.matches (PTransformMatchers.java:194)
at org.apache.beam.sdk.Pipeline$2.visitPrimitiveTransform (Pipeline.java:278)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:668)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:660)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:660)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600 (TransformHierarchy.java:311)
at org.apache.beam.sdk.runners.TransformHierarchy.visit (TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:458)
at org.apache.beam.sdk.Pipeline.replace (Pipeline.java:256)
at org.apache.beam.sdk.Pipeline.replaceAll (Pipeline.java:209)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:173)
at org.apache.beam.runners.direct.DirectRunner.run (DirectRunner.java:62)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run (Pipeline.java:297)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.lambda$createSource$0 (NexmarkLauncher.java:952)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.invokeBuilderForPublishOnlyPipeline (NexmarkLauncher.java:453)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.createSource (NexmarkLauncher.java:943)
at org.apache.beam.sdk.nexmark.NexmarkLauncher.run (NexmarkLauncher.java:1098)
at org.apache.beam.sdk.nexmark.Main.runAll (Main.java:69)
at org.apache.beam.sdk.nexmark.Main.main (Main.java:301)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.913 s
[INFO] Finished at: 2018-04-09T12:01:15+02:00
[INFO] Final Memory: 33M/329M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java (default-cli) on project beam-sdks-java-nexmark: An exception occured while executing the Java class. null: InvocationTargetException: unable to serialize DoFnAndMainOutput{doFn=org.apache.beam.sdk.nexmark.NexmarkLauncher$2@16aeca13, mainOutputTag=Tag<output>} -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment