Skip to content

Instantly share code, notes, and snippets.

@amontalenti

amontalenti/example_storm_crashing_lein.txt

Created January 28, 2014 22:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save amontalenti/8677464 to your computer and use it in GitHub Desktop.
Save amontalenti/8677464 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
example_storm_crashing_lein.txt
$ lein repl
nREPL server started on port 59905 on host 127.0.0.1
REPL-y 0.3.0
Clojure 1.5.1
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
user=> (ns parsely.stormtest)
nil
parsely.stormtest=> (run-local!)
0 [nREPL-worker-0] INFO backtype.storm.zookeeper - Starting inprocess zookeeper at port 2000 and dir /tmp/968c99d5-a9cf-483c-bcc4-d7339974f703
159 [nREPL-worker-0] INFO backtype.storm.daemon.nimbus - Starting Nimbus with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/a479bfd0-baf3-4c9f-a6be-f9adc7eba69b", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
165 [nREPL-worker-0] INFO backtype.storm.daemon.nimbus - Using default scheduler
200 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
250 [nREPL-worker-0-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
286 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
340 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
350 [nREPL-worker-0-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
360 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
362 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
371 [nREPL-worker-0-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
379 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
397 [nREPL-worker-0] INFO backtype.storm.daemon.supervisor - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/7e5975a0-95f8-42a4-8de2-3947ec537dbc", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1 2 3), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
408 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
414 [nREPL-worker-0-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
420 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
460 [nREPL-worker-0] INFO backtype.storm.daemon.supervisor - Starting supervisor with id f3266c90-c27d-4924-abfa-be36f5ec23a4 at host pixelmonkey-X220
463 [nREPL-worker-0] INFO backtype.storm.daemon.supervisor - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/7e4880cf-c27e-4b32-b689-cdf0913c14ff", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (4 5 6), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
465 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
470 [nREPL-worker-0-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
477 [nREPL-worker-0] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
495 [nREPL-worker-0] INFO backtype.storm.daemon.supervisor - Starting supervisor with id e9d2bc92-41c3-4b52-a696-809cab4dc7bb at host pixelmonkey-X220
564 [nREPL-worker-0] INFO backtype.storm.daemon.nimbus - Received topology submission for test-topology with conf {"topology.max.task.parallelism" nil, "topology.acker.executors" 1, "topology.kryo.register" nil, "topology.kryo.decorators" (), "topology.name" "test-topology", "storm.id" "test-topology-1-1390946585", "topology.debug" true}
582 [nREPL-worker-0] INFO backtype.storm.daemon.nimbus - Activating test-topology: test-topology-1-1390946585
659 [nREPL-worker-0] INFO backtype.storm.scheduler.EvenScheduler - Available slots: (["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 2] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 3] ["e9d2bc92-41c3-4b52-a696-809cab4dc7bb" 4] ["e9d2bc92-41c3-4b52-a696-809cab4dc7bb" 5] ["e9d2bc92-41c3-4b52-a696-809cab4dc7bb" 6])
673 [nREPL-worker-0] INFO backtype.storm.daemon.nimbus - Setting new assignment for topology id test-topology-1-1390946585: #backtype.storm.daemon.common.Assignment{:master-code-dir "/tmp/a479bfd0-baf3-4c9f-a6be-f9adc7eba69b/nimbus/stormdist/test-topology-1-1390946585", :node->host {"f3266c90-c27d-4924-abfa-be36f5ec23a4" "pixelmonkey-X220"}, :executor->node+port {[3 3] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1], [2 2] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1], [1 1] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1], [4 4] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1]}, :executor->start-time-secs {[4 4] 1390946585, [1 1] 1390946585, [2 2] 1390946585, [3 3] 1390946585}}
1215 [Thread-10] INFO backtype.storm.daemon.nimbus - Setting new assignment for topology id test-topology-1-1390946585: #backtype.storm.daemon.common.Assignment{:master-code-dir "/tmp/a479bfd0-baf3-4c9f-a6be-f9adc7eba69b/nimbus/stormdist/test-topology-1-1390946585", :node->host {"f3266c90-c27d-4924-abfa-be36f5ec23a4" "pixelmonkey-X220"}, :executor->node+port {[3 3] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1], [4 4] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1], [2 2] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1], [1 1] ["f3266c90-c27d-4924-abfa-be36f5ec23a4" 1]}, :executor->start-time-secs {[4 4] 1390946586, [1 1] 1390946586, [2 2] 1390946586, [3 3] 1390946586}}
1433 [Thread-13] INFO backtype.storm.daemon.supervisor - Downloading code for storm id test-topology-1-1390946585 from /tmp/a479bfd0-baf3-4c9f-a6be-f9adc7eba69b/nimbus/stormdist/test-topology-1-1390946585
1652 [Thread-13] INFO backtype.storm.daemon.supervisor - Copying resources at file:/home/pixelmonkey/repos/lab/storm-test/multilang/resources to /tmp/7e5975a0-95f8-42a4-8de2-3947ec537dbc/supervisor/stormdist/test-topology-1-1390946585/resources
1653 [Thread-13] INFO backtype.storm.daemon.supervisor - Finished downloading code for storm id test-topology-1-1390946585 from /tmp/a479bfd0-baf3-4c9f-a6be-f9adc7eba69b/nimbus/stormdist/test-topology-1-1390946585
1668 [Thread-14] INFO backtype.storm.daemon.supervisor - Launching worker with assignment #backtype.storm.daemon.supervisor.LocalAssignment{:storm-id "test-topology-1-1390946585", :executors ([3 3] [4 4] [2 2] [1 1])} for this supervisor f3266c90-c27d-4924-abfa-be36f5ec23a4 on port 1 with id 2b74e4d0-9ead-4845-8a8e-465810a7e25a
1671 [Thread-14] INFO backtype.storm.daemon.worker - Launching worker for test-topology-1-1390946585 on f3266c90-c27d-4924-abfa-be36f5ec23a4:1 with id 2b74e4d0-9ead-4845-8a8e-465810a7e25a and conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/7e5975a0-95f8-42a4-8de2-3947ec537dbc", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1 2 3), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
1672 [Thread-14] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
1676 [Thread-14-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
1679 [Thread-14] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
1808 [Thread-14] INFO backtype.storm.daemon.executor - Loading executor pageview-spout:[2 2]
1815 [Thread-14] INFO backtype.storm.daemon.task - Emitting: pageview-spout __system ["startup"]
1815 [Thread-14] INFO backtype.storm.daemon.executor - Loaded executor tasks pageview-spout:[2 2]
1823 [Thread-14] INFO backtype.storm.daemon.executor - Finished loading executor pageview-spout:[2 2]
1825 [Thread-24] INFO backtype.storm.daemon.executor - Opening spout pageview-spout:(2)
1832 [Thread-14] INFO backtype.storm.daemon.executor - Loading executor tweetanalyze-bolt:[3 3]
1834 [Thread-14] INFO backtype.storm.daemon.task - Emitting: tweetanalyze-bolt __system ["startup"]
1835 [Thread-14] INFO backtype.storm.daemon.executor - Loaded executor tasks tweetanalyze-bolt:[3 3]
1841 [Thread-14] INFO backtype.storm.daemon.executor - Finished loading executor tweetanalyze-bolt:[3 3]
1846 [Thread-14] INFO backtype.storm.daemon.executor - Loading executor tweetanalyze-bolt:[4 4]
1847 [Thread-14] INFO backtype.storm.daemon.task - Emitting: tweetanalyze-bolt __system ["startup"]
1851 [Thread-26] INFO backtype.storm.daemon.executor - Preparing bolt tweetanalyze-bolt:(3)
1854 [Thread-14] INFO backtype.storm.daemon.executor - Loaded executor tasks tweetanalyze-bolt:[4 4]
1860 [Thread-14] INFO backtype.storm.daemon.executor - Finished loading executor tweetanalyze-bolt:[4 4]
1862 [Thread-28] INFO backtype.storm.daemon.executor - Preparing bolt tweetanalyze-bolt:(4)
1866 [Thread-14] INFO backtype.storm.daemon.executor - Loading executor __acker:[1 1]
1868 [Thread-14] INFO backtype.storm.daemon.task - Emitting: __acker __system ["startup"]
1868 [Thread-14] INFO backtype.storm.daemon.executor - Loaded executor tasks __acker:[1 1]
1887 [Thread-14] INFO backtype.storm.daemon.executor - Finished loading executor __acker:[1 1]
1887 [Thread-14] INFO backtype.storm.daemon.worker - Launching receive-thread for f3266c90-c27d-4924-abfa-be36f5ec23a4:1
1889 [Thread-30] INFO backtype.storm.daemon.executor - Preparing bolt __acker:(1)
1895 [Thread-26] ERROR backtype.storm.util - Async loop died!
java.lang.RuntimeException: Error when launching multilang subprocess
python: can't open file 'tweetanalyze.py': [Errno 2] No such file or directory
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:82)
at backtype.storm.daemon.executor$fn__4050$fn__4059.invoke(executor.clj:610)
at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Broken pipe
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at backtype.storm.utils.ShellProcess.writeString(ShellProcess.java:58)
at backtype.storm.utils.ShellProcess.writeMessage(ShellProcess.java:51)
at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:41)
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:79)
... 4 more
1908 [Thread-26] ERROR backtype.storm.daemon.executor -
java.lang.RuntimeException: Error when launching multilang subprocess
python: can't open file 'tweetanalyze.py': [Errno 2] No such file or directory
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:82)
at backtype.storm.daemon.executor$fn__4050$fn__4059.invoke(executor.clj:610)
at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.io.IOException: Broken pipe
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at backtype.storm.utils.ShellProcess.writeString(ShellProcess.java:58)
at backtype.storm.utils.ShellProcess.writeMessage(ShellProcess.java:51)
at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:41)
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:79)
... 4 more
1921 [Thread-30] INFO backtype.storm.daemon.executor - Prepared bolt __acker:(1)
1927 [Thread-14] INFO backtype.storm.daemon.worker - Worker has topology config {"storm.id" "test-topology-1-1390946585", "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/tmp/7e5975a0-95f8-42a4-8de2-3947ec537dbc", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.kryo.decorators" (), "topology.name" "test-topology", "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1 2 3), "topology.debug" true, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.kryo.register" nil, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
1927 [Thread-14] INFO backtype.storm.daemon.worker - Worker 2b74e4d0-9ead-4845-8a8e-465810a7e25a for storm test-topology-1-1390946585 on f3266c90-c27d-4924-abfa-be36f5ec23a4:1 has finished loading
1921 [Thread-28] ERROR backtype.storm.util - Async loop died!
java.lang.RuntimeException: Pipe to subprocess seems to be broken! No output read.
Shell Process Exception:
python: can't open file 'tweetanalyze.py': [Errno 2] No such file or directory
at backtype.storm.utils.ShellProcess.readString(ShellProcess.java:100)
at backtype.storm.utils.ShellProcess.readMessage(ShellProcess.java:62)
at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:43)
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:79)
at backtype.storm.daemon.executor$fn__4050$fn__4059.invoke(executor.clj:610)
at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:744)
1930 [Thread-28] ERROR backtype.storm.daemon.executor -
java.lang.RuntimeException: Pipe to subprocess seems to be broken! No output read.
Shell Process Exception:
python: can't open file 'tweetanalyze.py': [Errno 2] No such file or directory
at backtype.storm.utils.ShellProcess.readString(ShellProcess.java:100)
at backtype.storm.utils.ShellProcess.readMessage(ShellProcess.java:62)
at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:43)
at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:79)
at backtype.storm.daemon.executor$fn__4050$fn__4059.invoke(executor.clj:610)
at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:744)
1950 [Thread-26] INFO backtype.storm.util - Halting process: ("Worker died")
Exception in thread "Thread-4" SocketException The transport's socket appears to have lost its connection to the nREPL server
clojure.lang.ExceptionInfo: Subprocess failed {:exit-code 1}
at clojure.core$ex_info.invoke(core.clj:4327)
at leiningen.core.eval$fn__3532.invoke(eval.clj:226)
at clojure.lang.MultiFn.invoke(MultiFn.java:231)
at leiningen.core.eval$eval_in_project.invoke(eval.clj:326)
at clojure.lang.AFn.applyToHelper(AFn.java:167)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:619)
at leiningen.repl$server$fn__7443.invoke(repl.clj:201)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:617)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1788)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$bound_fn_STAR_$fn__4102.doInvoke(core.clj:1810)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:744)
clojure.tools.nrepl.transport/bencode/fn--4287/fn--4288 (transport.clj:95)
clojure.tools.nrepl.transport/bencode/fn--4287 (transport.clj:95)
clojure.tools.nrepl.transport/fn-transport/fn--4261 (transport.clj:42)
clojure.core/binding-conveyor-fn/fn--4107 (core.clj:1836)
java.util.concurrent.FutureTask.run (FutureTask.java:262)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615)
java.lang.Thread.run (Thread.java:744)
Bye for now!
» ~/repos/lab/storm-test [master]
$
@koushikckm
Copy link

Did u find out any solution for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment