###Bootstrapping node sees StreamException and then gives up -- DSEDeamon shuts down
ERROR [main] 2015-05-15 12:46:50,652 CassandraDaemon.java (line 584) Exception encountered during startup
java.lang.RuntimeException: Error during boostrap: Stream failed
at org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:86)
at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1010)
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:813)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:626)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:515)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:437)
at com.datastax.bdp.server.DseDaemon.setup(DseDaemon.java:423)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
at com.datastax.bdp.server.DseDaemon.main(DseDaemon.java:641)
Caused by: org.apache.cassandra.streaming.StreamException: Stream failed
at org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1160)
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:216)
at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:191)
at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:377)
at org.apache.cassandra.streaming.StreamSession.convict(StreamSession.java:647)
at org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:241)
at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:719)
at org.apache.cassandra.gms.Gossiper.access$800(Gossiper.java:65)
at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:176)
at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExe cutor.java:80)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
INFO [Thread-2] 2015-05-15 12:46:50,657 DseDaemon.java (line 502) DSE shutting down...
WARN [StorageServiceShutdownHook] 2015-05-15 12:46:50,658 Gossiper.java (line 1407) No local state or state is in silent shutdown, not announcing shutdown
INFO [StorageServiceShutdownHook] 2015-05-15 12:46:50,658 MessagingService.java (line 702) Waiting for messaging service to quiesce
INFO [ACCEPT-/10.0.0.27] 2015-05-15 12:46:50,659 MessagingService.java (line 946) MessagingService has terminated the accept() thread
INFO [Thread-2] 2015-05-15 12:46:50,668 PluginManager.java (line 356) All plugins are stopped.
INFO [Thread-2] 2015-05-15 12:46:50,668 CassandraDaemon.java (line 522) Cassandra shutting down...
ERROR [Thread-2] 2015-05-15 12:46:50,669 CassandraDaemon.java (line 258) Exception in thread Thread[Thread-2,5,main]
java.lang.NullPointerException
at org.apache.cassandra.service.CassandraDaemon.stop(CassandraDaemon.java:523)
at com.datastax.bdp.server.DseDaemon.stop(DseDaemon.java:505)
at com.datastax.bdp.server.DseDaemon$1.run(DseDaemon.java:412)
###grepping for the specific stream session shows it complete twice and then fail. Are these messages displayed for each Section in the stream?
35535: INFO [main] 2015-05-14 10:22:58,081 StreamResultFuture.java (line 87) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Executing streaming plan for Bootstrap
35536: INFO [main] 2015-05-14 10:22:58,081 StreamResultFuture.java (line 91) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Beginning stream session with /10.0.0.24
35537: INFO [main] 2015-05-14 10:22:58,082 StreamResultFuture.java (line 91) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Beginning stream session with /10.0.0.25
35538: INFO [main] 2015-05-14 10:22:58,083 StreamResultFuture.java (line 91) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Beginning stream session with /10.0.0.23
35539: INFO [StreamConnectionEstablisher:2] 2015-05-14 10:22:58,083 StreamSession.java (line 218) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Starting streaming to /10.0.0.25
35540: INFO [StreamConnectionEstablisher:1] 2015-05-14 10:22:58,083 StreamSession.java (line 218) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Starting streaming to /10.0.0.24
35541: INFO [StreamConnectionEstablisher:3] 2015-05-14 10:22:58,084 StreamSession.java (line 218) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Starting streaming to /10.0.0.23
35542: INFO [STREAM-IN-/10.0.0.25] 2015-05-14 10:22:58,113 StreamResultFuture.java (line 186) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Session with /10.0.0.25 is complete
35543: INFO [STREAM-IN-/10.0.0.23] 2015-05-14 10:24:10,735 StreamResultFuture.java (line 173) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Prepare completed. Receiving 3862 files(199325668421 bytes), sending 0 files(0 bytes)
35544: INFO [STREAM-IN-/10.0.0.24] 2015-05-14 10:24:11,752 StreamResultFuture.java (line 173) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Prepare completed. Receiving 280 files(279660135124 bytes), sending 0 files(0 bytes)
75483: INFO [StreamReceiveTask:81] 2015-05-15 06:29:44,388 StreamResultFuture.java (line 186) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Session with /10.0.0.23 is complete
86294: INFO [GossipTasks:1] 2015-05-15 12:46:50,649 StreamResultFuture.java (line 186) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Session with /10.0.0.24 is complete
86295: WARN [GossipTasks:1] 2015-05-15 12:46:50,651 StreamResultFuture.java (line 215) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Stream failed
###On the sending side of the stream. The sending (existing) node we see an uncaught exception (closed channel exception)
ERROR [STREAM-OUT-/10.0.0.27] 2015-05-15 12:33:17,155 StreamSession.java (line 467) [Stream #26eaf810-fa23-11e4-8d65-5315ab24f1e9] Streaming error occurred
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:547)
at org.apache.cassandra.streaming.compress.CompressedStreamWriter.write(CompressedStreamWriter.java:74)
at org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:59)
at org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:42)
at org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:45)
at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:339)
at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:311)
at java.lang.Thread.run(Thread.java:745)