Skip to content

Instantly share code, notes, and snippets.

@ortizfabio
Created February 19, 2020 23:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ortizfabio/84f791a129debbc8f5844e186a849f07 to your computer and use it in GitHub Desktop.
Save ortizfabio/84f791a129debbc8f5844e186a849f07 to your computer and use it in GitHub Desktop.
Stack trace of Spark executor hanging on Cassandra Spark driver
java.lang.Thread.sleep(Native Method) com.datastax.spark.connector.writer.RateLimiter$$anonfun$$lessinit$greater$default$4$1.apply$mcVJ$sp(RateLimiter.scala:29) com.datastax.spark.connector.writer.RateLimiter$$anonfun$$lessinit$greater$default$4$1.apply(RateLimiter.scala:29) com.datastax.spark.connector.writer.RateLimiter$$anonfun$$lessinit$greater$default$4$1.apply(RateLimiter.scala:29) com.datastax.spark.connector.writer.RateLimiter.maybeSleep(RateLimiter.scala:63) com.datastax.spark.connector.writer.TableWriter$$anonfun$writeInternal$1$$anonfun$apply$2.apply(TableWriter.scala:236) com.datastax.spark.connector.writer.TableWriter$$anonfun$writeInternal$1$$anonfun$apply$2.apply(TableWriter.scala:233) scala.collection.Iterator$class.foreach(Iterator.scala:893) com.datastax.spark.connector.writer.GroupingBatchBuilder.foreach(GroupingBatchBuilder.scala:31) com.datastax.spark.connector.writer.TableWriter$$anonfun$writeInternal$1.apply(TableWriter.scala:233) com.datastax.spark.connector.writer.TableWriter$$anonfun$writeInternal$1.apply(TableWriter.scala:210) com.datastax.spark.connector.cql.CassandraConnector$$anonfun$withSessionDo$1.apply(CassandraConnector.scala:112) com.datastax.spark.connector.cql.CassandraConnector$$anonfun$withSessionDo$1.apply(CassandraConnector.scala:111) com.datastax.spark.connector.cql.CassandraConnector.closeResourceAfterUse(CassandraConnector.scala:145) com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:111) com.datastax.spark.connector.writer.TableWriter.writeInternal(TableWriter.scala:210) com.datastax.spark.connector.writer.TableWriter.insert(TableWriter.scala:197) com.datastax.spark.connector.writer.TableWriter.write(TableWriter.scala:183) com.datastax.spark.connector.RDDFunctions$$anonfun$saveToCassandra$1.apply(RDDFunctions.scala:36) com.datastax.spark.connector.RDDFunctions$$anonfun$saveToCassandra$1.apply(RDDFunctions.scala:36) org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) org.apache.spark.scheduler.Task.run(Task.scala:108) org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:338) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment