Last active
April 16, 2019 08:25
-
-
Save ssimeonov/72c8a9b01f99e35ba470 to your computer and use it in GitHub Desktop.
SPARK-9210 test: Spark SQL first() vs. first_value()
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // This code is designed to be pasted in spark-shell in a *nix environment | |
| // On Windows, replace sys.env("HOME") with a directory of your choice | |
| import java.io.File | |
| import java.io.PrintWriter | |
| import org.apache.spark.sql.hive.HiveContext | |
| import org.apache.spark.sql.SaveMode | |
| import org.apache.spark.sql.SaveMode | |
| val ctx = sqlContext.asInstanceOf[HiveContext] | |
| import ctx.implicits._ | |
| val json = """{"category" : "A", "num" : 5}""" | |
| val path = sys.env("HOME") + "/spark_sql_first.jsonlines" | |
| new PrintWriter(path) { write(json); close } | |
| ctx.read.json("file://" + path).registerTempTable("test_first") | |
| // OK, proof that the data was loaded correctly | |
| ctx.sql("select * from test_first").show | |
| // org.apache.spark.sql.AnalysisException: expression 'num' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() if you don't care which value you get. | |
| ctx.sql("select num from test_first group by category").show | |
| // ERROR RetryingHMSHandler: MetaException(message:NoSuchObjectException(message:Function default.first does not exist)) | |
| // INFO FunctionRegistry: Unable to lookup UDF in metastore: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:NoSuchObjectException(message:Function default.first does not exist)) | |
| // java.lang.RuntimeException: Couldn't find function first | |
| ctx.sql("select first(num) from test_first group by category").show | |
| // OK | |
| ctx.sql("select first_value(num) from test_first group by category").show | |
| new File(path).delete() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ubuntu@ip-10-88-50-23:~$ ~/spark/bin/spark-shell --packages com.databricks:spark-csv_2.10:1.0.3 --driver-memory 48g --executor-memory 12g --conf "spark.driver.extraJavaOptions=-XX:MaxPermSize=512m" --conf "spark.local.dir=/data/spark/tmp" | |
| Ivy Default Cache set to: /home/ubuntu/.ivy2/cache | |
| The jars for the packages stored in: /home/ubuntu/.ivy2/jars | |
| :: loading settings :: url = jar:file:/home/ubuntu/spark-1.4.1-bin-hadoop2.6/lib/spark-assembly-1.4.1-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml | |
| com.databricks#spark-csv_2.10 added as a dependency | |
| :: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0 | |
| confs: [default] | |
| found com.databricks#spark-csv_2.10;1.0.3 in central | |
| found org.apache.commons#commons-csv;1.1 in central | |
| :: resolution report :: resolve 213ms :: artifacts dl 6ms | |
| :: modules in use: | |
| com.databricks#spark-csv_2.10;1.0.3 from central in [default] | |
| org.apache.commons#commons-csv;1.1 from central in [default] | |
| --------------------------------------------------------------------- | |
| | | modules || artifacts | | |
| | conf | number| search|dwnlded|evicted|| number|dwnlded| | |
| --------------------------------------------------------------------- | |
| | default | 2 | 0 | 0 | 0 || 2 | 0 | | |
| --------------------------------------------------------------------- | |
| :: retrieving :: org.apache.spark#spark-submit-parent | |
| confs: [default] | |
| 0 artifacts copied, 2 already retrieved (0kB/6ms) | |
| log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). | |
| log4j:WARN Please initialize the log4j system properly. | |
| log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. | |
| Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties | |
| 15/07/22 01:28:55 INFO SecurityManager: Changing view acls to: ubuntu | |
| 15/07/22 01:28:55 INFO SecurityManager: Changing modify acls to: ubuntu | |
| 15/07/22 01:28:55 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(ubuntu); users with modify permissions: Set(ubuntu) | |
| 15/07/22 01:28:55 INFO HttpServer: Starting HTTP Server | |
| 15/07/22 01:28:55 INFO Utils: Successfully started service 'HTTP class server' on port 44542. | |
| Welcome to | |
| ____ __ | |
| / __/__ ___ _____/ /__ | |
| _\ \/ _ \/ _ `/ __/ '_/ | |
| /___/ .__/\_,_/_/ /_/\_\ version 1.4.1 | |
| /_/ | |
| Using Scala version 2.10.4 (OpenJDK 64-Bit Server VM, Java 1.7.0_79) | |
| Type in expressions to have them evaluated. | |
| Type :help for more information. | |
| 15/07/22 01:28:58 WARN Utils: Your hostname, ip-10-88-50-23 resolves to a loopback address: 127.0.0.1; using 10.88.50.23 instead (on interface eth0) | |
| 15/07/22 01:28:58 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address | |
| 15/07/22 01:28:58 INFO SparkContext: Running Spark version 1.4.1 | |
| 15/07/22 01:28:58 WARN SparkConf: In Spark 1.0 and later spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone and LOCAL_DIRS in YARN). | |
| 15/07/22 01:28:58 INFO SecurityManager: Changing view acls to: ubuntu | |
| 15/07/22 01:28:58 INFO SecurityManager: Changing modify acls to: ubuntu | |
| 15/07/22 01:28:58 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(ubuntu); users with modify permissions: Set(ubuntu) | |
| 15/07/22 01:28:59 INFO Slf4jLogger: Slf4jLogger started | |
| 15/07/22 01:28:59 INFO Remoting: Starting remoting | |
| 15/07/22 01:28:59 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@10.88.50.23:59804] | |
| 15/07/22 01:28:59 INFO Utils: Successfully started service 'sparkDriver' on port 59804. | |
| 15/07/22 01:28:59 INFO SparkEnv: Registering MapOutputTracker | |
| 15/07/22 01:28:59 INFO SparkEnv: Registering BlockManagerMaster | |
| 15/07/22 01:28:59 INFO DiskBlockManager: Created local directory at /data/spark/tmp/spark-63e9be6d-4c39-4362-be3b-45823d342371/blockmgr-dd9e32fe-bbc8-4344-92a2-3d2c5130e6d1 | |
| 15/07/22 01:28:59 INFO MemoryStore: MemoryStore started with capacity 24.8 GB | |
| 15/07/22 01:28:59 INFO HttpFileServer: HTTP File server directory is /data/spark/tmp/spark-63e9be6d-4c39-4362-be3b-45823d342371/httpd-6858ed37-1871-4e83-b475-48182fb0a637 | |
| 15/07/22 01:28:59 INFO HttpServer: Starting HTTP Server | |
| 15/07/22 01:28:59 INFO Utils: Successfully started service 'HTTP file server' on port 60159. | |
| 15/07/22 01:28:59 INFO SparkEnv: Registering OutputCommitCoordinator | |
| 15/07/22 01:28:59 INFO Utils: Successfully started service 'SparkUI' on port 4040. | |
| 15/07/22 01:28:59 INFO SparkUI: Started SparkUI at http://10.88.50.23:4040 | |
| 15/07/22 01:28:59 INFO SparkContext: Added JAR file:/home/ubuntu/.ivy2/jars/com.databricks_spark-csv_2.10-1.0.3.jar at http://10.88.50.23:60159/jars/com.databricks_spark-csv_2.10-1.0.3.jar with timestamp 1437528539740 | |
| 15/07/22 01:28:59 INFO SparkContext: Added JAR file:/home/ubuntu/.ivy2/jars/org.apache.commons_commons-csv-1.1.jar at http://10.88.50.23:60159/jars/org.apache.commons_commons-csv-1.1.jar with timestamp 1437528539740 | |
| 15/07/22 01:28:59 INFO Executor: Starting executor ID driver on host localhost | |
| 15/07/22 01:28:59 INFO Executor: Using REPL class URI: http://10.88.50.23:44542 | |
| 15/07/22 01:28:59 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 35934. | |
| 15/07/22 01:28:59 INFO NettyBlockTransferService: Server created on 35934 | |
| 15/07/22 01:28:59 INFO BlockManagerMaster: Trying to register BlockManager | |
| 15/07/22 01:28:59 INFO BlockManagerMasterEndpoint: Registering block manager localhost:35934 with 24.8 GB RAM, BlockManagerId(driver, localhost, 35934) | |
| 15/07/22 01:28:59 INFO BlockManagerMaster: Registered BlockManager | |
| 15/07/22 01:29:00 INFO SparkILoop: Created spark context.. | |
| Spark context available as sc. | |
| 15/07/22 01:29:00 INFO HiveContext: Initializing execution hive, version 0.13.1 | |
| 15/07/22 01:29:00 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore | |
| 15/07/22 01:29:00 INFO ObjectStore: ObjectStore, initialize called | |
| 15/07/22 01:29:00 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/home/ubuntu/spark/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/home/ubuntu/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar." | |
| 15/07/22 01:29:00 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/home/ubuntu/spark/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/home/ubuntu/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar." | |
| 15/07/22 01:29:00 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/home/ubuntu/spark/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/home/ubuntu/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar." | |
| 15/07/22 01:29:00 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored | |
| 15/07/22 01:29:00 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored | |
| 15/07/22 01:29:00 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) | |
| 15/07/22 01:29:01 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) | |
| 15/07/22 01:29:02 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" | |
| 15/07/22 01:29:02 INFO MetaStoreDirectSql: MySQL check failed, assuming we are not on mysql: Lexical error at line 1, column 5. Encountered: "@" (64), after : "". | |
| 15/07/22 01:29:02 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:02 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:04 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:04 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:04 INFO ObjectStore: Initialized ObjectStore | |
| 15/07/22 01:29:04 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 0.13.1aa | |
| 15/07/22 01:29:04 INFO HiveMetaStore: Added admin role in metastore | |
| 15/07/22 01:29:04 INFO HiveMetaStore: Added public role in metastore | |
| 15/07/22 01:29:04 INFO HiveMetaStore: No user is added in admin role, since config is empty | |
| 15/07/22 01:29:05 INFO SessionState: No Tez session required at this point. hive.execution.engine=mr. | |
| 15/07/22 01:29:05 INFO SparkILoop: Created sql context (with Hive support).. | |
| SQL context available as sqlContext. | |
| scala> import java.io.File | |
| import java.io.File | |
| scala> import java.io.PrintWriter | |
| import java.io.PrintWriter | |
| scala> import org.apache.spark.sql.hive.HiveContext | |
| import org.apache.spark.sql.hive.HiveContext | |
| scala> import org.apache.spark.sql.SaveMode | |
| import org.apache.spark.sql.SaveMode | |
| scala> import org.apache.spark.sql.SaveMode | |
| import org.apache.spark.sql.SaveMode | |
| scala> | |
| scala> val ctx = sqlContext.asInstanceOf[HiveContext] | |
| ctx: org.apache.spark.sql.hive.HiveContext = org.apache.spark.sql.hive.HiveContext@2a5c58ae | |
| scala> import ctx.implicits._ | |
| import ctx.implicits._ | |
| scala> | |
| scala> val json = """{"category" : "A", "num" : 5}""" | |
| json: String = {"category" : "A", "num" : 5} | |
| scala> val path = sys.env("HOME") + "/spark_sql_first.jsonlines" | |
| path: String = /home/ubuntu/spark_sql_first.jsonlines | |
| scala> new PrintWriter(path) { write(json); close } | |
| res0: java.io.PrintWriter = $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anon$1@308780c1 | |
| scala> ctx.read.json("file://" + path).registerTempTable("test_first") | |
| 15/07/22 01:29:13 INFO MemoryStore: ensureFreeSpace(112480) called with curMem=0, maxMem=26671746908 | |
| 15/07/22 01:29:13 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 109.8 KB, free 24.8 GB) | |
| 15/07/22 01:29:13 INFO MemoryStore: ensureFreeSpace(19750) called with curMem=112480, maxMem=26671746908 | |
| 15/07/22 01:29:13 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 19.3 KB, free 24.8 GB) | |
| 15/07/22 01:29:13 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:35934 (size: 19.3 KB, free: 24.8 GB) | |
| 15/07/22 01:29:13 INFO SparkContext: Created broadcast 0 from json at <console>:33 | |
| 15/07/22 01:29:13 INFO FileInputFormat: Total input paths to process : 1 | |
| 15/07/22 01:29:13 INFO SparkContext: Starting job: json at <console>:33 | |
| 15/07/22 01:29:13 INFO DAGScheduler: Got job 0 (json at <console>:33) with 2 output partitions (allowLocal=false) | |
| 15/07/22 01:29:13 INFO DAGScheduler: Final stage: ResultStage 0(json at <console>:33) | |
| 15/07/22 01:29:13 INFO DAGScheduler: Parents of final stage: List() | |
| 15/07/22 01:29:13 INFO DAGScheduler: Missing parents: List() | |
| 15/07/22 01:29:13 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[3] at json at <console>:33), which has no missing parents | |
| 15/07/22 01:29:13 INFO MemoryStore: ensureFreeSpace(4360) called with curMem=132230, maxMem=26671746908 | |
| 15/07/22 01:29:13 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 4.3 KB, free 24.8 GB) | |
| 15/07/22 01:29:13 INFO MemoryStore: ensureFreeSpace(2397) called with curMem=136590, maxMem=26671746908 | |
| 15/07/22 01:29:13 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.3 KB, free 24.8 GB) | |
| 15/07/22 01:29:13 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:35934 (size: 2.3 KB, free: 24.8 GB) | |
| 15/07/22 01:29:13 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:874 | |
| 15/07/22 01:29:13 INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at json at <console>:33) | |
| 15/07/22 01:29:13 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks | |
| 15/07/22 01:29:13 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, PROCESS_LOCAL, 1573 bytes) | |
| 15/07/22 01:29:13 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, localhost, PROCESS_LOCAL, 1573 bytes) | |
| 15/07/22 01:29:13 INFO Executor: Running task 0.0 in stage 0.0 (TID 0) | |
| 15/07/22 01:29:13 INFO Executor: Running task 1.0 in stage 0.0 (TID 1) | |
| 15/07/22 01:29:13 INFO Executor: Fetching http://10.88.50.23:60159/jars/com.databricks_spark-csv_2.10-1.0.3.jar with timestamp 1437528539740 | |
| 15/07/22 01:29:13 INFO Utils: Fetching http://10.88.50.23:60159/jars/com.databricks_spark-csv_2.10-1.0.3.jar to /data/spark/tmp/spark-63e9be6d-4c39-4362-be3b-45823d342371/userFiles-2725852d-e105-49ca-837d-1411e1a861a8/fetchFileTemp3306845902988777149.tmp | |
| 15/07/22 01:29:13 INFO Executor: Adding file:/data/spark/tmp/spark-63e9be6d-4c39-4362-be3b-45823d342371/userFiles-2725852d-e105-49ca-837d-1411e1a861a8/com.databricks_spark-csv_2.10-1.0.3.jar to class loader | |
| 15/07/22 01:29:13 INFO Executor: Fetching http://10.88.50.23:60159/jars/org.apache.commons_commons-csv-1.1.jar with timestamp 1437528539740 | |
| 15/07/22 01:29:13 INFO Utils: Fetching http://10.88.50.23:60159/jars/org.apache.commons_commons-csv-1.1.jar to /data/spark/tmp/spark-63e9be6d-4c39-4362-be3b-45823d342371/userFiles-2725852d-e105-49ca-837d-1411e1a861a8/fetchFileTemp7121429094131947532.tmp | |
| 15/07/22 01:29:13 INFO Executor: Adding file:/data/spark/tmp/spark-63e9be6d-4c39-4362-be3b-45823d342371/userFiles-2725852d-e105-49ca-837d-1411e1a861a8/org.apache.commons_commons-csv-1.1.jar to class loader | |
| 15/07/22 01:29:13 INFO HadoopRDD: Input split: file:/home/ubuntu/spark_sql_first.jsonlines:14+15 | |
| 15/07/22 01:29:13 INFO HadoopRDD: Input split: file:/home/ubuntu/spark_sql_first.jsonlines:0+14 | |
| 15/07/22 01:29:13 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id | |
| 15/07/22 01:29:13 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id | |
| 15/07/22 01:29:13 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap | |
| 15/07/22 01:29:13 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition | |
| 15/07/22 01:29:13 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id | |
| 15/07/22 01:29:13 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 2137 bytes result sent to driver | |
| 15/07/22 01:29:13 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 246 ms on localhost (1/2) | |
| 15/07/22 01:29:13 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 2597 bytes result sent to driver | |
| 15/07/22 01:29:13 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 272 ms on localhost (2/2) | |
| 15/07/22 01:29:13 INFO DAGScheduler: ResultStage 0 (json at <console>:33) finished in 0.280 s | |
| 15/07/22 01:29:13 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool | |
| 15/07/22 01:29:13 INFO DAGScheduler: Job 0 finished: json at <console>:33, took 0.321459 s | |
| 15/07/22 01:29:13 INFO HiveContext: Initializing HiveMetastoreConnection version 0.13.1 using Spark classes. | |
| 15/07/22 01:29:14 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
| 15/07/22 01:29:14 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore | |
| 15/07/22 01:29:14 INFO ObjectStore: ObjectStore, initialize called | |
| 15/07/22 01:29:14 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/home/ubuntu/spark/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/home/ubuntu/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar." | |
| 15/07/22 01:29:14 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/home/ubuntu/spark/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/home/ubuntu/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar." | |
| 15/07/22 01:29:14 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/home/ubuntu/spark/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/home/ubuntu/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar." | |
| 15/07/22 01:29:14 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored | |
| 15/07/22 01:29:14 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored | |
| 15/07/22 01:29:14 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) | |
| 15/07/22 01:29:14 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) | |
| 15/07/22 01:29:15 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" | |
| 15/07/22 01:29:15 INFO MetaStoreDirectSql: MySQL check failed, assuming we are not on mysql: Lexical error at line 1, column 5. Encountered: "@" (64), after : "". | |
| 15/07/22 01:29:15 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:15 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:16 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:16 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:16 INFO Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing | |
| 15/07/22 01:29:16 INFO ObjectStore: Initialized ObjectStore | |
| 15/07/22 01:29:16 INFO HiveMetaStore: Added admin role in metastore | |
| 15/07/22 01:29:16 INFO HiveMetaStore: Added public role in metastore | |
| 15/07/22 01:29:16 INFO HiveMetaStore: No user is added in admin role, since config is empty | |
| 15/07/22 01:29:16 INFO SessionState: No Tez session required at this point. hive.execution.engine=mr. | |
| scala> | |
| scala> ctx.sql("select * from test_first").show | |
| 15/07/22 01:29:17 INFO ParseDriver: Parsing command: select * from test_first | |
| 15/07/22 01:29:17 INFO ParseDriver: Parse Completed | |
| 15/07/22 01:29:17 INFO MemoryStore: ensureFreeSpace(294016) called with curMem=138987, maxMem=26671746908 | |
| 15/07/22 01:29:17 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 287.1 KB, free 24.8 GB) | |
| 15/07/22 01:29:17 INFO MemoryStore: ensureFreeSpace(19750) called with curMem=433003, maxMem=26671746908 | |
| 15/07/22 01:29:17 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 19.3 KB, free 24.8 GB) | |
| 15/07/22 01:29:17 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:35934 (size: 19.3 KB, free: 24.8 GB) | |
| 15/07/22 01:29:17 INFO SparkContext: Created broadcast 2 from show at <console>:31 | |
| 15/07/22 01:29:17 INFO FileInputFormat: Total input paths to process : 1 | |
| 15/07/22 01:29:17 INFO SparkContext: Starting job: show at <console>:31 | |
| 15/07/22 01:29:17 INFO DAGScheduler: Got job 1 (show at <console>:31) with 1 output partitions (allowLocal=false) | |
| 15/07/22 01:29:17 INFO DAGScheduler: Final stage: ResultStage 1(show at <console>:31) | |
| 15/07/22 01:29:17 INFO DAGScheduler: Parents of final stage: List() | |
| 15/07/22 01:29:17 INFO DAGScheduler: Missing parents: List() | |
| 15/07/22 01:29:17 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[7] at show at <console>:31), which has no missing parents | |
| 15/07/22 01:29:17 INFO MemoryStore: ensureFreeSpace(4296) called with curMem=452753, maxMem=26671746908 | |
| 15/07/22 01:29:17 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 4.2 KB, free 24.8 GB) | |
| 15/07/22 01:29:17 INFO MemoryStore: ensureFreeSpace(2413) called with curMem=457049, maxMem=26671746908 | |
| 15/07/22 01:29:17 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 2.4 KB, free 24.8 GB) | |
| 15/07/22 01:29:17 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on localhost:35934 (size: 2.4 KB, free: 24.8 GB) | |
| 15/07/22 01:29:17 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:874 | |
| 15/07/22 01:29:17 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[7] at show at <console>:31) | |
| 15/07/22 01:29:17 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks | |
| 15/07/22 01:29:17 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, localhost, PROCESS_LOCAL, 1573 bytes) | |
| 15/07/22 01:29:17 INFO Executor: Running task 0.0 in stage 1.0 (TID 2) | |
| 15/07/22 01:29:17 INFO HadoopRDD: Input split: file:/home/ubuntu/spark_sql_first.jsonlines:0+14 | |
| 15/07/22 01:29:17 INFO Executor: Finished task 0.0 in stage 1.0 (TID 2). 2112 bytes result sent to driver | |
| 15/07/22 01:29:17 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 13 ms on localhost (1/1) | |
| 15/07/22 01:29:17 INFO DAGScheduler: ResultStage 1 (show at <console>:31) finished in 0.013 s | |
| 15/07/22 01:29:17 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool | |
| 15/07/22 01:29:17 INFO DAGScheduler: Job 1 finished: show at <console>:31, took 0.021917 s | |
| 15/07/22 01:29:17 INFO SparkContext: Starting job: show at <console>:31 | |
| 15/07/22 01:29:17 INFO DAGScheduler: Got job 2 (show at <console>:31) with 1 output partitions (allowLocal=false) | |
| 15/07/22 01:29:17 INFO DAGScheduler: Final stage: ResultStage 2(show at <console>:31) | |
| 15/07/22 01:29:17 INFO DAGScheduler: Parents of final stage: List() | |
| 15/07/22 01:29:17 INFO DAGScheduler: Missing parents: List() | |
| 15/07/22 01:29:17 INFO DAGScheduler: Submitting ResultStage 2 (MapPartitionsRDD[7] at show at <console>:31), which has no missing parents | |
| 15/07/22 01:29:17 INFO MemoryStore: ensureFreeSpace(4296) called with curMem=459462, maxMem=26671746908 | |
| 15/07/22 01:29:17 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 4.2 KB, free 24.8 GB) | |
| 15/07/22 01:29:17 INFO MemoryStore: ensureFreeSpace(2413) called with curMem=463758, maxMem=26671746908 | |
| 15/07/22 01:29:17 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 2.4 KB, free 24.8 GB) | |
| 15/07/22 01:29:17 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on localhost:35934 (size: 2.4 KB, free: 24.8 GB) | |
| 15/07/22 01:29:17 INFO SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:874 | |
| 15/07/22 01:29:17 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (MapPartitionsRDD[7] at show at <console>:31) | |
| 15/07/22 01:29:17 INFO TaskSchedulerImpl: Adding task set 2.0 with 1 tasks | |
| 15/07/22 01:29:17 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 3, localhost, PROCESS_LOCAL, 1573 bytes) | |
| 15/07/22 01:29:17 INFO Executor: Running task 0.0 in stage 2.0 (TID 3) | |
| 15/07/22 01:29:17 INFO HadoopRDD: Input split: file:/home/ubuntu/spark_sql_first.jsonlines:14+15 | |
| 15/07/22 01:29:17 INFO Executor: Finished task 0.0 in stage 2.0 (TID 3). 1800 bytes result sent to driver | |
| 15/07/22 01:29:17 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 3) in 7 ms on localhost (1/1) | |
| 15/07/22 01:29:17 INFO DAGScheduler: ResultStage 2 (show at <console>:31) finished in 0.007 s | |
| 15/07/22 01:29:17 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool | |
| 15/07/22 01:29:17 INFO DAGScheduler: Job 2 finished: show at <console>:31, took 0.014699 s | |
| +--------+---+ | |
| |category|num| | |
| +--------+---+ | |
| | A| 5| | |
| +--------+---+ | |
| scala> ctx.sql("select num from test_first group by category").show | |
| 15/07/22 01:29:17 INFO ParseDriver: Parsing command: select num from test_first group by category | |
| 15/07/22 01:29:17 INFO ParseDriver: Parse Completed | |
| org.apache.spark.sql.AnalysisException: expression 'num' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() if you don't care which value you get.; | |
| at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:38) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:42) | |
| at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.org$apache$spark$sql$catalyst$analysis$CheckAnalysis$class$$anonfun$$checkValidAggregateExpression$1(CheckAnalysis.scala:95) | |
| at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$4.apply(CheckAnalysis.scala:111) | |
| at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$4.apply(CheckAnalysis.scala:111) | |
| at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) | |
| at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) | |
| at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:111) | |
| at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:50) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:98) | |
| at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:50) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:42) | |
| at org.apache.spark.sql.SQLContext$QueryExecution.assertAnalyzed(SQLContext.scala:931) | |
| at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:131) | |
| at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) | |
| at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:31) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:36) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:38) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:40) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:42) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:44) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:46) | |
| at $iwC$$iwC$$iwC$$iwC.<init>(<console>:48) | |
| at $iwC$$iwC$$iwC.<init>(<console>:50) | |
| at $iwC$$iwC.<init>(<console>:52) | |
| at $iwC.<init>(<console>:54) | |
| at <init>(<console>:56) | |
| at .<init>(<console>:60) | |
| at .<clinit>(<console>) | |
| at .<init>(<console>:7) | |
| at .<clinit>(<console>) | |
| at $print(<console>) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065) | |
| at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338) | |
| at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840) | |
| at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871) | |
| at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819) | |
| at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857) | |
| at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902) | |
| at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814) | |
| at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657) | |
| at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665) | |
| at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) | |
| at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) | |
| at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) | |
| at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) | |
| at org.apache.spark.repl.Main$.main(Main.scala:31) | |
| at org.apache.spark.repl.Main.main(Main.scala) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665) | |
| at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170) | |
| at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193) | |
| at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112) | |
| at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) | |
| scala> ctx.sql("select first(num) from test_first group by category").show | |
| 15/07/22 01:29:18 INFO ParseDriver: Parsing command: select first(num) from test_first group by category | |
| 15/07/22 01:29:18 INFO ParseDriver: Parse Completed | |
| 15/07/22 01:29:18 INFO HiveMetaStore: 0: get_function: default.first | |
| 15/07/22 01:29:18 INFO audit: ugi=ubuntu ip=unknown-ip-addr cmd=get_function: default.first | |
| 15/07/22 01:29:18 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table. | |
| 15/07/22 01:29:18 ERROR RetryingHMSHandler: MetaException(message:NoSuchObjectException(message:Function default.first does not exist)) | |
| at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:4613) | |
| at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_function(HiveMetaStore.java:4740) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105) | |
| at com.sun.proxy.$Proxy13.get_function(Unknown Source) | |
| at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getFunction(HiveMetaStoreClient.java:1721) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) | |
| at com.sun.proxy.$Proxy14.getFunction(Unknown Source) | |
| at org.apache.hadoop.hive.ql.metadata.Hive.getFunction(Hive.java:2662) | |
| at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionInfoFromMetastore(FunctionRegistry.java:546) | |
| at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getQualifiedFunctionInfo(FunctionRegistry.java:579) | |
| at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionInfo(FunctionRegistry.java:645) | |
| at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionInfo(FunctionRegistry.java:652) | |
| at org.apache.spark.sql.hive.HiveFunctionRegistry.lookupFunction(hiveUdfs.scala:54) | |
| at org.apache.spark.sql.hive.HiveContext$$anon$3.org$apache$spark$sql$catalyst$analysis$OverrideFunctionRegistry$$super$lookupFunction(HiveContext.scala:376) | |
| at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$$anonfun$lookupFunction$2.apply(FunctionRegistry.scala:44) | |
| at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$$anonfun$lookupFunction$2.apply(FunctionRegistry.scala:44) | |
| at scala.Option.getOrElse(Option.scala:120) | |
| at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$class.lookupFunction(FunctionRegistry.scala:44) | |
| at org.apache.spark.sql.hive.HiveContext$$anon$3.lookupFunction(HiveContext.scala:376) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13$$anonfun$applyOrElse$5.applyOrElse(Analyzer.scala:465) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13$$anonfun$applyOrElse$5.applyOrElse(Analyzer.scala:463) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:51) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:221) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:242) | |
| at scala.collection.Iterator$$anon$11.next(Iterator.scala:328) | |
| at scala.collection.Iterator$class.foreach(Iterator.scala:727) | |
| at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) | |
| at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47) | |
| at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273) | |
| at scala.collection.AbstractIterator.to(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265) | |
| at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252) | |
| at scala.collection.AbstractIterator.toArray(Iterator.scala:1157) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:272) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:227) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan.org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionDown$1(QueryPlan.scala:75) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$1$$anonfun$apply$1.apply(QueryPlan.scala:90) | |
| at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) | |
| at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) | |
| at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) | |
| at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) | |
| at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) | |
| at scala.collection.AbstractTraversable.map(Traversable.scala:105) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$1.apply(QueryPlan.scala:89) | |
| at scala.collection.Iterator$$anon$11.next(Iterator.scala:328) | |
| at scala.collection.Iterator$class.foreach(Iterator.scala:727) | |
| at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) | |
| at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47) | |
| at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273) | |
| at scala.collection.AbstractIterator.to(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265) | |
| at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252) | |
| at scala.collection.AbstractIterator.toArray(Iterator.scala:1157) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionsDown(QueryPlan.scala:94) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressions(QueryPlan.scala:64) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13.applyOrElse(Analyzer.scala:463) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13.applyOrElse(Analyzer.scala:461) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:51) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:221) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:212) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$.apply(Analyzer.scala:461) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$.apply(Analyzer.scala:460) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:61) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:59) | |
| at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111) | |
| at scala.collection.immutable.List.foldLeft(List.scala:84) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:59) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:51) | |
| at scala.collection.immutable.List.foreach(List.scala:318) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:51) | |
| at org.apache.spark.sql.SQLContext$QueryExecution.analyzed$lzycompute(SQLContext.scala:933) | |
| at org.apache.spark.sql.SQLContext$QueryExecution.analyzed(SQLContext.scala:933) | |
| at org.apache.spark.sql.SQLContext$QueryExecution.assertAnalyzed(SQLContext.scala:931) | |
| at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:131) | |
| at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) | |
| at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:31) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:36) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:38) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:40) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:42) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:44) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:46) | |
| at $line36.$read$$iwC$$iwC$$iwC$$iwC.<init>(<console>:48) | |
| at $line36.$read$$iwC$$iwC$$iwC.<init>(<console>:50) | |
| at $line36.$read$$iwC$$iwC.<init>(<console>:52) | |
| at $line36.$read$$iwC.<init>(<console>:54) | |
| at $line36.$read.<init>(<console>:56) | |
| at $line36.$read$.<init>(<console>:60) | |
| at $line36.$read$.<clinit>(<console>) | |
| at $line36.$eval$.<init>(<console>:7) | |
| at $line36.$eval$.<clinit>(<console>) | |
| at $line36.$eval.$print(<console>) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065) | |
| at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338) | |
| at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840) | |
| at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871) | |
| at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819) | |
| at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857) | |
| at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902) | |
| at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814) | |
| at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657) | |
| at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665) | |
| at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) | |
| at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) | |
| at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) | |
| at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) | |
| at org.apache.spark.repl.Main$.main(Main.scala:31) | |
| at org.apache.spark.repl.Main.main(Main.scala) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665) | |
| at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170) | |
| at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193) | |
| at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112) | |
| at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) | |
| Caused by: NoSuchObjectException(message:Function default.first does not exist) | |
| at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_function(HiveMetaStore.java:4735) | |
| ... 141 more | |
| 15/07/22 01:29:18 INFO FunctionRegistry: Unable to lookup UDF in metastore: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:NoSuchObjectException(message:Function default.first does not exist)) | |
| java.lang.RuntimeException: Couldn't find function first | |
| at scala.sys.package$.error(package.scala:27) | |
| at org.apache.spark.sql.hive.HiveFunctionRegistry$$anonfun$1.apply(hiveUdfs.scala:55) | |
| at org.apache.spark.sql.hive.HiveFunctionRegistry$$anonfun$1.apply(hiveUdfs.scala:55) | |
| at scala.Option.getOrElse(Option.scala:120) | |
| at org.apache.spark.sql.hive.HiveFunctionRegistry.lookupFunction(hiveUdfs.scala:54) | |
| at org.apache.spark.sql.hive.HiveContext$$anon$3.org$apache$spark$sql$catalyst$analysis$OverrideFunctionRegistry$$super$lookupFunction(HiveContext.scala:376) | |
| at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$$anonfun$lookupFunction$2.apply(FunctionRegistry.scala:44) | |
| at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$$anonfun$lookupFunction$2.apply(FunctionRegistry.scala:44) | |
| at scala.Option.getOrElse(Option.scala:120) | |
| at org.apache.spark.sql.catalyst.analysis.OverrideFunctionRegistry$class.lookupFunction(FunctionRegistry.scala:44) | |
| at org.apache.spark.sql.hive.HiveContext$$anon$3.lookupFunction(HiveContext.scala:376) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13$$anonfun$applyOrElse$5.applyOrElse(Analyzer.scala:465) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13$$anonfun$applyOrElse$5.applyOrElse(Analyzer.scala:463) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:51) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:221) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:242) | |
| at scala.collection.Iterator$$anon$11.next(Iterator.scala:328) | |
| at scala.collection.Iterator$class.foreach(Iterator.scala:727) | |
| at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) | |
| at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47) | |
| at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273) | |
| at scala.collection.AbstractIterator.to(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265) | |
| at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252) | |
| at scala.collection.AbstractIterator.toArray(Iterator.scala:1157) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:272) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:227) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan.org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionDown$1(QueryPlan.scala:75) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$1$$anonfun$apply$1.apply(QueryPlan.scala:90) | |
| at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) | |
| at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) | |
| at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) | |
| at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) | |
| at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) | |
| at scala.collection.AbstractTraversable.map(Traversable.scala:105) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$1.apply(QueryPlan.scala:89) | |
| at scala.collection.Iterator$$anon$11.next(Iterator.scala:328) | |
| at scala.collection.Iterator$class.foreach(Iterator.scala:727) | |
| at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) | |
| at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103) | |
| at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47) | |
| at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273) | |
| at scala.collection.AbstractIterator.to(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265) | |
| at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157) | |
| at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252) | |
| at scala.collection.AbstractIterator.toArray(Iterator.scala:1157) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionsDown(QueryPlan.scala:94) | |
| at org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressions(QueryPlan.scala:64) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13.applyOrElse(Analyzer.scala:463) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$13.applyOrElse(Analyzer.scala:461) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:222) | |
| at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:51) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:221) | |
| at org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:212) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$.apply(Analyzer.scala:461) | |
| at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$.apply(Analyzer.scala:460) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:61) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:59) | |
| at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111) | |
| at scala.collection.immutable.List.foldLeft(List.scala:84) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:59) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:51) | |
| at scala.collection.immutable.List.foreach(List.scala:318) | |
| at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:51) | |
| at org.apache.spark.sql.SQLContext$QueryExecution.analyzed$lzycompute(SQLContext.scala:933) | |
| at org.apache.spark.sql.SQLContext$QueryExecution.analyzed(SQLContext.scala:933) | |
| at org.apache.spark.sql.SQLContext$QueryExecution.assertAnalyzed(SQLContext.scala:931) | |
| at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:131) | |
| at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) | |
| at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:31) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:36) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:38) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:40) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:42) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:44) | |
| at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:46) | |
| at $iwC$$iwC$$iwC$$iwC.<init>(<console>:48) | |
| at $iwC$$iwC$$iwC.<init>(<console>:50) | |
| at $iwC$$iwC.<init>(<console>:52) | |
| at $iwC.<init>(<console>:54) | |
| at <init>(<console>:56) | |
| at .<init>(<console>:60) | |
| at .<clinit>(<console>) | |
| at .<init>(<console>:7) | |
| at .<clinit>(<console>) | |
| at $print(<console>) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065) | |
| at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338) | |
| at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840) | |
| at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871) | |
| at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819) | |
| at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857) | |
| at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902) | |
| at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814) | |
| at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657) | |
| at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665) | |
| at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) | |
| at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945) | |
| at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) | |
| at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945) | |
| at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059) | |
| at org.apache.spark.repl.Main$.main(Main.scala:31) | |
| at org.apache.spark.repl.Main.main(Main.scala) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) | |
| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) | |
| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
| at java.lang.reflect.Method.invoke(Method.java:606) | |
| at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665) | |
| at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170) | |
| at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193) | |
| at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112) | |
| at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) | |
| scala> ctx.sql("select first_value(num) from test_first group by category").show | |
| 15/07/22 01:29:18 INFO ParseDriver: Parsing command: select first_value(num) from test_first group by category | |
| 15/07/22 01:29:18 INFO ParseDriver: Parse Completed | |
| 15/07/22 01:29:18 INFO MemoryStore: ensureFreeSpace(294016) called with curMem=466171, maxMem=26671746908 | |
| 15/07/22 01:29:18 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 287.1 KB, free 24.8 GB) | |
| 15/07/22 01:29:18 INFO MemoryStore: ensureFreeSpace(19750) called with curMem=760187, maxMem=26671746908 | |
| 15/07/22 01:29:18 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 19.3 KB, free 24.8 GB) | |
| 15/07/22 01:29:18 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on localhost:35934 (size: 19.3 KB, free: 24.8 GB) | |
| 15/07/22 01:29:18 INFO SparkContext: Created broadcast 5 from show at <console>:31 | |
| 15/07/22 01:29:18 INFO Exchange: Using SparkSqlSerializer2. | |
| 15/07/22 01:29:19 INFO SparkContext: Starting job: show at <console>:31 | |
| 15/07/22 01:29:19 INFO FileInputFormat: Total input paths to process : 1 | |
| 15/07/22 01:29:19 INFO DAGScheduler: Registering RDD 11 (show at <console>:31) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Got job 3 (show at <console>:31) with 1 output partitions (allowLocal=false) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Final stage: ResultStage 4(show at <console>:31) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 3) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 3) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Submitting ShuffleMapStage 3 (MapPartitionsRDD[11] at show at <console>:31), which has no missing parents | |
| 15/07/22 01:29:19 INFO MemoryStore: ensureFreeSpace(6336) called with curMem=779937, maxMem=26671746908 | |
| 15/07/22 01:29:19 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 6.2 KB, free 24.8 GB) | |
| 15/07/22 01:29:19 INFO MemoryStore: ensureFreeSpace(3436) called with curMem=786273, maxMem=26671746908 | |
| 15/07/22 01:29:19 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 3.4 KB, free 24.8 GB) | |
| 15/07/22 01:29:19 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on localhost:35934 (size: 3.4 KB, free: 24.8 GB) | |
| 15/07/22 01:29:19 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:874 | |
| 15/07/22 01:29:19 INFO DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 3 (MapPartitionsRDD[11] at show at <console>:31) | |
| 15/07/22 01:29:19 INFO TaskSchedulerImpl: Adding task set 3.0 with 2 tasks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 4, localhost, PROCESS_LOCAL, 1562 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 1.0 in stage 3.0 (TID 5, localhost, PROCESS_LOCAL, 1562 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 1.0 in stage 3.0 (TID 5) | |
| 15/07/22 01:29:19 INFO Executor: Running task 0.0 in stage 3.0 (TID 4) | |
| 15/07/22 01:29:19 INFO HadoopRDD: Input split: file:/home/ubuntu/spark_sql_first.jsonlines:14+15 | |
| 15/07/22 01:29:19 INFO HadoopRDD: Input split: file:/home/ubuntu/spark_sql_first.jsonlines:0+14 | |
| 15/07/22 01:29:19 INFO Executor: Finished task 1.0 in stage 3.0 (TID 5). 2203 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 1.0 in stage 3.0 (TID 5) in 364 ms on localhost (1/2) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 0.0 in stage 3.0 (TID 4). 2203 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 4) in 436 ms on localhost (2/2) | |
| 15/07/22 01:29:19 INFO DAGScheduler: ShuffleMapStage 3 (show at <console>:31) finished in 0.436 s | |
| 15/07/22 01:29:19 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool | |
| 15/07/22 01:29:19 INFO DAGScheduler: looking for newly runnable stages | |
| 15/07/22 01:29:19 INFO DAGScheduler: running: Set() | |
| 15/07/22 01:29:19 INFO DAGScheduler: waiting: Set(ResultStage 4) | |
| 15/07/22 01:29:19 INFO DAGScheduler: failed: Set() | |
| 15/07/22 01:29:19 INFO DAGScheduler: Missing parents for ResultStage 4: List() | |
| 15/07/22 01:29:19 INFO DAGScheduler: Submitting ResultStage 4 (MapPartitionsRDD[15] at show at <console>:31), which is now runnable | |
| 15/07/22 01:29:19 INFO MemoryStore: ensureFreeSpace(9288) called with curMem=789709, maxMem=26671746908 | |
| 15/07/22 01:29:19 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 9.1 KB, free 24.8 GB) | |
| 15/07/22 01:29:19 INFO MemoryStore: ensureFreeSpace(4679) called with curMem=798997, maxMem=26671746908 | |
| 15/07/22 01:29:19 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 4.6 KB, free 24.8 GB) | |
| 15/07/22 01:29:19 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on localhost:35934 (size: 4.6 KB, free: 24.8 GB) | |
| 15/07/22 01:29:19 INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:874 | |
| 15/07/22 01:29:19 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (MapPartitionsRDD[15] at show at <console>:31) | |
| 15/07/22 01:29:19 INFO TaskSchedulerImpl: Adding task set 4.0 with 1 tasks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 6, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 0.0 in stage 4.0 (TID 6) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 0.0 in stage 4.0 (TID 6). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 6) in 39 ms on localhost (1/1) | |
| 15/07/22 01:29:19 INFO DAGScheduler: ResultStage 4 (show at <console>:31) finished in 0.039 s | |
| 15/07/22 01:29:19 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool | |
| 15/07/22 01:29:19 INFO DAGScheduler: Job 3 finished: show at <console>:31, took 0.523702 s | |
| 15/07/22 01:29:19 INFO SparkContext: Starting job: show at <console>:31 | |
| 15/07/22 01:29:19 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 164 bytes | |
| 15/07/22 01:29:19 INFO DAGScheduler: Got job 4 (show at <console>:31) with 199 output partitions (allowLocal=false) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Final stage: ResultStage 6(show at <console>:31) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 5) | |
| 15/07/22 01:29:19 INFO DAGScheduler: Missing parents: List() | |
| 15/07/22 01:29:19 INFO DAGScheduler: Submitting ResultStage 6 (MapPartitionsRDD[15] at show at <console>:31), which has no missing parents | |
| 15/07/22 01:29:19 INFO MemoryStore: ensureFreeSpace(9288) called with curMem=803676, maxMem=26671746908 | |
| 15/07/22 01:29:19 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 9.1 KB, free 24.8 GB) | |
| 15/07/22 01:29:19 INFO MemoryStore: ensureFreeSpace(4679) called with curMem=812964, maxMem=26671746908 | |
| 15/07/22 01:29:19 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 4.6 KB, free 24.8 GB) | |
| 15/07/22 01:29:19 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on localhost:35934 (size: 4.6 KB, free: 24.8 GB) | |
| 15/07/22 01:29:19 INFO SparkContext: Created broadcast 8 from broadcast at DAGScheduler.scala:874 | |
| 15/07/22 01:29:19 INFO DAGScheduler: Submitting 199 missing tasks from ResultStage 6 (MapPartitionsRDD[15] at show at <console>:31) | |
| 15/07/22 01:29:19 INFO TaskSchedulerImpl: Adding task set 6.0 with 199 tasks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 7, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 1.0 in stage 6.0 (TID 8, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 2.0 in stage 6.0 (TID 9, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 3.0 in stage 6.0 (TID 10, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 4.0 in stage 6.0 (TID 11, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 5.0 in stage 6.0 (TID 12, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 6.0 in stage 6.0 (TID 13, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 7.0 in stage 6.0 (TID 14, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 8.0 in stage 6.0 (TID 15, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 9.0 in stage 6.0 (TID 16, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 10.0 in stage 6.0 (TID 17, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 11.0 in stage 6.0 (TID 18, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 12.0 in stage 6.0 (TID 19, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 13.0 in stage 6.0 (TID 20, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 14.0 in stage 6.0 (TID 21, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 15.0 in stage 6.0 (TID 22, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 16.0 in stage 6.0 (TID 23, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 17.0 in stage 6.0 (TID 24, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 18.0 in stage 6.0 (TID 25, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 19.0 in stage 6.0 (TID 26, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 20.0 in stage 6.0 (TID 27, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 21.0 in stage 6.0 (TID 28, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 22.0 in stage 6.0 (TID 29, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 23.0 in stage 6.0 (TID 30, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 24.0 in stage 6.0 (TID 31, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 25.0 in stage 6.0 (TID 32, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 26.0 in stage 6.0 (TID 33, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 27.0 in stage 6.0 (TID 34, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 28.0 in stage 6.0 (TID 35, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 29.0 in stage 6.0 (TID 36, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 30.0 in stage 6.0 (TID 37, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 31.0 in stage 6.0 (TID 38, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 0.0 in stage 6.0 (TID 7) | |
| 15/07/22 01:29:19 INFO Executor: Running task 1.0 in stage 6.0 (TID 8) | |
| 15/07/22 01:29:19 INFO Executor: Running task 2.0 in stage 6.0 (TID 9) | |
| 15/07/22 01:29:19 INFO Executor: Running task 3.0 in stage 6.0 (TID 10) | |
| 15/07/22 01:29:19 INFO Executor: Running task 4.0 in stage 6.0 (TID 11) | |
| 15/07/22 01:29:19 INFO Executor: Running task 5.0 in stage 6.0 (TID 12) | |
| 15/07/22 01:29:19 INFO Executor: Running task 6.0 in stage 6.0 (TID 13) | |
| 15/07/22 01:29:19 INFO Executor: Running task 7.0 in stage 6.0 (TID 14) | |
| 15/07/22 01:29:19 INFO Executor: Running task 8.0 in stage 6.0 (TID 15) | |
| 15/07/22 01:29:19 INFO Executor: Running task 9.0 in stage 6.0 (TID 16) | |
| 15/07/22 01:29:19 INFO Executor: Running task 10.0 in stage 6.0 (TID 17) | |
| 15/07/22 01:29:19 INFO Executor: Running task 11.0 in stage 6.0 (TID 18) | |
| 15/07/22 01:29:19 INFO Executor: Running task 12.0 in stage 6.0 (TID 19) | |
| 15/07/22 01:29:19 INFO Executor: Running task 13.0 in stage 6.0 (TID 20) | |
| 15/07/22 01:29:19 INFO Executor: Running task 14.0 in stage 6.0 (TID 21) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 15.0 in stage 6.0 (TID 22) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 0.0 in stage 6.0 (TID 7). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 16.0 in stage 6.0 (TID 23) | |
| 15/07/22 01:29:19 INFO Executor: Running task 18.0 in stage 6.0 (TID 25) | |
| 15/07/22 01:29:19 INFO Executor: Running task 19.0 in stage 6.0 (TID 26) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 17.0 in stage 6.0 (TID 24) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 3.0 in stage 6.0 (TID 10). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 21.0 in stage 6.0 (TID 28) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 4.0 in stage 6.0 (TID 11). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 20.0 in stage 6.0 (TID 27) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 22.0 in stage 6.0 (TID 29) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 7.0 in stage 6.0 (TID 14). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 6.0 in stage 6.0 (TID 13). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 2.0 in stage 6.0 (TID 9). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 1.0 in stage 6.0 (TID 8). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 24.0 in stage 6.0 (TID 31) | |
| 15/07/22 01:29:19 INFO Executor: Running task 28.0 in stage 6.0 (TID 35) | |
| 15/07/22 01:29:19 INFO Executor: Running task 26.0 in stage 6.0 (TID 33) | |
| 15/07/22 01:29:19 INFO Executor: Running task 23.0 in stage 6.0 (TID 30) | |
| 15/07/22 01:29:19 INFO Executor: Running task 25.0 in stage 6.0 (TID 32) | |
| 15/07/22 01:29:19 INFO Executor: Running task 27.0 in stage 6.0 (TID 34) | |
| 15/07/22 01:29:19 INFO Executor: Running task 29.0 in stage 6.0 (TID 36) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 32.0 in stage 6.0 (TID 39, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 31.0 in stage 6.0 (TID 38) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 30.0 in stage 6.0 (TID 37) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 7) in 44 ms on localhost (1/199) | |
| 15/07/22 01:29:19 INFO Executor: Running task 32.0 in stage 6.0 (TID 39) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 18.0 in stage 6.0 (TID 25). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 10.0 in stage 6.0 (TID 17). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 33.0 in stage 6.0 (TID 40, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 33.0 in stage 6.0 (TID 40) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 5.0 in stage 6.0 (TID 12). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 3.0 in stage 6.0 (TID 10) in 44 ms on localhost (2/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 34.0 in stage 6.0 (TID 41, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 34.0 in stage 6.0 (TID 41) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 35.0 in stage 6.0 (TID 42, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 11.0 in stage 6.0 (TID 18). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 35.0 in stage 6.0 (TID 42) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 21.0 in stage 6.0 (TID 28). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 36.0 in stage 6.0 (TID 43, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 14.0 in stage 6.0 (TID 21). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 36.0 in stage 6.0 (TID 43) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 4.0 in stage 6.0 (TID 11) in 50 ms on localhost (3/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 28.0 in stage 6.0 (TID 35). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 7.0 in stage 6.0 (TID 14) in 49 ms on localhost (4/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 6.0 in stage 6.0 (TID 13) in 50 ms on localhost (5/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 25.0 in stage 6.0 (TID 32). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 37.0 in stage 6.0 (TID 44, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 9.0 in stage 6.0 (TID 16). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 27.0 in stage 6.0 (TID 34). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 37.0 in stage 6.0 (TID 44) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 38.0 in stage 6.0 (TID 45, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 38.0 in stage 6.0 (TID 45) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 2.0 in stage 6.0 (TID 9) in 53 ms on localhost (6/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 24.0 in stage 6.0 (TID 31). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 1.0 in stage 6.0 (TID 8) in 55 ms on localhost (7/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 26.0 in stage 6.0 (TID 33). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 12.0 in stage 6.0 (TID 19). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 39.0 in stage 6.0 (TID 46, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 39.0 in stage 6.0 (TID 46) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 40.0 in stage 6.0 (TID 47, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 40.0 in stage 6.0 (TID 47) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 18.0 in stage 6.0 (TID 25) in 48 ms on localhost (8/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 10.0 in stage 6.0 (TID 17) in 52 ms on localhost (9/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 41.0 in stage 6.0 (TID 48, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 41.0 in stage 6.0 (TID 48) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 5.0 in stage 6.0 (TID 12) in 55 ms on localhost (10/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 33.0 in stage 6.0 (TID 40). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 42.0 in stage 6.0 (TID 49, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 8.0 in stage 6.0 (TID 15). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 42.0 in stage 6.0 (TID 49) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 11.0 in stage 6.0 (TID 18) in 54 ms on localhost (11/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 34.0 in stage 6.0 (TID 41). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 43.0 in stage 6.0 (TID 50, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 43.0 in stage 6.0 (TID 50) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 36.0 in stage 6.0 (TID 43). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 35.0 in stage 6.0 (TID 42). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 44.0 in stage 6.0 (TID 51, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 31.0 in stage 6.0 (TID 38). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 44.0 in stage 6.0 (TID 51) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 21.0 in stage 6.0 (TID 28) in 50 ms on localhost (12/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 45.0 in stage 6.0 (TID 52, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 45.0 in stage 6.0 (TID 52) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 14.0 in stage 6.0 (TID 21) in 55 ms on localhost (13/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 46.0 in stage 6.0 (TID 53, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 46.0 in stage 6.0 (TID 53) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 28.0 in stage 6.0 (TID 35) in 50 ms on localhost (14/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 25.0 in stage 6.0 (TID 32) in 51 ms on localhost (15/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 37.0 in stage 6.0 (TID 44). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 47.0 in stage 6.0 (TID 54, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 22.0 in stage 6.0 (TID 29). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 47.0 in stage 6.0 (TID 54) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 48.0 in stage 6.0 (TID 55, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 48.0 in stage 6.0 (TID 55) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 9.0 in stage 6.0 (TID 16) in 61 ms on localhost (16/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 27.0 in stage 6.0 (TID 34) in 52 ms on localhost (17/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 29.0 in stage 6.0 (TID 36). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 49.0 in stage 6.0 (TID 56, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 49.0 in stage 6.0 (TID 56) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 39.0 in stage 6.0 (TID 46). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 38.0 in stage 6.0 (TID 45). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 50.0 in stage 6.0 (TID 57, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 50.0 in stage 6.0 (TID 57) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 51.0 in stage 6.0 (TID 58, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 51.0 in stage 6.0 (TID 58) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 26.0 in stage 6.0 (TID 33) in 56 ms on localhost (18/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 41.0 in stage 6.0 (TID 48). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 20.0 in stage 6.0 (TID 27). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 12.0 in stage 6.0 (TID 19) in 65 ms on localhost (19/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 23.0 in stage 6.0 (TID 30). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 24.0 in stage 6.0 (TID 31) in 60 ms on localhost (20/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 40.0 in stage 6.0 (TID 47). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 42.0 in stage 6.0 (TID 49). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 52.0 in stage 6.0 (TID 59, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 52.0 in stage 6.0 (TID 59) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 33.0 in stage 6.0 (TID 40) in 28 ms on localhost (21/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 53.0 in stage 6.0 (TID 60, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 53.0 in stage 6.0 (TID 60) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 8.0 in stage 6.0 (TID 15) in 70 ms on localhost (22/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 44.0 in stage 6.0 (TID 51). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 54.0 in stage 6.0 (TID 61, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 54.0 in stage 6.0 (TID 61) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 55.0 in stage 6.0 (TID 62, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 17.0 in stage 6.0 (TID 24). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 46.0 in stage 6.0 (TID 53). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 55.0 in stage 6.0 (TID 62) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 56.0 in stage 6.0 (TID 63, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 56.0 in stage 6.0 (TID 63) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 34.0 in stage 6.0 (TID 41) in 29 ms on localhost (23/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 36.0 in stage 6.0 (TID 43) in 28 ms on localhost (24/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 16.0 in stage 6.0 (TID 23). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 57.0 in stage 6.0 (TID 64, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 48.0 in stage 6.0 (TID 55). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 57.0 in stage 6.0 (TID 64) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 35.0 in stage 6.0 (TID 42) in 30 ms on localhost (25/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 31.0 in stage 6.0 (TID 38) in 62 ms on localhost (26/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 58.0 in stage 6.0 (TID 65, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 58.0 in stage 6.0 (TID 65) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 43.0 in stage 6.0 (TID 50). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 45.0 in stage 6.0 (TID 52). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 47.0 in stage 6.0 (TID 54). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 59.0 in stage 6.0 (TID 66, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 59.0 in stage 6.0 (TID 66) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 22.0 in stage 6.0 (TID 29) in 69 ms on localhost (27/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 37.0 in stage 6.0 (TID 44) in 28 ms on localhost (28/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 60.0 in stage 6.0 (TID 67, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 15.0 in stage 6.0 (TID 22). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 60.0 in stage 6.0 (TID 67) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 61.0 in stage 6.0 (TID 68, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 61.0 in stage 6.0 (TID 68) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 29.0 in stage 6.0 (TID 36) in 68 ms on localhost (29/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 13.0 in stage 6.0 (TID 20). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 62.0 in stage 6.0 (TID 69, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 62.0 in stage 6.0 (TID 69) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 38.0 in stage 6.0 (TID 45) in 30 ms on localhost (30/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 39.0 in stage 6.0 (TID 46) in 28 ms on localhost (31/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 19.0 in stage 6.0 (TID 26). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 63.0 in stage 6.0 (TID 70, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 63.0 in stage 6.0 (TID 70) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 64.0 in stage 6.0 (TID 71, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 52.0 in stage 6.0 (TID 59). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 64.0 in stage 6.0 (TID 71) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 65.0 in stage 6.0 (TID 72, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 49.0 in stage 6.0 (TID 56). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 65.0 in stage 6.0 (TID 72) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 41.0 in stage 6.0 (TID 48) in 29 ms on localhost (32/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 50.0 in stage 6.0 (TID 57). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 20.0 in stage 6.0 (TID 27) in 77 ms on localhost (33/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 51.0 in stage 6.0 (TID 58). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 54.0 in stage 6.0 (TID 61). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 66.0 in stage 6.0 (TID 73, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 66.0 in stage 6.0 (TID 73) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 23.0 in stage 6.0 (TID 30) in 77 ms on localhost (34/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 32.0 in stage 6.0 (TID 39). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 40.0 in stage 6.0 (TID 47) in 33 ms on localhost (35/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 57.0 in stage 6.0 (TID 64). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 67.0 in stage 6.0 (TID 74, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 53.0 in stage 6.0 (TID 60). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 58.0 in stage 6.0 (TID 65). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 42.0 in stage 6.0 (TID 49) in 32 ms on localhost (36/199) | |
| 15/07/22 01:29:19 INFO Executor: Running task 67.0 in stage 6.0 (TID 74) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 68.0 in stage 6.0 (TID 75, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 68.0 in stage 6.0 (TID 75) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 56.0 in stage 6.0 (TID 63). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 69.0 in stage 6.0 (TID 76, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 30.0 in stage 6.0 (TID 37). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 69.0 in stage 6.0 (TID 76) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 55.0 in stage 6.0 (TID 62). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 44.0 in stage 6.0 (TID 51) in 31 ms on localhost (37/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 70.0 in stage 6.0 (TID 77, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 70.0 in stage 6.0 (TID 77) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 17.0 in stage 6.0 (TID 24) in 85 ms on localhost (38/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 46.0 in stage 6.0 (TID 53) in 31 ms on localhost (39/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 60.0 in stage 6.0 (TID 67). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 62.0 in stage 6.0 (TID 69). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 71.0 in stage 6.0 (TID 78, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 61.0 in stage 6.0 (TID 68). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 71.0 in stage 6.0 (TID 78) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 59.0 in stage 6.0 (TID 66). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 72.0 in stage 6.0 (TID 79, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 72.0 in stage 6.0 (TID 79) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 16.0 in stage 6.0 (TID 23) in 89 ms on localhost (40/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 73.0 in stage 6.0 (TID 80, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 73.0 in stage 6.0 (TID 80) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 48.0 in stage 6.0 (TID 55) in 32 ms on localhost (41/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 74.0 in stage 6.0 (TID 81, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 74.0 in stage 6.0 (TID 81) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 43.0 in stage 6.0 (TID 50) in 38 ms on localhost (42/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 75.0 in stage 6.0 (TID 82, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 75.0 in stage 6.0 (TID 82) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 45.0 in stage 6.0 (TID 52) in 38 ms on localhost (43/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 65.0 in stage 6.0 (TID 72). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 47.0 in stage 6.0 (TID 54) in 36 ms on localhost (44/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 66.0 in stage 6.0 (TID 73). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 76.0 in stage 6.0 (TID 83, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 63.0 in stage 6.0 (TID 70). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 67.0 in stage 6.0 (TID 74). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 15.0 in stage 6.0 (TID 22) in 94 ms on localhost (45/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 76.0 in stage 6.0 (TID 83) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 68.0 in stage 6.0 (TID 75). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 77.0 in stage 6.0 (TID 84, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 77.0 in stage 6.0 (TID 84) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 69.0 in stage 6.0 (TID 76). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 13.0 in stage 6.0 (TID 20) in 96 ms on localhost (46/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 78.0 in stage 6.0 (TID 85, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 70.0 in stage 6.0 (TID 77). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 78.0 in stage 6.0 (TID 85) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 19.0 in stage 6.0 (TID 26) in 95 ms on localhost (47/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 79.0 in stage 6.0 (TID 86, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 79.0 in stage 6.0 (TID 86) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 52.0 in stage 6.0 (TID 59) in 34 ms on localhost (48/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 71.0 in stage 6.0 (TID 78). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 80.0 in stage 6.0 (TID 87, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 80.0 in stage 6.0 (TID 87) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 49.0 in stage 6.0 (TID 56) in 42 ms on localhost (49/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 73.0 in stage 6.0 (TID 80). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 81.0 in stage 6.0 (TID 88, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 64.0 in stage 6.0 (TID 71). 1115 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 81.0 in stage 6.0 (TID 88) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 82.0 in stage 6.0 (TID 89, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 82.0 in stage 6.0 (TID 89) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 50.0 in stage 6.0 (TID 57) in 43 ms on localhost (50/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 51.0 in stage 6.0 (TID 58) in 41 ms on localhost (51/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 72.0 in stage 6.0 (TID 79). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 83.0 in stage 6.0 (TID 90, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 83.0 in stage 6.0 (TID 90) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 54.0 in stage 6.0 (TID 61) in 38 ms on localhost (52/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 84.0 in stage 6.0 (TID 91, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 84.0 in stage 6.0 (TID 91) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 76.0 in stage 6.0 (TID 83). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 77.0 in stage 6.0 (TID 84). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 85.0 in stage 6.0 (TID 92, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 85.0 in stage 6.0 (TID 92) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 32.0 in stage 6.0 (TID 39) in 72 ms on localhost (53/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 74.0 in stage 6.0 (TID 81). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 86.0 in stage 6.0 (TID 93, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 86.0 in stage 6.0 (TID 93) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 78.0 in stage 6.0 (TID 85). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 75.0 in stage 6.0 (TID 82). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 87.0 in stage 6.0 (TID 94, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 79.0 in stage 6.0 (TID 86). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 57.0 in stage 6.0 (TID 64) in 40 ms on localhost (54/199) | |
| 15/07/22 01:29:19 INFO Executor: Running task 87.0 in stage 6.0 (TID 94) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 88.0 in stage 6.0 (TID 95, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 80.0 in stage 6.0 (TID 87). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 88.0 in stage 6.0 (TID 95) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 53.0 in stage 6.0 (TID 60) in 46 ms on localhost (55/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 58.0 in stage 6.0 (TID 65) in 40 ms on localhost (56/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 89.0 in stage 6.0 (TID 96, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 89.0 in stage 6.0 (TID 96) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 56.0 in stage 6.0 (TID 63) in 44 ms on localhost (57/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 81.0 in stage 6.0 (TID 88). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 90.0 in stage 6.0 (TID 97, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 30.0 in stage 6.0 (TID 37) in 107 ms on localhost (58/199) | |
| 15/07/22 01:29:19 INFO Executor: Running task 90.0 in stage 6.0 (TID 97) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 82.0 in stage 6.0 (TID 89). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 91.0 in stage 6.0 (TID 98, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 91.0 in stage 6.0 (TID 98) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 55.0 in stage 6.0 (TID 62) in 48 ms on localhost (59/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 60.0 in stage 6.0 (TID 67) in 42 ms on localhost (60/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 92.0 in stage 6.0 (TID 99, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 92.0 in stage 6.0 (TID 99) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 62.0 in stage 6.0 (TID 69) in 42 ms on localhost (61/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 93.0 in stage 6.0 (TID 100, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 93.0 in stage 6.0 (TID 100) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 61.0 in stage 6.0 (TID 68) in 44 ms on localhost (62/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 84.0 in stage 6.0 (TID 91). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 94.0 in stage 6.0 (TID 101, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 94.0 in stage 6.0 (TID 101) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 85.0 in stage 6.0 (TID 92). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 59.0 in stage 6.0 (TID 66) in 49 ms on localhost (63/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 95.0 in stage 6.0 (TID 102, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 95.0 in stage 6.0 (TID 102) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 96.0 in stage 6.0 (TID 103, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 96.0 in stage 6.0 (TID 103) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 65.0 in stage 6.0 (TID 72) in 45 ms on localhost (64/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 83.0 in stage 6.0 (TID 90). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 97.0 in stage 6.0 (TID 104, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 88.0 in stage 6.0 (TID 95). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 97.0 in stage 6.0 (TID 104) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 66.0 in stage 6.0 (TID 73) in 45 ms on localhost (65/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 63.0 in stage 6.0 (TID 70) in 49 ms on localhost (66/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 89.0 in stage 6.0 (TID 96). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 98.0 in stage 6.0 (TID 105, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 98.0 in stage 6.0 (TID 105) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 67.0 in stage 6.0 (TID 74) in 45 ms on localhost (67/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 86.0 in stage 6.0 (TID 93). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 99.0 in stage 6.0 (TID 106, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 90.0 in stage 6.0 (TID 97). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 99.0 in stage 6.0 (TID 106) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 100.0 in stage 6.0 (TID 107, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 91.0 in stage 6.0 (TID 98). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 100.0 in stage 6.0 (TID 107) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 68.0 in stage 6.0 (TID 75) in 45 ms on localhost (68/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 87.0 in stage 6.0 (TID 94). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 92.0 in stage 6.0 (TID 99). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 101.0 in stage 6.0 (TID 108, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 101.0 in stage 6.0 (TID 108) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 69.0 in stage 6.0 (TID 76) in 46 ms on localhost (69/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 70.0 in stage 6.0 (TID 77) in 45 ms on localhost (70/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 93.0 in stage 6.0 (TID 100). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 102.0 in stage 6.0 (TID 109, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 94.0 in stage 6.0 (TID 101). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 102.0 in stage 6.0 (TID 109) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 71.0 in stage 6.0 (TID 78) in 45 ms on localhost (71/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 103.0 in stage 6.0 (TID 110, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 95.0 in stage 6.0 (TID 102). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 103.0 in stage 6.0 (TID 110) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 104.0 in stage 6.0 (TID 111, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 104.0 in stage 6.0 (TID 111) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 73.0 in stage 6.0 (TID 80) in 45 ms on localhost (72/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 105.0 in stage 6.0 (TID 112, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 105.0 in stage 6.0 (TID 112) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 64.0 in stage 6.0 (TID 71) in 59 ms on localhost (73/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 72.0 in stage 6.0 (TID 79) in 48 ms on localhost (74/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 98.0 in stage 6.0 (TID 105). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 106.0 in stage 6.0 (TID 113, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 106.0 in stage 6.0 (TID 113) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 97.0 in stage 6.0 (TID 104). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 107.0 in stage 6.0 (TID 114, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 107.0 in stage 6.0 (TID 114) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 76.0 in stage 6.0 (TID 83) in 46 ms on localhost (75/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 100.0 in stage 6.0 (TID 107). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 108.0 in stage 6.0 (TID 115, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 108.0 in stage 6.0 (TID 115) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 77.0 in stage 6.0 (TID 84) in 46 ms on localhost (76/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 101.0 in stage 6.0 (TID 108). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 109.0 in stage 6.0 (TID 116, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 96.0 in stage 6.0 (TID 103). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 109.0 in stage 6.0 (TID 116) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 74.0 in stage 6.0 (TID 81) in 52 ms on localhost (77/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 110.0 in stage 6.0 (TID 117, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 102.0 in stage 6.0 (TID 109). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 110.0 in stage 6.0 (TID 117) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 78.0 in stage 6.0 (TID 85) in 47 ms on localhost (78/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 75.0 in stage 6.0 (TID 82) in 52 ms on localhost (79/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 99.0 in stage 6.0 (TID 106). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 103.0 in stage 6.0 (TID 110). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 111.0 in stage 6.0 (TID 118, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 111.0 in stage 6.0 (TID 118) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 104.0 in stage 6.0 (TID 111). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 112.0 in stage 6.0 (TID 119, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 112.0 in stage 6.0 (TID 119) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 79.0 in stage 6.0 (TID 86) in 49 ms on localhost (80/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 113.0 in stage 6.0 (TID 120, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 113.0 in stage 6.0 (TID 120) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 114.0 in stage 6.0 (TID 121, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 114.0 in stage 6.0 (TID 121) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 80.0 in stage 6.0 (TID 87) in 50 ms on localhost (81/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 81.0 in stage 6.0 (TID 88) in 48 ms on localhost (82/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 82.0 in stage 6.0 (TID 89) in 48 ms on localhost (83/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 106.0 in stage 6.0 (TID 113). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 115.0 in stage 6.0 (TID 122, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 115.0 in stage 6.0 (TID 122) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 84.0 in stage 6.0 (TID 91) in 45 ms on localhost (84/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 108.0 in stage 6.0 (TID 115). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 116.0 in stage 6.0 (TID 123, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 116.0 in stage 6.0 (TID 123) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 85.0 in stage 6.0 (TID 92) in 46 ms on localhost (85/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 117.0 in stage 6.0 (TID 124, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 117.0 in stage 6.0 (TID 124) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 109.0 in stage 6.0 (TID 116). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 118.0 in stage 6.0 (TID 125, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 118.0 in stage 6.0 (TID 125) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 105.0 in stage 6.0 (TID 112). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 119.0 in stage 6.0 (TID 126, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 83.0 in stage 6.0 (TID 90) in 51 ms on localhost (86/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 110.0 in stage 6.0 (TID 117). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 88.0 in stage 6.0 (TID 95) in 44 ms on localhost (87/199) | |
| 15/07/22 01:29:19 INFO Executor: Running task 119.0 in stage 6.0 (TID 126) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 120.0 in stage 6.0 (TID 127, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 120.0 in stage 6.0 (TID 127) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 89.0 in stage 6.0 (TID 96) in 44 ms on localhost (88/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 111.0 in stage 6.0 (TID 118). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 121.0 in stage 6.0 (TID 128, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 121.0 in stage 6.0 (TID 128) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 112.0 in stage 6.0 (TID 119). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 107.0 in stage 6.0 (TID 114). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 122.0 in stage 6.0 (TID 129, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 122.0 in stage 6.0 (TID 129) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 86.0 in stage 6.0 (TID 93) in 50 ms on localhost (89/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 90.0 in stage 6.0 (TID 97) in 45 ms on localhost (90/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 91.0 in stage 6.0 (TID 98) in 44 ms on localhost (91/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 123.0 in stage 6.0 (TID 130, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 123.0 in stage 6.0 (TID 130) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 124.0 in stage 6.0 (TID 131, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 124.0 in stage 6.0 (TID 131) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 125.0 in stage 6.0 (TID 132, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 125.0 in stage 6.0 (TID 132) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 87.0 in stage 6.0 (TID 94) in 53 ms on localhost (92/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 114.0 in stage 6.0 (TID 121). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 92.0 in stage 6.0 (TID 99) in 45 ms on localhost (93/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 113.0 in stage 6.0 (TID 120). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 115.0 in stage 6.0 (TID 122). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 126.0 in stage 6.0 (TID 133, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 126.0 in stage 6.0 (TID 133) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 116.0 in stage 6.0 (TID 123). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 93.0 in stage 6.0 (TID 100) in 46 ms on localhost (94/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 94.0 in stage 6.0 (TID 101) in 44 ms on localhost (95/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 127.0 in stage 6.0 (TID 134, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 127.0 in stage 6.0 (TID 134) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 118.0 in stage 6.0 (TID 125). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 128.0 in stage 6.0 (TID 135, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 128.0 in stage 6.0 (TID 135) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 95.0 in stage 6.0 (TID 102) in 44 ms on localhost (96/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 117.0 in stage 6.0 (TID 124). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 98.0 in stage 6.0 (TID 105) in 41 ms on localhost (97/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 120.0 in stage 6.0 (TID 127). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 129.0 in stage 6.0 (TID 136, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 129.0 in stage 6.0 (TID 136) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 97.0 in stage 6.0 (TID 104) in 44 ms on localhost (98/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 130.0 in stage 6.0 (TID 137, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 130.0 in stage 6.0 (TID 137) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 100.0 in stage 6.0 (TID 107) in 42 ms on localhost (99/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 131.0 in stage 6.0 (TID 138, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 123.0 in stage 6.0 (TID 130). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 131.0 in stage 6.0 (TID 138) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 132.0 in stage 6.0 (TID 139, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 96.0 in stage 6.0 (TID 103) in 49 ms on localhost (100/199) | |
| 15/07/22 01:29:19 INFO Executor: Running task 132.0 in stage 6.0 (TID 139) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 101.0 in stage 6.0 (TID 108) in 42 ms on localhost (101/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 119.0 in stage 6.0 (TID 126). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 133.0 in stage 6.0 (TID 140, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 121.0 in stage 6.0 (TID 128). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 133.0 in stage 6.0 (TID 140) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 126.0 in stage 6.0 (TID 133). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 134.0 in stage 6.0 (TID 141, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 134.0 in stage 6.0 (TID 141) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 122.0 in stage 6.0 (TID 129). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 135.0 in stage 6.0 (TID 142, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 127.0 in stage 6.0 (TID 134). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 135.0 in stage 6.0 (TID 142) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 99.0 in stage 6.0 (TID 106) in 49 ms on localhost (102/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 124.0 in stage 6.0 (TID 131). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 102.0 in stage 6.0 (TID 109) in 45 ms on localhost (103/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 103.0 in stage 6.0 (TID 110) in 44 ms on localhost (104/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 136.0 in stage 6.0 (TID 143, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 128.0 in stage 6.0 (TID 135). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 136.0 in stage 6.0 (TID 143) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 125.0 in stage 6.0 (TID 132). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 104.0 in stage 6.0 (TID 111) in 44 ms on localhost (105/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 137.0 in stage 6.0 (TID 144, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 137.0 in stage 6.0 (TID 144) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 106.0 in stage 6.0 (TID 113) in 42 ms on localhost (106/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 129.0 in stage 6.0 (TID 136). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 138.0 in stage 6.0 (TID 145, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 138.0 in stage 6.0 (TID 145) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 108.0 in stage 6.0 (TID 115) in 41 ms on localhost (107/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 130.0 in stage 6.0 (TID 137). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 139.0 in stage 6.0 (TID 146, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 139.0 in stage 6.0 (TID 146) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 109.0 in stage 6.0 (TID 116) in 41 ms on localhost (108/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 140.0 in stage 6.0 (TID 147, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 132.0 in stage 6.0 (TID 139). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 140.0 in stage 6.0 (TID 147) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 131.0 in stage 6.0 (TID 138). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 105.0 in stage 6.0 (TID 112) in 48 ms on localhost (109/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 141.0 in stage 6.0 (TID 148, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 141.0 in stage 6.0 (TID 148) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 110.0 in stage 6.0 (TID 117) in 42 ms on localhost (110/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 134.0 in stage 6.0 (TID 141). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 142.0 in stage 6.0 (TID 149, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 142.0 in stage 6.0 (TID 149) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 143.0 in stage 6.0 (TID 150, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 143.0 in stage 6.0 (TID 150) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 135.0 in stage 6.0 (TID 142). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 111.0 in stage 6.0 (TID 118) in 43 ms on localhost (111/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 144.0 in stage 6.0 (TID 151, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 144.0 in stage 6.0 (TID 151) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 133.0 in stage 6.0 (TID 140). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 112.0 in stage 6.0 (TID 119) in 44 ms on localhost (112/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 107.0 in stage 6.0 (TID 114) in 51 ms on localhost (113/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 136.0 in stage 6.0 (TID 143). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 145.0 in stage 6.0 (TID 152, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 145.0 in stage 6.0 (TID 152) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 114.0 in stage 6.0 (TID 121) in 44 ms on localhost (114/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 146.0 in stage 6.0 (TID 153, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 138.0 in stage 6.0 (TID 145). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 146.0 in stage 6.0 (TID 153) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 147.0 in stage 6.0 (TID 154, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 147.0 in stage 6.0 (TID 154) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 113.0 in stage 6.0 (TID 120) in 47 ms on localhost (115/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 115.0 in stage 6.0 (TID 122) in 44 ms on localhost (116/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 148.0 in stage 6.0 (TID 155, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 148.0 in stage 6.0 (TID 155) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 116.0 in stage 6.0 (TID 123) in 44 ms on localhost (117/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 137.0 in stage 6.0 (TID 144). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 139.0 in stage 6.0 (TID 146). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 140.0 in stage 6.0 (TID 147). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 149.0 in stage 6.0 (TID 156, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 149.0 in stage 6.0 (TID 156) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 118.0 in stage 6.0 (TID 125) in 42 ms on localhost (118/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 150.0 in stage 6.0 (TID 157, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 150.0 in stage 6.0 (TID 157) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 151.0 in stage 6.0 (TID 158, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 151.0 in stage 6.0 (TID 158) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 142.0 in stage 6.0 (TID 149). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 120.0 in stage 6.0 (TID 127) in 42 ms on localhost (119/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 141.0 in stage 6.0 (TID 148). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 117.0 in stage 6.0 (TID 124) in 46 ms on localhost (120/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 143.0 in stage 6.0 (TID 150). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 152.0 in stage 6.0 (TID 159, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 152.0 in stage 6.0 (TID 159) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 123.0 in stage 6.0 (TID 130) in 41 ms on localhost (121/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 153.0 in stage 6.0 (TID 160, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 153.0 in stage 6.0 (TID 160) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 154.0 in stage 6.0 (TID 161, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 144.0 in stage 6.0 (TID 151). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 154.0 in stage 6.0 (TID 161) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 119.0 in stage 6.0 (TID 126) in 47 ms on localhost (122/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 155.0 in stage 6.0 (TID 162, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 145.0 in stage 6.0 (TID 152). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 155.0 in stage 6.0 (TID 162) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 121.0 in stage 6.0 (TID 128) in 46 ms on localhost (123/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 156.0 in stage 6.0 (TID 163, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 156.0 in stage 6.0 (TID 163) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 157.0 in stage 6.0 (TID 164, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 157.0 in stage 6.0 (TID 164) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 126.0 in stage 6.0 (TID 133) in 43 ms on localhost (124/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 147.0 in stage 6.0 (TID 154). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 122.0 in stage 6.0 (TID 129) in 47 ms on localhost (125/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 127.0 in stage 6.0 (TID 134) in 42 ms on localhost (126/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 146.0 in stage 6.0 (TID 153). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 158.0 in stage 6.0 (TID 165, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 158.0 in stage 6.0 (TID 165) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 148.0 in stage 6.0 (TID 155). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 159.0 in stage 6.0 (TID 166, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 159.0 in stage 6.0 (TID 166) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 124.0 in stage 6.0 (TID 131) in 48 ms on localhost (127/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 149.0 in stage 6.0 (TID 156). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 160.0 in stage 6.0 (TID 167, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 160.0 in stage 6.0 (TID 167) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 161.0 in stage 6.0 (TID 168, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 161.0 in stage 6.0 (TID 168) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 125.0 in stage 6.0 (TID 132) in 49 ms on localhost (128/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 151.0 in stage 6.0 (TID 158). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 128.0 in stage 6.0 (TID 135) in 45 ms on localhost (129/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 129.0 in stage 6.0 (TID 136) in 44 ms on localhost (130/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 150.0 in stage 6.0 (TID 157). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 162.0 in stage 6.0 (TID 169, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 162.0 in stage 6.0 (TID 169) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 163.0 in stage 6.0 (TID 170, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 163.0 in stage 6.0 (TID 170) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 130.0 in stage 6.0 (TID 137) in 44 ms on localhost (131/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 164.0 in stage 6.0 (TID 171, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 164.0 in stage 6.0 (TID 171) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 132.0 in stage 6.0 (TID 139) in 43 ms on localhost (132/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 154.0 in stage 6.0 (TID 161). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 153.0 in stage 6.0 (TID 160). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 165.0 in stage 6.0 (TID 172, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 165.0 in stage 6.0 (TID 172) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 131.0 in stage 6.0 (TID 138) in 45 ms on localhost (133/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 166.0 in stage 6.0 (TID 173, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 155.0 in stage 6.0 (TID 162). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 166.0 in stage 6.0 (TID 173) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 134.0 in stage 6.0 (TID 141) in 42 ms on localhost (134/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 167.0 in stage 6.0 (TID 174, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 167.0 in stage 6.0 (TID 174) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 152.0 in stage 6.0 (TID 159). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 135.0 in stage 6.0 (TID 142) in 43 ms on localhost (135/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 168.0 in stage 6.0 (TID 175, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 168.0 in stage 6.0 (TID 175) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 169.0 in stage 6.0 (TID 176, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 169.0 in stage 6.0 (TID 176) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 133.0 in stage 6.0 (TID 140) in 47 ms on localhost (136/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 159.0 in stage 6.0 (TID 166). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 170.0 in stage 6.0 (TID 177, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 170.0 in stage 6.0 (TID 177) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 136.0 in stage 6.0 (TID 143) in 44 ms on localhost (137/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 160.0 in stage 6.0 (TID 167). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 171.0 in stage 6.0 (TID 178, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 171.0 in stage 6.0 (TID 178) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 158.0 in stage 6.0 (TID 165). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 138.0 in stage 6.0 (TID 145) in 43 ms on localhost (138/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 161.0 in stage 6.0 (TID 168). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 156.0 in stage 6.0 (TID 163). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 172.0 in stage 6.0 (TID 179, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 172.0 in stage 6.0 (TID 179) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 137.0 in stage 6.0 (TID 144) in 45 ms on localhost (139/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 140.0 in stage 6.0 (TID 147) in 41 ms on localhost (140/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 139.0 in stage 6.0 (TID 146) in 43 ms on localhost (141/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 157.0 in stage 6.0 (TID 164). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 162.0 in stage 6.0 (TID 169). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 173.0 in stage 6.0 (TID 180, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 173.0 in stage 6.0 (TID 180) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 174.0 in stage 6.0 (TID 181, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 174.0 in stage 6.0 (TID 181) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 142.0 in stage 6.0 (TID 149) in 41 ms on localhost (142/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 175.0 in stage 6.0 (TID 182, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 175.0 in stage 6.0 (TID 182) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 164.0 in stage 6.0 (TID 171). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 176.0 in stage 6.0 (TID 183, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 176.0 in stage 6.0 (TID 183) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 141.0 in stage 6.0 (TID 148) in 45 ms on localhost (143/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 143.0 in stage 6.0 (TID 150) in 43 ms on localhost (144/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 177.0 in stage 6.0 (TID 184, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 177.0 in stage 6.0 (TID 184) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 144.0 in stage 6.0 (TID 151) in 42 ms on localhost (145/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 166.0 in stage 6.0 (TID 173). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 165.0 in stage 6.0 (TID 172). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 145.0 in stage 6.0 (TID 152) in 40 ms on localhost (146/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 163.0 in stage 6.0 (TID 170). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 167.0 in stage 6.0 (TID 174). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 178.0 in stage 6.0 (TID 185, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 168.0 in stage 6.0 (TID 175). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 178.0 in stage 6.0 (TID 185) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 179.0 in stage 6.0 (TID 186, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 179.0 in stage 6.0 (TID 186) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 147.0 in stage 6.0 (TID 154) in 40 ms on localhost (147/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 180.0 in stage 6.0 (TID 187, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 180.0 in stage 6.0 (TID 187) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 146.0 in stage 6.0 (TID 153) in 42 ms on localhost (148/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 170.0 in stage 6.0 (TID 177). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 169.0 in stage 6.0 (TID 176). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 171.0 in stage 6.0 (TID 178). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 181.0 in stage 6.0 (TID 188, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 181.0 in stage 6.0 (TID 188) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 148.0 in stage 6.0 (TID 155) in 41 ms on localhost (149/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 149.0 in stage 6.0 (TID 156) in 40 ms on localhost (150/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 182.0 in stage 6.0 (TID 189, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 182.0 in stage 6.0 (TID 189) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 183.0 in stage 6.0 (TID 190, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 183.0 in stage 6.0 (TID 190) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 151.0 in stage 6.0 (TID 158) in 40 ms on localhost (151/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 184.0 in stage 6.0 (TID 191, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 184.0 in stage 6.0 (TID 191) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 185.0 in stage 6.0 (TID 192, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 185.0 in stage 6.0 (TID 192) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Finished task 173.0 in stage 6.0 (TID 180). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 175.0 in stage 6.0 (TID 182). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 186.0 in stage 6.0 (TID 193, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 186.0 in stage 6.0 (TID 193) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 155.0 in stage 6.0 (TID 162) in 40 ms on localhost (152/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 150.0 in stage 6.0 (TID 157) in 45 ms on localhost (153/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 172.0 in stage 6.0 (TID 179). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 154.0 in stage 6.0 (TID 161) in 42 ms on localhost (154/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 176.0 in stage 6.0 (TID 183). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 187.0 in stage 6.0 (TID 194, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 177.0 in stage 6.0 (TID 184). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 153.0 in stage 6.0 (TID 160) in 45 ms on localhost (155/199) | |
| 15/07/22 01:29:19 INFO Executor: Running task 187.0 in stage 6.0 (TID 194) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 152.0 in stage 6.0 (TID 159) in 46 ms on localhost (156/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 179.0 in stage 6.0 (TID 186). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 174.0 in stage 6.0 (TID 181). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 188.0 in stage 6.0 (TID 195, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO Executor: Running task 188.0 in stage 6.0 (TID 195) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 178.0 in stage 6.0 (TID 185). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 180.0 in stage 6.0 (TID 187). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 189.0 in stage 6.0 (TID 196, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 189.0 in stage 6.0 (TID 196) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 160.0 in stage 6.0 (TID 167) in 40 ms on localhost (157/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 159.0 in stage 6.0 (TID 166) in 41 ms on localhost (158/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 190.0 in stage 6.0 (TID 197, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 190.0 in stage 6.0 (TID 197) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 181.0 in stage 6.0 (TID 188). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 191.0 in stage 6.0 (TID 198, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 191.0 in stage 6.0 (TID 198) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 158.0 in stage 6.0 (TID 165) in 45 ms on localhost (159/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 192.0 in stage 6.0 (TID 199, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 192.0 in stage 6.0 (TID 199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 161.0 in stage 6.0 (TID 168) in 44 ms on localhost (160/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 193.0 in stage 6.0 (TID 200, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 183.0 in stage 6.0 (TID 190). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 193.0 in stage 6.0 (TID 200) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 156.0 in stage 6.0 (TID 163) in 51 ms on localhost (161/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 182.0 in stage 6.0 (TID 189). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 194.0 in stage 6.0 (TID 201, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 194.0 in stage 6.0 (TID 201) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 157.0 in stage 6.0 (TID 164) in 52 ms on localhost (162/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 162.0 in stage 6.0 (TID 169) in 45 ms on localhost (163/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 187.0 in stage 6.0 (TID 194). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 195.0 in stage 6.0 (TID 202, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 185.0 in stage 6.0 (TID 192). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Running task 195.0 in stage 6.0 (TID 202) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 184.0 in stage 6.0 (TID 191). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 164.0 in stage 6.0 (TID 171) in 45 ms on localhost (164/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 188.0 in stage 6.0 (TID 195). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 196.0 in stage 6.0 (TID 203, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 196.0 in stage 6.0 (TID 203) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 186.0 in stage 6.0 (TID 193). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 197.0 in stage 6.0 (TID 204, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Running task 197.0 in stage 6.0 (TID 204) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 166.0 in stage 6.0 (TID 173) in 44 ms on localhost (165/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 189.0 in stage 6.0 (TID 196). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Starting task 198.0 in stage 6.0 (TID 205, localhost, PROCESS_LOCAL, 1322 bytes) | |
| 15/07/22 01:29:19 INFO Executor: Running task 198.0 in stage 6.0 (TID 205) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 165.0 in stage 6.0 (TID 172) in 48 ms on localhost (166/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 163.0 in stage 6.0 (TID 170) in 50 ms on localhost (167/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 167.0 in stage 6.0 (TID 174) in 46 ms on localhost (168/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 168.0 in stage 6.0 (TID 175) in 45 ms on localhost (169/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 170.0 in stage 6.0 (TID 177) in 44 ms on localhost (170/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 171.0 in stage 6.0 (TID 178) in 43 ms on localhost (171/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 169.0 in stage 6.0 (TID 176) in 46 ms on localhost (172/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 173.0 in stage 6.0 (TID 180) in 41 ms on localhost (173/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 191.0 in stage 6.0 (TID 198). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 172.0 in stage 6.0 (TID 179) in 43 ms on localhost (174/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 175.0 in stage 6.0 (TID 182) in 39 ms on localhost (175/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 176.0 in stage 6.0 (TID 183) in 39 ms on localhost (176/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 192.0 in stage 6.0 (TID 199). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 177.0 in stage 6.0 (TID 184) in 37 ms on localhost (177/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 179.0 in stage 6.0 (TID 186) in 36 ms on localhost (178/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 194.0 in stage 6.0 (TID 201). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO Executor: Finished task 190.0 in stage 6.0 (TID 197). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms | |
| 15/07/22 01:29:19 INFO Executor: Finished task 193.0 in stage 6.0 (TID 200). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 174.0 in stage 6.0 (TID 181) in 42 ms on localhost (179/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 178.0 in stage 6.0 (TID 185) in 37 ms on localhost (180/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 195.0 in stage 6.0 (TID 202). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 181.0 in stage 6.0 (TID 188) in 34 ms on localhost (181/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 180.0 in stage 6.0 (TID 187) in 37 ms on localhost (182/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 196.0 in stage 6.0 (TID 203). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 182.0 in stage 6.0 (TID 189) in 34 ms on localhost (183/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 185.0 in stage 6.0 (TID 192) in 32 ms on localhost (184/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 183.0 in stage 6.0 (TID 190) in 34 ms on localhost (185/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 198.0 in stage 6.0 (TID 205). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 187.0 in stage 6.0 (TID 194) in 29 ms on localhost (186/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 184.0 in stage 6.0 (TID 191) in 33 ms on localhost (187/199) | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks out of 2 blocks | |
| 15/07/22 01:29:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 188.0 in stage 6.0 (TID 195) in 27 ms on localhost (188/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 186.0 in stage 6.0 (TID 193) in 32 ms on localhost (189/199) | |
| 15/07/22 01:29:19 INFO Executor: Finished task 197.0 in stage 6.0 (TID 204). 894 bytes result sent to driver | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 189.0 in stage 6.0 (TID 196) in 27 ms on localhost (190/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 191.0 in stage 6.0 (TID 198) in 24 ms on localhost (191/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 192.0 in stage 6.0 (TID 199) in 24 ms on localhost (192/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 190.0 in stage 6.0 (TID 197) in 26 ms on localhost (193/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 193.0 in stage 6.0 (TID 200) in 23 ms on localhost (194/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 195.0 in stage 6.0 (TID 202) in 19 ms on localhost (195/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 194.0 in stage 6.0 (TID 201) in 22 ms on localhost (196/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 196.0 in stage 6.0 (TID 203) in 18 ms on localhost (197/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 198.0 in stage 6.0 (TID 205) in 16 ms on localhost (198/199) | |
| 15/07/22 01:29:19 INFO TaskSetManager: Finished task 197.0 in stage 6.0 (TID 204) in 17 ms on localhost (199/199) | |
| 15/07/22 01:29:19 INFO DAGScheduler: ResultStage 6 (show at <console>:31) finished in 0.284 s | |
| 15/07/22 01:29:19 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool | |
| 15/07/22 01:29:19 INFO DAGScheduler: Job 4 finished: show at <console>:31, took 0.309742 s | |
| +---+ | |
| |_c0| | |
| +---+ | |
| | 5| | |
| +---+ | |
| scala> | |
| scala> new File(path).delete() | |
| res6: Boolean = true | |
| scala> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment