Skip to content

Instantly share code, notes, and snippets.

@orwa-te
Created June 24, 2020 10:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save orwa-te/90df71ddba52d5bb94ac9d1453f108a1 to your computer and use it in GitHub Desktop.
Save orwa-te/90df71ddba52d5bb94ac9d1453f108a1 to your computer and use it in GitHub Desktop.
Error logs when executing mnist_tf.py on Jupyter notebook
Spark Executor Command: "/usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java" "-cp" "/home/orwa/spark/conf/:/home/orwa/spark/jars/*" "-Xmx1024M" "-Dspark.driver.port=36393" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@192.168.198.131:36393" "--executor-id" "0" "--hostname" "192.168.198.131" "--cores" "2" "--app-id" "app-20200624132821-0010" "--worker-url" "spark://Worker@192.168.198.131:36489"
========================================
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
20/06/24 13:28:22 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 6648@orwa-virtual-machine
20/06/24 13:28:22 INFO SignalUtils: Registered signal handler for TERM
20/06/24 13:28:22 INFO SignalUtils: Registered signal handler for HUP
20/06/24 13:28:22 INFO SignalUtils: Registered signal handler for INT
20/06/24 13:28:22 WARN Utils: Your hostname, orwa-virtual-machine resolves to a loopback address: 127.0.1.1; using 192.168.198.131 instead (on interface ens33)
20/06/24 13:28:22 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
20/06/24 13:28:22 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20/06/24 13:28:22 INFO SecurityManager: Changing view acls to: orwa
20/06/24 13:28:22 INFO SecurityManager: Changing modify acls to: orwa
20/06/24 13:28:22 INFO SecurityManager: Changing view acls groups to:
20/06/24 13:28:22 INFO SecurityManager: Changing modify acls groups to:
20/06/24 13:28:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(orwa); groups with view permissions: Set(); users with modify permissions: Set(orwa); groups with modify permissions: Set()
20/06/24 13:28:23 INFO TransportClientFactory: Successfully created connection to /192.168.198.131:36393 after 51 ms (0 ms spent in bootstraps)
20/06/24 13:28:23 INFO SecurityManager: Changing view acls to: orwa
20/06/24 13:28:23 INFO SecurityManager: Changing modify acls to: orwa
20/06/24 13:28:23 INFO SecurityManager: Changing view acls groups to:
20/06/24 13:28:23 INFO SecurityManager: Changing modify acls groups to:
20/06/24 13:28:23 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(orwa); groups with view permissions: Set(); users with modify permissions: Set(orwa); groups with modify permissions: Set()
20/06/24 13:28:23 INFO TransportClientFactory: Successfully created connection to /192.168.198.131:36393 after 2 ms (0 ms spent in bootstraps)
20/06/24 13:28:23 INFO DiskBlockManager: Created local directory at /tmp/spark-e25807a2-6029-4394-bf20-18b8e3b9aaf9/executor-ba7b29dc-7504-4314-916b-6680d4767086/blockmgr-51513551-ed4f-4998-b0c7-20556d7b5770
20/06/24 13:28:23 INFO MemoryStore: MemoryStore started with capacity 366.3 MB
20/06/24 13:28:23 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@192.168.198.131:36393
20/06/24 13:28:23 INFO WorkerWatcher: Connecting to worker spark://Worker@192.168.198.131:36489
20/06/24 13:28:23 INFO TransportClientFactory: Successfully created connection to /192.168.198.131:36489 after 4 ms (0 ms spent in bootstraps)
20/06/24 13:28:23 INFO WorkerWatcher: Successfully connected to spark://Worker@192.168.198.131:36489
20/06/24 13:28:23 INFO CoarseGrainedExecutorBackend: Successfully registered with driver
20/06/24 13:28:23 INFO Executor: Starting executor ID 0 on host 192.168.198.131
20/06/24 13:28:23 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 42355.
20/06/24 13:28:23 INFO NettyBlockTransferService: Server created on 192.168.198.131:42355
20/06/24 13:28:23 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
20/06/24 13:28:23 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(0, 192.168.198.131, 42355, None)
20/06/24 13:28:23 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(0, 192.168.198.131, 42355, None)
20/06/24 13:28:23 INFO BlockManager: Initialized BlockManager: BlockManagerId(0, 192.168.198.131, 42355, None)
20/06/24 13:28:28 INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown
20/06/24 13:28:28 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
tdown
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment