Created
January 23, 2015 17:17
-
-
Save skrasser/e3e2ee8dede5ef6b082c to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2015-01-23 15:36:53,654 INFO [Reporter] yarn.YarnAllocationHandler (Logging.scala:logInfo(59)) - Container marked as failed: container_1421692415636_0052_01_000030. Exit status: 143. Diagnostics: Container [pid=35211,containerID=container_1421692415636_0052_01_000030] is running beyond physical memory limits. Current usage: 14.9 GB of 14.5 GB physical memory used; 41.3 GB of 72.5 GB virtual memory used. Killing container. | |
| Dump of the process-tree for container_1421692415636_0052_01_000030 : | |
| |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE | |
| |- 54101 36625 36625 35211 (python) 78 1 332730368 16834 python -m pyspark.daemon | |
| |- 52140 36625 36625 35211 (python) 58 1 332730368 16837 python -m pyspark.daemon | |
| |- 36625 35228 36625 35211 (python) 65 604 331685888 17694 python -m pyspark.daemon | |
| |- 54612 36625 36625 35211 (python) 83 1 332730368 16868 python -m pyspark.daemon | |
| |- 54104 36625 36625 35211 (python) 77 1 333344768 16988 python -m pyspark.daemon | |
| |- 54107 36625 36625 35211 (python) 78 2 332730368 16834 python -m pyspark.daemon | |
| |- 51006 36625 36625 35211 (python) 90 1 333139968 16988 python -m pyspark.daemon | |
| |- 54493 36625 36625 35211 (python) 68 2 332730368 16837 python -m pyspark.daemon | |
| |- 54415 36625 36625 35211 (python) 74 2 332730368 16834 python -m pyspark.daemon | |
| |- 53477 36625 36625 35211 (python) 138 2 333500416 17051 python -m pyspark.daemon | |
| |- 51952 36625 36625 35211 (python) 68 2 332533760 16837 python -m pyspark.daemon | |
| |- 50990 36625 36625 35211 (python) 1769 114 527429632 64425 python -m pyspark.daemon | |
| |- 52561 36625 36625 35211 (python) 78 2 332611584 16837 python -m pyspark.daemon | |
| |- 54496 36625 36625 35211 (python) 66 2 332730368 16834 python -m pyspark.daemon | |
| |- 51776 36625 36625 35211 (python) 63 1 332611584 16892 python -m pyspark.daemon | |
| |- 51773 36625 36625 35211 (python) 68 0 332730368 16834 python -m pyspark.daemon | |
| |- 53804 36625 36625 35211 (python) 166 2 333393920 17053 python -m pyspark.daemon | |
| |- 51017 36625 36625 35211 (python) 99 2 333307904 17050 python -m pyspark.daemon | |
| |- 35228 35211 35211 35211 (java) 299915 77999 11481403392 2150129 /usr/java/latest/bin/java -server -XX:OnOutOfMemoryError=kill %p -Xms8550m -Xmx8550m -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=/mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1421692415636_0052/container_1421692415636_0052_01_000030/tmp -Dspark.yarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1421692415636_0052/container_1421692415636_0052_01_000030 org.apache.spark.executor.CoarseGrainedExecutorBackend akka.tcp://sparkDriver@ip-10-20-80-7.us-west-1.compute.internal:39747/user/CoarseGrainedScheduler 29 ip-10-20-80-12.us-west-1.compute.internal 2 application_1421692415636_0052 | |
| |- 53175 36625 36625 35211 (python) 142 2 333459456 17052 python -m pyspark.daemon | |
| |- 54621 36625 36625 35211 (python) 76 2 332730368 16836 python -m pyspark.daemon | |
| |- 52416 36625 36625 35211 (python) 82 2 332730368 16837 python -m pyspark.daemon | |
| |- 54178 36625 36625 35211 (python) 86 2 332730368 16837 python -m pyspark.daemon | |
| |- 53661 36625 36625 35211 (python) 176 3 333438976 17055 python -m pyspark.daemon | |
| |- 52231 36625 36625 35211 (python) 61 2 332730368 16835 python -m pyspark.daemon | |
| |- 53270 36625 36625 35211 (python) 136 4 333328384 17032 python -m pyspark.daemon | |
| |- 54826 36625 36625 35211 (python) 76 3 332730368 16835 python -m pyspark.daemon | |
| |- 54695 36625 36625 35211 (python) 428 24 451633152 45971 python -m pyspark.daemon | |
| |- 52836 36625 36625 35211 (python) 151 1 333402112 17060 python -m pyspark.daemon | |
| |- 54535 36625 36625 35211 (python) 74 3 332611584 16829 python -m pyspark.daemon | |
| |- 51599 36625 36625 35211 (python) 76 0 332730368 16837 python -m pyspark.daemon | |
| |- 35211 6662 35211 35211 (bash) 0 0 115630080 302 /bin/bash -c /usr/java/latest/bin/java -server -XX:OnOutOfMemoryError='kill %p' -Xms8550m -Xmx8550m -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=/mnt/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1421692415636_0052/container_1421692415636_0052_01_000030/tmp -Dspark.yarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1421692415636_0052/container_1421692415636_0052_01_000030 org.apache.spark.executor.CoarseGrainedExecutorBackend akka.tcp://sparkDriver@ip-10-20-80-7.us-west-1.compute.internal:39747/user/CoarseGrainedScheduler 29 ip-10-20-80-12.us-west-1.compute.internal 2 application_1421692415636_0052 1> /mnt/var/log/hadoop/userlogs/application_1421692415636_0052/container_1421692415636_0052_01_000030/stdout 2> /mnt/var/log/hadoop/userlogs/application_1421692415636_0052/container_1421692415636_0052_01_000030/stderr | |
| |- 52932 36625 36625 35211 (python) 155 2 333373440 17054 python -m pyspark.daemon | |
| |- 54279 36625 36625 35211 (python) 76 2 332730368 16887 python -m pyspark.daemon | |
| |- 51302 36625 36625 35211 (python) 78 1 332730368 16870 python -m pyspark.daemon | |
| |- 53409 36625 36625 35211 (python) 135 2 333406208 17045 python -m pyspark.daemon | |
| |- 54330 36625 36625 35211 (python) 72 2 332730368 16836 python -m pyspark.daemon | |
| |- 54270 36625 36625 35211 (python) 82 0 332730368 16837 python -m pyspark.daemon | |
| |- 54684 36625 36625 35211 (python) 66 2 332730368 16835 python -m pyspark.daemon | |
| |- 52091 36625 36625 35211 (python) 16 0 331685888 16678 python -m pyspark.daemon | |
| |- 53096 36625 36625 35211 (python) 138 3 333254656 17025 python -m pyspark.daemon | |
| |- 53348 36625 36625 35211 (python) 140 3 333385728 17056 python -m pyspark.daemon | |
| |- 51682 36625 36625 35211 (python) 54 1 332730368 16834 python -m pyspark.daemon | |
| |- 54409 36625 36625 35211 (python) 74 1 332730368 16832 python -m pyspark.daemon | |
| |- 54342 36625 36625 35211 (python) 72 2 332730368 16836 python -m pyspark.daemon | |
| |- 53549 36625 36625 35211 (python) 159 1 333434880 17043 python -m pyspark.daemon | |
| |- 54155 36625 36625 35211 (python) 80 1 332730368 16889 python -m pyspark.daemon | |
| |- 54200 36625 36625 35211 (python) 73 2 332730368 16835 python -m pyspark.daemon | |
| |- 51417 36625 36625 35211 (python) 62 0 332730368 16837 python -m pyspark.daemon | |
| |- 52218 36625 36625 35211 (python) 68 2 332730368 16834 python -m pyspark.daemon | |
| |- 51505 36625 36625 35211 (python) 46 0 332730368 16837 python -m pyspark.daemon | |
| |- 54359 36625 36625 35211 (python) 75 4 332611584 16889 python -m pyspark.daemon | |
| |- 54553 36625 36625 35211 (python) 74 3 332611584 16829 python -m pyspark.daemon | |
| |- 52912 36625 36625 35211 (python) 142 1 333418496 17050 python -m pyspark.daemon | |
| |- 52109 36625 36625 35211 (python) 59 5 332730368 16834 python -m pyspark.daemon | |
| |- 52662 36625 36625 35211 (python) 158 2 333430784 17042 python -m pyspark.daemon | |
| |- 54457 36625 36625 35211 (python) 73 0 332730368 16837 python -m pyspark.daemon | |
| |- 53330 36625 36625 35211 (python) 143 1 333312000 17037 python -m pyspark.daemon | |
| |- 54098 36625 36625 35211 (python) 98 2 333283328 17059 python -m pyspark.daemon | |
| |- 51045 36625 36625 35211 (python) 79 0 332730368 16837 python -m pyspark.daemon | |
| |- 51870 36625 36625 35211 (python) 64 1 332533760 16832 python -m pyspark.daemon | |
| |- 54666 36625 36625 35211 (python) 74 2 332611584 16834 python -m pyspark.daemon | |
| |- 52995 36625 36625 35211 (python) 132 2 333336576 17030 python -m pyspark.daemon | |
| |- 53020 36625 36625 35211 (python) 132 4 333246464 17029 python -m pyspark.daemon | |
| |- 53712 36625 36625 35211 (python) 166 5 333492224 17054 python -m pyspark.daemon | |
| |- 53128 36625 36625 35211 (python) 136 3 333393920 17054 python -m pyspark.daemon | |
| |- 52659 36625 36625 35211 (python) 1180 116 333492224 17103 python -m pyspark.daemon | |
| |- 53638 36625 36625 35211 (python) 147 2 333402112 17025 python -m pyspark.daemon | |
| |- 52706 36625 36625 35211 (python) 161 3 333410304 17096 python -m pyspark.daemon | |
| |- 51525 36625 36625 35211 (python) 72 2 332730368 16833 python -m pyspark.daemon | |
| |- 52300 36625 36625 35211 (python) 70 0 332730368 16834 python -m pyspark.daemon | |
| |- 51864 36625 36625 35211 (python) 72 1 332611584 16832 python -m pyspark.daemon | |
| |- 51983 36625 36625 35211 (python) 68 0 332730368 16837 python -m pyspark.daemon | |
| |- 54567 36625 36625 35211 (python) 75 1 332730368 16834 python -m pyspark.daemon | |
| |- 53425 36625 36625 35211 (python) 142 2 333410304 17061 python -m pyspark.daemon | |
| |- 53795 36625 36625 35211 (python) 166 2 333479936 17054 python -m pyspark.daemon | |
| |- 51669 36625 36625 35211 (python) 52 1 332730368 16835 python -m pyspark.daemon | |
| |- 54758 36625 36625 35211 (python) 69 0 332730368 16834 python -m pyspark.daemon | |
| |- 52375 36625 36625 35211 (python) 55 2 332644352 16837 python -m pyspark.daemon | |
| |- 54131 36625 36625 35211 (python) 86 0 332730368 16837 python -m pyspark.daemon | |
| |- 53842 36625 36625 35211 (python) 160 2 333484032 17064 python -m pyspark.daemon | |
| |- 54261 36625 36625 35211 (python) 34 1 332533760 16773 python -m pyspark.daemon | |
| |- 50977 36625 36625 35211 (python) 1273 120 523436032 63454 python -m pyspark.daemon | |
| |- 53506 36625 36625 35211 (python) 147 4 333369344 17050 python -m pyspark.daemon | |
| |- 52479 36625 36625 35211 (python) 87 2 332730368 16837 python -m pyspark.daemon | |
| |- 53734 36625 36625 35211 (python) 195 1 333328384 17059 python -m pyspark.daemon | |
| |- 51060 36625 36625 35211 (python) 63 2 332730368 16837 python -m pyspark.daemon | |
| |- 51449 36625 36625 35211 (python) 84 0 332730368 16869 python -m pyspark.daemon | |
| |- 52605 36625 36625 35211 (python) 80 0 332730368 16884 python -m pyspark.daemon | |
| |- 52703 36625 36625 35211 (python) 1148 50 333520896 17106 python -m pyspark.daemon | |
| |- 51346 36625 36625 35211 (python) 77 0 332730368 16834 python -m pyspark.daemon | |
| |- 53196 36625 36625 35211 (python) 147 0 333373440 17043 python -m pyspark.daemon | |
| |- 52326 36625 36625 35211 (python) 74 1 332730368 16837 python -m pyspark.daemon | |
| |- 54905 36625 36625 35211 (python) 15 0 332271616 16709 python -m pyspark.daemon | |
| |- 52801 36625 36625 35211 (python) 152 3 333488128 17048 python -m pyspark.daemon | |
| |- 52051 36625 36625 35211 (python) 63 0 332730368 16831 python -m pyspark.daemon | |
| |- 53585 36625 36625 35211 (python) 144 3 333393920 17035 python -m pyspark.daemon | |
| |- 53054 36625 36625 35211 (python) 150 2 333434880 17068 python -m pyspark.daemon | |
| |- 53253 36625 36625 35211 (python) 145 0 333459456 17076 python -m pyspark.daemon | |
| Container killed on request. Exit code is 143 | |
| Container exited with a non-zero exit code 143 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment