Skip to content

Instantly share code, notes, and snippets.

@knub

knub/gist:8f2f953da95c8d7adefc Secret

Created April 17, 2015 13:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save knub/8f2f953da95c8d7adefc to your computer and use it in GitHub Desktop.
Save knub/8f2f953da95c8d7adefc to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
17.Apr. 14:18:58 WARN NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17.Apr. 14:18:58 INFO TaskManager - --------------------------------------------------------------------------------
17.Apr. 14:18:58 INFO TaskManager - Starting TaskManager (Version: 0.9.0-milestone-1, Rev:865049f, Date:08.04.2015 @ 16:48:46 CEST)
17.Apr. 14:18:58 INFO TaskManager - Current user: hadoop10
17.Apr. 14:18:58 INFO TaskManager - JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.7/24.65-b04
17.Apr. 14:18:58 INFO TaskManager - Maximum heap size: 5888 MiBytes
17.Apr. 14:18:58 INFO TaskManager - JAVA_HOME: /usr/local/java/default/
17.Apr. 14:18:58 INFO TaskManager - JVM Options:
17.Apr. 14:18:58 INFO TaskManager - -XX:MaxPermSize=256m
17.Apr. 14:18:58 INFO TaskManager - -Xms6144m
17.Apr. 14:18:58 INFO TaskManager - -Xmx6144m
17.Apr. 14:18:58 INFO TaskManager - -Dcom.sun.management.jmxremote.port=3333
17.Apr. 14:18:58 INFO TaskManager - -Dcom.sun.management.jmxremote.ssl=false
17.Apr. 14:18:58 INFO TaskManager - -Dcom.sun.management.jmxremote.authenticate=false
17.Apr. 14:18:58 INFO TaskManager - -Dlog.file=/opt/flink/flink-0.9.0-milestone-1/bin/../log/flink-hadoop10-taskmanager-tenem11.log
17.Apr. 14:18:58 INFO TaskManager - -Dlog4j.configuration=file:/opt/flink/flink-0.9.0-milestone-1/bin/../conf/log4j.properties
17.Apr. 14:18:58 INFO TaskManager - -Dlogback.configurationFile=file:/opt/flink/flink-0.9.0-milestone-1/bin/../conf/logback.xml
17.Apr. 14:18:58 INFO TaskManager - Program Arguments:
17.Apr. 14:18:58 INFO TaskManager - --configDir
17.Apr. 14:18:58 INFO TaskManager - /opt/flink/flink-0.9.0-milestone-1/bin/../conf
17.Apr. 14:18:58 INFO TaskManager - --------------------------------------------------------------------------------
17.Apr. 14:18:58 INFO TaskManager - Loading configuration from /opt/flink/flink-0.9.0-milestone-1/bin/../conf
17.Apr. 14:18:58 INFO TaskManager - Security is not enabled. Starting non-authenticated TaskManager.
17.Apr. 14:18:58 INFO TaskManager - Trying to select the network interface and address to use by connecting to the configured JobManager.
17.Apr. 14:18:58 INFO TaskManager - TaskManager will try to connect for 120000 seconds before falling back to heuristics
17.Apr. 14:19:01 INFO TaskManager - TaskManager will use hostname/address 'tenem11.hpi.uni-potsdam.de' (172.16.19.209) for communication.
17.Apr. 14:19:01 INFO TaskManager - Starting TaskManager
17.Apr. 14:19:01 INFO TaskManager - Starting TaskManager actor system at tenem11.hpi.uni-potsdam.de:0
17.Apr. 14:19:02 INFO Slf4jLogger - Slf4jLogger started
17.Apr. 14:19:02 INFO Remoting - Starting remoting
17.Apr. 14:19:02 INFO Remoting - Remoting started; listening on addresses :[akka.tcp://flink@172.16.19.209:49606]
17.Apr. 14:19:02 INFO TaskManager - Starting TaskManager actor
17.Apr. 14:19:02 INFO NettyConfig - NettyConfig [server address: tenem11.hpi.uni-potsdam.de/172.16.19.209, server port: 53946, memory segment size (bytes): 32768, transport type: NIO, number of server threads: 0 (use Netty's default), number of client threads: 0 (use Netty's default), server connect backlog: 0 (use Netty's default), client connect timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's default)]
17.Apr. 14:19:02 INFO TaskManager - Messages between TaskManager and JobManager have a max timeout of 100000 milliseconds
17.Apr. 14:19:02 INFO TaskManager - Temporary file directory '/tmp/opt/flink': total 908 GB, usable 599 GB (65.97% usable)
17.Apr. 14:19:02 INFO NetworkBufferPool - Allocated 64 MB for network buffer pool (number of memory segments: 2048, bytes per segment: 32768).
17.Apr. 14:19:03 INFO TaskManager - Using 0.5 of the currently free heap space for Flink managed memory (2905 MB).
17.Apr. 14:19:06 INFO IOManager - I/O manager uses directory /tmp/opt/flink/flink-io-89025cd6-1a1f-48a7-9dd0-03094c5e89cd for spill files.
17.Apr. 14:19:11 INFO TaskManager - Starting TaskManager actor at akka://flink/user/taskmanager#1527269226.
17.Apr. 14:19:11 INFO TaskManager - TaskManager data connection information: tenem11.hpi.uni-potsdam.de (dataPort=53946)
17.Apr. 14:19:11 INFO TaskManager - TaskManager has 2 task slot(s).
17.Apr. 14:19:11 INFO TaskManager - Memory usage stats: [HEAP: 3058/5888/5888 MB, NON HEAP: 25/53/304 MB (used/committed/max)]
17.Apr. 14:19:11 INFO TaskManager - Trying to register at JobManager akka.tcp://flink@172.16.21.111:6123/user/jobmanager (attempt 1, timeout: 500 milliseconds)
17.Apr. 14:19:11 INFO TaskManager - Successful registration at JobManager (akka.tcp://flink@172.16.21.111:6123/user/jobmanager), starting network stack and library cache.
17.Apr. 14:19:11 INFO NettyClient - Successful initialization (took 64 ms).
17.Apr. 14:19:11 INFO NettyServer - Successful initialization (took 42 ms). Listening on SocketAddress /172.16.19.209:53946.
17.Apr. 14:19:11 INFO TaskManager - Determined BLOB server address to be /172.16.21.111:55178. Starting BLOB cache.
17.Apr. 14:19:11 INFO BlobCache - Created BLOB cache storage directory /tmp/blobStore-89fcdf77-e3a3-4ca8-b22c-5bb625cb78ec
17.Apr. 14:27:14 INFO BlobCache - Downloading cc73de0e92e6a81a383fa0db1a305261f0f96aa3 from /172.16.21.111:55178
17.Apr. 14:27:23 INFO TaskManager - Register task CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14)) -> Filter (Redirects) -> Map (Key Extractor 1) (1/10) [DEPLOYING]
17.Apr. 14:27:23 INFO Task - Starting execution of task CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14)) -> Filter (Redirects) -> Map (Key Extractor 1)
17.Apr. 14:27:23 INFO TaskManager - Register task CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19)) -> Filter (Context-Links) (1/10) [DEPLOYING]
17.Apr. 14:27:23 INFO Task - Starting execution of task CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19)) -> Filter (Context-Links)
17.Apr. 14:27:23 INFO TaskManager - Register task CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Resolved-Redirects-From-Iteration) (1/10) [DEPLOYING]
17.Apr. 14:27:23 INFO Task - Starting execution of task CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Resolved-Redirects-From-Iteration)
17.Apr. 14:28:41 INFO Task - CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19)) -> Filter (Context-Links) (1/10) switched to FINISHED
17.Apr. 14:28:41 INFO TaskManager - Unregister task with execution ID b1af12e91fa26a08b2a15ba923a4101d.
17.Apr. 14:28:41 INFO TaskManager - Updating FINAL execution state of CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:19)) -> Filter (Context-Links) (b1af12e91fa26a08b2a15ba923a4101d) to FINISHED.
17.Apr. 14:28:44 INFO TaskManager - Register task IterationHead(WorksetIteration (Resolved-Redirects)) (1/10) [DEPLOYING]
17.Apr. 14:28:44 INFO Task - Starting execution of task IterationHead(WorksetIteration (Resolved-Redirects))
17.Apr. 14:28:51 INFO Task - IterationHead(WorksetIteration (Resolved-Redirects)) (1/10) switched to FAILED : java.io.EOFException
at org.apache.flink.runtime.operators.hash.InMemoryPartition$WriteView.nextSegment(InMemoryPartition.java:333)
at org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.advance(AbstractPagedOutputView.java:140)
at org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.writeByte(AbstractPagedOutputView.java:223)
at org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.writeLong(AbstractPagedOutputView.java:291)
at org.apache.flink.runtime.memorymanager.AbstractPagedOutputView.writeDouble(AbstractPagedOutputView.java:307)
at org.apache.flink.api.common.typeutils.base.DoubleSerializer.serialize(DoubleSerializer.java:62)
at org.apache.flink.api.common.typeutils.base.DoubleSerializer.serialize(DoubleSerializer.java:26)
at org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:89)
at org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:29)
at org.apache.flink.runtime.operators.hash.InMemoryPartition.appendRecord(InMemoryPartition.java:219)
at org.apache.flink.runtime.operators.hash.CompactingHashTable.insertOrReplaceRecord(CompactingHashTable.java:536)
at org.apache.flink.runtime.operators.hash.CompactingHashTable.buildTableWithUniqueKey(CompactingHashTable.java:347)
at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.readInitialSolutionSet(IterationHeadPactTask.java:209)
at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.run(IterationHeadPactTask.java:270)
at org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
at org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
at java.lang.Thread.run(Thread.java:745)
17.Apr. 14:28:51 INFO TaskManager - Unregister task with execution ID 1e419a271d3057f104047e3d14a1343c.
17.Apr. 14:28:51 INFO TaskManager - Updating FINAL execution state of IterationHead(WorksetIteration (Resolved-Redirects)) (1e419a271d3057f104047e3d14a1343c) to FAILED.
17.Apr. 14:28:51 INFO Task - CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14)) -> Filter (Redirects) -> Map (Key Extractor 1) (1/10) switched to CANCELING
17.Apr. 14:28:51 INFO RuntimeEnvironment - Canceling CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14)) -> Filter (Redirects) -> Map (Key Extractor 1) (1/10) (de3cc969c8b4539d76efc9c15e4fac86).
17.Apr. 14:28:51 INFO TaskManager - Discarding the results produced by task execution b1af12e91fa26a08b2a15ba923a4101d
17.Apr. 14:28:51 INFO Task - CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14)) -> Filter (Redirects) -> Map (Key Extractor 1) (1/10) switched to CANCELED
17.Apr. 14:28:51 INFO TaskManager - Discarding the results produced by task execution 1e419a271d3057f104047e3d14a1343c
17.Apr. 14:28:51 INFO Task - CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Resolved-Redirects-From-Iteration) (1/10) switched to CANCELING
17.Apr. 14:28:51 INFO TaskManager - Unregister task with execution ID de3cc969c8b4539d76efc9c15e4fac86.
17.Apr. 14:28:51 INFO RuntimeEnvironment - Canceling CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Resolved-Redirects-From-Iteration) (1/10) (122cda6b65a6bcd384c91ea53c196dc5).
17.Apr. 14:28:51 INFO Task - CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Resolved-Redirects-From-Iteration) (1/10) switched to CANCELED
17.Apr. 14:28:51 INFO TaskManager - Updating FINAL execution state of CHAIN DataSource (at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14) (org.apache.flink.api.java.io.TextInput) -> Map (Map at de.uni_potsdam.hpi.coheel.programs.RedirectResolvingProgram.buildProgram(RedirectResolvingProgram.scala:14)) -> Filter (Redirects) -> Map (Key Extractor 1) (de3cc969c8b4539d76efc9c15e4fac86) to CANCELED.
17.Apr. 14:28:51 INFO TaskManager - Unregister task with execution ID 122cda6b65a6bcd384c91ea53c196dc5.
17.Apr. 14:28:51 INFO TaskManager - Updating FINAL execution state of CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Resolved-Redirects-From-Iteration) (122cda6b65a6bcd384c91ea53c196dc5) to CANCELED.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment