Last active
April 29, 2018 16:27
-
-
Save imamitjain/066d0e99990ee24f2da1ddc83eba2012 to your computer and use it in GitHub Desktop.
Batch Job are getting stuck
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2018-04-29 14:57:11,436 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -------------------------------------------------------------------------------- | |
2018-04-29 14:57:11,438 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Starting YARN TaskExecutor runner (Version: 1.5.0, Rev:2af481a, Date:02.04.2018 @ 15:17:35 UTC) | |
2018-04-29 14:57:11,438 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - OS current user: yarn | |
2018-04-29 14:57:13,017 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Current Hadoop/Kerberos user: hadoop | |
2018-04-29 14:57:13,017 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.161-b14 | |
2018-04-29 14:57:13,017 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Maximum heap size: 6647 MiBytes | |
2018-04-29 14:57:13,018 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - JAVA_HOME: /usr/lib/jvm/java-openjdk | |
2018-04-29 14:57:13,019 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Hadoop version: 2.8.3 | |
2018-04-29 14:57:13,019 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - JVM Options: | |
2018-04-29 14:57:13,020 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -Xms6936m | |
2018-04-29 14:57:13,020 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -Xmx6936m | |
2018-04-29 14:57:13,020 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -XX:MaxDirectMemorySize=4072m | |
2018-04-29 14:57:13,020 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -Dlog.file=/var/log/hadoop-yarn/containers/application_1523726493647_2844/container_1523726493647_2844_01_041902/taskmanager.log | |
2018-04-29 14:57:13,020 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -Dlogback.configurationFile=file:./logback.xml | |
2018-04-29 14:57:13,021 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -Dlog4j.configuration=file:./log4j.properties | |
2018-04-29 14:57:13,021 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Program Arguments: | |
2018-04-29 14:57:13,021 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - --configDir | |
2018-04-29 14:57:13,021 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - . | |
2018-04-29 14:57:13,021 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Classpath: lib/flink-dist_2.11-1.5.0.jar:lib/flink-python_2.11-1.5.0.jar:lib/flink-shaded-hadoop2-uber-1.5.0.jar:lib/flink-shaded-include-yarn-0.9.1.jar:lib/flink-yarn_2.11-1.5-SNAPSHOT.jar:lib/guava-18.0.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.7.jar:log4j.properties:logback.xml:flink.jar:flink-conf.yaml::/etc/hadoop/conf:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-annotations-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-streaming.jar:/usr/lib/hadoop/hadoop-distcp.jar:/usr/lib/hadoop/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-archives.jar:/usr/lib/hadoop/hadoop-ant.jar:/usr/lib/hadoop/hadoop-aws.jar:/usr/lib/hadoop/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-rumen.jar:/usr/lib/hadoop/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-openstack.jar:/usr/lib/hadoop/hadoop-azure.jar:/usr/lib/hadoop/hadoop-extras.jar:/usr/lib/hadoop/hadoop-azure-datalake.jar:/usr/lib/hadoop/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-gridmix.jar:/usr/lib/hadoop/hadoop-nfs.jar:/usr/lib/hadoop/hadoop-datajoin.jar:/usr/lib/hadoop/hadoop-common-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop/hadoop-sls.jar:/usr/lib/hadoop/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop/hadoop-common.jar:/usr/lib/hadoop/lib/curator-framework-2.7.1.jar:/usr/lib/hadoop/lib/json-smart-1.1.1.jar:/usr/lib/hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/lib/jcip-annotations-1.0.jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop/lib/gson-2.2.4.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop/lib/jsch-0.1.54.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/jersey-server-1.9.jar:/usr/lib/hadoop/lib/httpcore-4.4.4.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop/lib/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/httpclient-4.5.3.jar:/usr/lib/hadoop/lib/commons-lang-2.6.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-native-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.8.3-amzn-0.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-hdfs/lib/netty-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/okio-1.4.0.jar:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/okhttp-2.4.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/ion-java-1.0.2.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/lib/hadoop-mapreduce/json-smart-1.1.1.jar:/usr/lib/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-core-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.10.jar:/usr/lib/hadoop-mapreduce/jmespath-java-1.11.267.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jcip-annotations-1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce/hadoop-archives-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-archive-logs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/commons-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-auth-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.6.7.jar:/usr/lib/hadoop-mapreduce/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/azure-data-lake-store-sdk-2.2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.54.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/commons-configuration-1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0-M15.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.6.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-aws.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.6.7.jar:/usr/lib/hadoop-mapreduce/httpcore-4.4.4.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/nimbus-jose-jwt-3.9.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.4.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr/lib/hadoop-mapreduce/htrace-core4-4.0.1-incubating.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-mapreduce/hadoop-extras-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-azure.jar:/usr/lib/hadoop-mapreduce/hadoop-extras.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-datalake.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.8.3-amzn-0-tests.jar:/usr/lib/hadoop-mapreduce/hadoop-azure-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/okio-1.4.0.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.2.0.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-s3-1.11.267.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/jackson-dataformat-cbor-2.6.7.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/usr/lib/hadoop-mapreduce/okhttp-2.4.0.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.5.3.jar:/usr/lib/hadoop-mapreduce/commons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/commons-digester-1.8.jar:/usr/lib/hadoop-mapreduce/commons-codec-1.4.jar:/usr/lib/hadoop-mapreduce/aws-java-sdk-kms-1.11.267.jar:/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.8.3-amzn-0.jar:/usr/lib/hadoop-mapreduce/jetty-sslengine-6.1.26-emr.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-mapreduce/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/hadoop-yarn-client-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.8.3-amzn-0.jar:/usr/lib/hadoop-yarn/lib/curator-test-2.7.1.jar:/usr/lib/hadoop-yarn/lib/java-util-1.9.0.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10-tests.jar:/usr/lib/hadoop-yarn/lib/zookeeper-3.4.10.jar:/usr/lib/hadoop-yarn/lib/jettison-1.1.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/usr/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/usr/lib/hadoop-yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/usr/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/usr/lib/hadoop-yarn/lib/commons-math-2.2.jar:/usr/lib/hadoop-yarn/lib/curator-client-2.7.1.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-yarn/lib/jetty-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/jetty-util-6.1.26-emr.jar:/usr/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/usr/lib/hadoop-yarn/lib/activation-1.1.jar:/usr/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/usr/lib/hadoop-yarn/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-yarn/lib/guava-11.0.2.jar:/usr/lib/hadoop-yarn/lib/json-io-2.5.1.jar:/usr/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/usr/lib/hadoop-yarn/lib/fst-2.50.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/jackson-xc-1.9.13.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/javassist-3.18.1-GA.jar:/usr/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/usr/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo.jar:/usr/lib/hadoop-lzo/lib/hadoop-lzo-0.4.19.jar:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/jcl-over-slf4j-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/ion-java-1.0.2.jar:/usr/share/aws/emr/emrfs/lib/emrfs-hadoop-assembly-2.21.0.jar:/usr/share/aws/emr/emrfs/lib/jmespath-java-1.11.267.jar:/usr/share/aws/emr/emrfs/lib/javax.inject-1.jar:/usr/share/aws/emr/emrfs/lib/bcpkix-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/lib/slf4j-api-1.7.21.jar:/usr/share/aws/emr/emrfs/lib/aopalliance-1.0.jar:/usr/share/aws/emr/emrfs/lib/bcprov-jdk15on-1.51.jar:/usr/share/aws/emr/emrfs/auxlib/*:/usr/share/aws/emr/lib/*:/usr/share/aws/emr/ddb/lib/emr-ddb-hadoop.jar:/usr/share/aws/emr/goodies/lib/emr-hadoop-goodies.jar:/usr/share/aws/emr/kinesis/lib/emr-kinesis-hadoop.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink-1.1.0.jar:/usr/share/aws/emr/cloudwatch-sink/lib/cloudwatch-sink.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacecommerceanalytics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rds-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-core-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-clouddirectory-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsmv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-emr-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediaconvert-1.11.267.jar:/usr/share/aws/aws-java-sdk/jmespath-java-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appsync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastore-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iot-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costandusagereport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpleworkflow-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-comprehend-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidentity-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ec2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudtrail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servermigration-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesis-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workmail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-polly-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-simpledb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscalingplans-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elastictranscoder-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pinpoint-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lexmodelbuilding-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplacemeteringservice-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dynamodb-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-storagegateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codegen-maven-plugin-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediapackage-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroupstaggingapi-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-acm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-machinelearning-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudhsm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mobile-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dax-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lightsail-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sns-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codedeploy-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-serverlessapplicationrepository-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-shield-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-applicationautoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mediastoredata-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloud9-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-budgets-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iotjobsdataplane-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-medialive-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticsearch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-inspector-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-importexport-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-test-utils-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitosync-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-api-gateway-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codebuild-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workspaces-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opensdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-snowball-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-autoscaling-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-pricing-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-code-generator-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glacier-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-guardduty-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicediscovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codestar-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-dms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-appstream-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticloadbalancingv2-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-greengrass-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lex-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-resourcegroups-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-support-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticbeanstalk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-route53-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-datapipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-rekognition-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-organizations-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-iam-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ecs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codepipeline-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-logs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworkscm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-xray-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-marketplaceentitlement-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-translate-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-events-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-alexaforbusiness-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemaker-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-s3-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-costexplorer-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-directconnect-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-elasticache-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-gamelift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sqs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sagemakerruntime-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-glue-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-waf-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mechanicalturkrequester-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudwatchmetrics-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-migrationhub-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-lambda-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-discovery-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cognitoidp-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-athena-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-servicecatalog-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-mq-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-batch-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-devicefarm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-sts-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-stepfunctions-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kinesisvideo-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-kms-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-opsworks-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-workdocs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-redshift-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudfront-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-config-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-efs-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-models-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ssm-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-codecommit-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-health-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-cloudformation-1.11.267.jar:/usr/share/aws/aws-java-sdk/aws-java-sdk-ses-1.11.267.jar | |
2018-04-29 14:57:13,023 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - -------------------------------------------------------------------------------- | |
2018-04-29 14:57:13,025 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Registered UNIX signal handlers for [TERM, HUP, INT] | |
2018-04-29 14:57:13,029 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Current working/local Directory: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844 | |
2018-04-29 14:57:13,030 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Current working Directory: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844/container_1523726493647_2844_01_041902 | |
2018-04-29 14:57:13,030 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - TM: remote keytab path obtained null | |
2018-04-29 14:57:13,030 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - TM: remote keytab principal obtained null | |
2018-04-29 14:57:13,036 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: akka.lookup.timeout, 1200 s | |
2018-04-29 14:57:13,037 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: fs.default-scheme, s3a | |
2018-04-29 14:57:13,037 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: high-availability.cluster-id, application_1523726493647_2844 | |
2018-04-29 14:57:13,037 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.address, ip-10-0-22-209.ap-southeast-1.compute.internal | |
2018-04-29 14:57:13,037 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: yarn.heap-cutoff-ratio, 0.3 | |
2018-04-29 14:57:13,037 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: yarn.maximum-failed-containers, 100000 | |
2018-04-29 14:57:13,038 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: io.tmp.dirs, /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844 | |
2018-04-29 14:57:13,038 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: akka.watch.heartbeat.pause, 600 s | |
2018-04-29 14:57:13,038 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.off-heap, on | |
2018-04-29 14:57:13,038 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.fraction, 0.1 | |
2018-04-29 14:57:13,039 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.numberOfTaskSlots, 2 | |
2018-04-29 14:57:13,039 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: akka.ask.timeout, 1200 s | |
2018-04-29 14:57:13,039 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.heap.mb, 11000 | |
2018-04-29 14:57:13,039 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: historyserver.archive.fs.dir, hdfs:///completed-jobs/ | |
2018-04-29 14:57:13,039 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.archive.fs.dir, hdfs:///completed-jobs/ | |
2018-04-29 14:57:13,039 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: web.port, 0 | |
2018-04-29 14:57:13,049 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: classloader.resolve-order, parent-first | |
2018-04-29 14:57:13,049 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: web.timeout, 600000 | |
2018-04-29 14:57:13,049 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.heap.mb, 5120 | |
2018-04-29 14:57:13,050 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: web.tmpdir, /tmp/flink-web-ddc68374-2af3-494c-b751-61c47287df4c | |
2018-04-29 14:57:13,050 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.preallocate, true | |
2018-04-29 14:57:13,050 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.port, 42937 | |
2018-04-29 14:57:13,050 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: rest.await-leader-timeout, 120000 | |
2018-04-29 14:57:13,050 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: rest.port, 0 | |
2018-04-29 14:57:13,051 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: internal.cluster.execution-mode, NORMAL | |
2018-04-29 14:57:13,051 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf | |
2018-04-29 14:57:13,051 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: rest.address, ip-10-0-22-209.ap-southeast-1.compute.internal | |
2018-04-29 14:57:13,106 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - Overriding YARN's temporary file directories with those specified in the Flink config: /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844 | |
2018-04-29 14:57:13,107 INFO org.apache.flink.yarn.YarnTaskExecutorRunner - YARN daemon is running as: hadoop Yarn client user obtainer: hadoop | |
2018-04-29 14:57:13,369 INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user set to hadoop (auth:SIMPLE) | |
2018-04-29 14:57:13,898 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner - Using configured hostname/address for TaskManager: ip-10-0-22-35.ap-southeast-1.compute.internal. | |
2018-04-29 14:57:13,899 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Starting AkkaRpcService at ip-10-0-22-35.ap-southeast-1.compute.internal:0. | |
2018-04-29 14:57:15,050 INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started | |
2018-04-29 14:57:15,455 INFO akka.remote.Remoting - Starting remoting | |
2018-04-29 14:57:15,960 INFO akka.remote.Remoting - Remoting started; listening on addresses :[akka.tcp://flink@ip-10-0-22-35.ap-southeast-1.compute.internal:42925] | |
2018-04-29 14:57:16,046 INFO org.apache.flink.runtime.metrics.MetricRegistryImpl - No metrics reporter configured, no metrics will be exposed/reported. | |
2018-04-29 14:57:16,078 INFO org.apache.flink.runtime.blob.PermanentBlobCache - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844/blobStore-fd77704a-136a-493b-86d6-ba2774b4048f | |
2018-04-29 14:57:16,096 INFO org.apache.flink.runtime.blob.TransientBlobCache - Created BLOB cache storage directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844/blobStore-5bb610c3-43ca-4d5a-8d5f-114391d53d25 | |
2018-04-29 14:57:16,114 INFO org.apache.flink.runtime.io.network.netty.NettyConfig - NettyConfig [server address: ip-10-0-22-35.ap-southeast-1.compute.internal/10.0.22.35, server port: 0, ssl enabled: false, memory segment size (bytes): 32768, transport type: NIO, number of server threads: 2 (manual), number of client threads: 2 (manual), server connect backlog: 0 (use Netty's default), client connect timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's default)] | |
2018-04-29 14:57:16,328 INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Temporary file directory '/mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844': total 69 GB, usable 66 GB (95.65% usable) | |
2018-04-29 14:57:17,799 INFO org.apache.flink.runtime.io.network.buffer.NetworkBufferPool - Allocated 738 MB for network buffer pool (number of memory segments: 23633, bytes per segment: 32768). | |
2018-04-29 14:57:17,992 INFO org.apache.flink.runtime.query.QueryableStateUtils - Could not load Queryable State Client Proxy. Probable reason: flink-queryable-state-runtime is not in the classpath. To enable Queryable State, please move the flink-queryable-state-runtime jar from the opt to the lib folder. | |
2018-04-29 14:57:17,993 INFO org.apache.flink.runtime.query.QueryableStateUtils - Could not load Queryable State Server. Probable reason: flink-queryable-state-runtime is not in the classpath. To enable Queryable State, please move the flink-queryable-state-runtime jar from the opt to the lib folder. | |
2018-04-29 14:57:17,995 INFO org.apache.flink.runtime.io.network.NetworkEnvironment - Starting the network environment and its components. | |
2018-04-29 14:57:18,047 INFO org.apache.flink.runtime.io.network.netty.NettyClient - Successful initialization (took 42 ms). | |
2018-04-29 14:57:18,099 INFO org.apache.flink.runtime.io.network.netty.NettyServer - Successful initialization (took 52 ms). Listening on SocketAddress /10.0.22.35:37603. | |
2018-04-29 14:57:18,101 INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Using 0.1 of the currently free heap space for managed heap memory (663 MB). | |
2018-04-29 14:57:18,743 INFO org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager uses directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844/flink-io-a02d2f13-8ec9-4eb5-bc12-e19dd46e7f9b for spill files. | |
2018-04-29 14:57:18,749 INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /mnt/yarn/usercache/hadoop/appcache/application_1523726493647_2844/flink-dist-cache-59404bb6-fb3e-44e2-a1aa-119473657fda | |
2018-04-29 14:57:18,893 INFO org.apache.flink.runtime.taskexecutor.TaskManagerConfiguration - Messages have a max timeout of 1200000 ms | |
2018-04-29 14:57:18,912 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Starting RPC endpoint for org.apache.flink.runtime.taskexecutor.TaskExecutor at akka://flink/user/taskmanager_0 . | |
2018-04-29 14:57:18,960 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Attempting to register at ResourceManager akka.tcp://flink@ip-10-0-22-209.ap-southeast-1.compute.internal:42937/user/resourcemanager (00000000000000000000000000000000) | |
2018-04-29 14:57:18,981 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Start job leader service. | |
2018-04-29 14:57:19,335 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Resolved ResourceManager address, beginning registration | |
2018-04-29 14:57:19,336 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Registration at ResourceManager attempt 1 (timeout=100ms) | |
2018-04-29 14:57:19,425 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Receive slot request 3adcd0c01f4e63c47ea88d6f876d27d3 for job 080e2a5c53a1d524be79a6a95166c8ff from resource manager with leader id 00000000000000000000000000000000. | |
2018-04-29 14:57:19,440 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Allocated slot for 3adcd0c01f4e63c47ea88d6f876d27d3. | |
2018-04-29 14:57:19,440 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Add job 080e2a5c53a1d524be79a6a95166c8ff for job leader monitoring. | |
2018-04-29 14:57:19,442 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Try to register at job manager akka.tcp://flink@ip-10-0-22-209.ap-southeast-1.compute.internal:42937/user/jobmanager_352 with leader id 00000000-0000-0000-0000-000000000000. | |
2018-04-29 14:57:19,446 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Receive slot request fc80b5a18c123070beb3c2a0e8cb3b6e for job 080e2a5c53a1d524be79a6a95166c8ff from resource manager with leader id 00000000000000000000000000000000. | |
2018-04-29 14:57:19,446 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Allocated slot for fc80b5a18c123070beb3c2a0e8cb3b6e. | |
2018-04-29 14:57:19,446 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Add job 080e2a5c53a1d524be79a6a95166c8ff for job leader monitoring. | |
2018-04-29 14:57:19,461 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Try to register at job manager akka.tcp://flink@ip-10-0-22-209.ap-southeast-1.compute.internal:42937/user/jobmanager_352 with leader id 00000000-0000-0000-0000-000000000000. | |
2018-04-29 14:57:19,463 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Successful registration at resource manager akka.tcp://flink@ip-10-0-22-209.ap-southeast-1.compute.internal:42937/user/resourcemanager under registration id 89e9d38429530d13816b9ca2a218d9ab. | |
2018-04-29 14:57:19,474 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Resolved JobManager address, beginning registration | |
2018-04-29 14:57:19,474 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Resolved JobManager address, beginning registration | |
2018-04-29 14:57:19,477 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Registration at JobManager attempt 1 (timeout=100ms) | |
2018-04-29 14:57:19,500 INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Successful registration at job manager akka.tcp://flink@ip-10-0-22-209.ap-southeast-1.compute.internal:42937/user/jobmanager_352 for job 080e2a5c53a1d524be79a6a95166c8ff. | |
2018-04-29 14:57:19,502 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Establish JobManager connection for job 080e2a5c53a1d524be79a6a95166c8ff. | |
2018-04-29 14:57:19,506 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Offer reserved slots to the leader of job 080e2a5c53a1d524be79a6a95166c8ff. | |
2018-04-29 14:57:19,507 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTable - Activate slot fc80b5a18c123070beb3c2a0e8cb3b6e. | |
2018-04-29 14:57:19,507 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTable - Activate slot 3adcd0c01f4e63c47ea88d6f876d27d3. | |
2018-04-29 14:57:19,631 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (6/20). | |
2018-04-29 14:57:19,655 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (6/20) (e296261ca3c40dc2b7143b86f5a46830) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,655 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (6/20) (e296261ca3c40dc2b7143b86f5a46830) [DEPLOYING] | |
2018-04-29 14:57:19,672 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (6/20) (e296261ca3c40dc2b7143b86f5a46830) [DEPLOYING]. | |
2018-04-29 14:57:19,682 INFO org.apache.flink.runtime.blob.BlobClient - Downloading 080e2a5c53a1d524be79a6a95166c8ff/p-cedebe6f474e494261c033702308a3d051ff6f42-2d19129e420baca5a9e98b00953c63a3 from ip-10-0-22-209.ap-southeast-1.compute.internal/10.0.22.209:35829 | |
2018-04-29 14:57:19,715 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (20/20). | |
2018-04-29 14:57:19,735 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (20/20) (c8cb8498ab2c5e17e4d9db98aa6df38e) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,735 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (20/20) (c8cb8498ab2c5e17e4d9db98aa6df38e) [DEPLOYING] | |
2018-04-29 14:57:19,735 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (20/20) (c8cb8498ab2c5e17e4d9db98aa6df38e) [DEPLOYING]. | |
2018-04-29 14:57:19,758 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (2/20). | |
2018-04-29 14:57:19,767 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (2/20) (ccbfabf4313e80a9093e623562bb3ff1) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,768 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (2/20) (ccbfabf4313e80a9093e623562bb3ff1) [DEPLOYING] | |
2018-04-29 14:57:19,768 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (2/20) (ccbfabf4313e80a9093e623562bb3ff1) [DEPLOYING]. | |
2018-04-29 14:57:19,782 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20). | |
2018-04-29 14:57:19,790 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,791 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255) [DEPLOYING] | |
2018-04-29 14:57:19,791 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255) [DEPLOYING]. | |
2018-04-29 14:57:19,795 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20). | |
2018-04-29 14:57:19,805 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,805 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff) [DEPLOYING] | |
2018-04-29 14:57:19,805 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff) [DEPLOYING]. | |
2018-04-29 14:57:19,808 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (9/20). | |
2018-04-29 14:57:19,819 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (9/20) (6f4a7784ee74f993de79571e88cff1b8) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,820 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (9/20) (6f4a7784ee74f993de79571e88cff1b8) [DEPLOYING] | |
2018-04-29 14:57:19,820 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (9/20) (6f4a7784ee74f993de79571e88cff1b8) [DEPLOYING]. | |
2018-04-29 14:57:19,830 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20). | |
2018-04-29 14:57:19,840 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,841 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) [DEPLOYING] | |
2018-04-29 14:57:19,841 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) [DEPLOYING]. | |
2018-04-29 14:57:19,844 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20). | |
2018-04-29 14:57:19,854 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8) switched from CREATED to DEPLOYING. | |
2018-04-29 14:57:19,854 INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8) [DEPLOYING] | |
2018-04-29 14:57:19,854 INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8) [DEPLOYING]. | |
2018-04-29 14:57:22,005 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8) [DEPLOYING]. | |
2018-04-29 14:57:22,009 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) [DEPLOYING]. | |
2018-04-29 14:57:22,010 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (9/20) (6f4a7784ee74f993de79571e88cff1b8) [DEPLOYING]. | |
2018-04-29 14:57:22,010 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff) [DEPLOYING]. | |
2018-04-29 14:57:22,010 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (20/20) (c8cb8498ab2c5e17e4d9db98aa6df38e) [DEPLOYING]. | |
2018-04-29 14:57:22,010 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (6/20) (e296261ca3c40dc2b7143b86f5a46830) [DEPLOYING]. | |
2018-04-29 14:57:22,011 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (2/20) (ccbfabf4313e80a9093e623562bb3ff1) [DEPLOYING]. | |
2018-04-29 14:57:22,017 INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255) [DEPLOYING]. | |
2018-04-29 14:57:22,061 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (2/20) (ccbfabf4313e80a9093e623562bb3ff1) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,061 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (6/20) (e296261ca3c40dc2b7143b86f5a46830) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,062 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN FullOuterJoin (Join at with(JoinOperatorSetsBase.java:235)) -> Filter (Filter at main(MergeTableSecond.java:225)) (20/20) (c8cb8498ab2c5e17e4d9db98aa6df38e) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,063 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,063 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) -> Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) -> Map (Key Extractor) (9/20) (6f4a7784ee74f993de79571e88cff1b8) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,064 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,066 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,090 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8) switched from DEPLOYING to RUNNING. | |
2018-04-29 14:57:22,228 WARN org.apache.flink.metrics.MetricGroup - The operator name Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:22,245 WARN org.apache.flink.metrics.MetricGroup - The operator name Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:22,259 WARN org.apache.flink.metrics.MetricGroup - The operator name DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:22,275 WARN org.apache.flink.metrics.MetricGroup - The operator name DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:22,349 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8) switched from RUNNING to FINISHED. | |
2018-04-29 14:57:22,350 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8). | |
2018-04-29 14:57:22,366 INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (5/20) (0fadf4f0f341a99bfcacc17b4a291da8) [FINISHED] | |
2018-04-29 14:57:22,372 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and sending final execution state FINISHED to JobManager for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) 0fadf4f0f341a99bfcacc17b4a291da8. | |
2018-04-29 14:57:22,379 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff) switched from RUNNING to FINISHED. | |
2018-04-29 14:57:22,379 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff). | |
2018-04-29 14:57:22,380 INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) (19/20) (3c08798a440ac85e201ff091b933c9ff) [FINISHED] | |
2018-04-29 14:57:22,393 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and sending final execution state FINISHED to JobManager for task CHAIN DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.io.TextInputFormat)) -> Map (Data Source org.apache.flink.api.java.io.TextInputFormat [s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/28/, s3a://limeroad-logs/mysqlbin_lradmin_qc_status/redshift_logs/2018/4/29/5/]) -> Map (Key Extractor) -> Combine (GroupReduce at first(SortedGrouping.java:210)) 3c08798a440ac85e201ff091b933c9ff. | |
2018-04-29 14:57:22,617 WARN org.apache.flink.metrics.MetricGroup - The operator name Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:22,621 WARN org.apache.flink.metrics.MetricGroup - The operator name Map (Data Source Parquet s3a://limeroad-logs/emr-testing/ldp_test/mysql/mysqlbin_lradmin_qc_status/main/1524923790277/) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:22,624 WARN org.apache.flink.metrics.MetricGroup - The operator name DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:22,626 WARN org.apache.flink.metrics.MetricGroup - The operator name DataSource (at createInput(ExecutionEnvironment.java:547) (org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat)) exceeded the 80 characters length limit and was truncated. | |
2018-04-29 14:57:24,437 INFO org.apache.flink.runtime.taskmanager.Task - Producer 98f5976716234236dc69fb0e82a0cc34 of partition 5b8dd5ac2df14266080a8e049defbe38 disposed. Cancelling execution. | |
org.apache.flink.runtime.jobmanager.PartitionProducerDisposedException: Execution 98f5976716234236dc69fb0e82a0cc34 producing partition 5b8dd5ac2df14266080a8e049defbe38 has already been disposed. | |
at org.apache.flink.runtime.jobmaster.JobMaster.requestPartitionState(JobMaster.java:610) | |
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source) | |
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) | |
at java.lang.reflect.Method.invoke(Method.java:498) | |
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:210) | |
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:154) | |
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleMessage(FencedAkkaRpcActor.java:69) | |
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$onReceive$1(AkkaRpcActor.java:132) | |
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544) | |
at akka.actor.Actor$class.aroundReceive(Actor.scala:502) | |
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95) | |
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) | |
at akka.actor.ActorCell.invoke(ActorCell.scala:495) | |
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) | |
at akka.dispatch.Mailbox.run(Mailbox.scala:224) | |
at akka.dispatch.Mailbox.exec(Mailbox.scala:234) | |
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) | |
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) | |
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) | |
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) | |
2018-04-29 14:57:24,442 INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef). | |
2018-04-29 14:57:24,442 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) switched from RUNNING to CANCELING. | |
2018-04-29 14:57:24,448 INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef). | |
2018-04-29 14:57:24,454 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) switched from CANCELING to CANCELED. | |
2018-04-29 14:57:24,455 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef). | |
2018-04-29 14:57:24,491 INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (8/20) (c4e714c0ac6a69d09c59a73f5f7393ef) [CANCELED] | |
2018-04-29 14:57:24,492 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and sending final execution state CANCELED to JobManager for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) c4e714c0ac6a69d09c59a73f5f7393ef. | |
2018-04-29 14:57:27,308 INFO org.apache.flink.runtime.taskmanager.Task - CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255) switched from RUNNING to FINISHED. | |
2018-04-29 14:57:27,308 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255). | |
2018-04-29 14:57:27,309 INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) (5/20) (978fc52a96221e50f2ac9f3c1ac29255) [FINISHED] | |
2018-04-29 14:57:27,309 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and sending final execution state FINISHED to JobManager for task CHAIN Reduce (GroupReduce at first(SortedGrouping.java:210)) -> Map (Key Extractor) 978fc52a96221e50f2ac9f3c1ac29255. | |
2018-04-29 14:57:28,581 WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. | |
2018-04-29 14:57:29,024 WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. | |
2018-04-29 14:57:29,469 INFO org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor [.snappy] | |
2018-04-29 14:57:29,473 INFO org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor [.snappy] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment