Created
June 23, 2017 17:22
-
-
Save cresny/0e109f843730b64d3a330f8fb06bb8a6 to your computer and use it in GitHub Desktop.
1 successful run and cp after resume, next 2 fail
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2017-06-23 16:59:56.580 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Removing web dashboard root cache directory /tmp/flink-web-dbb2f241-ccfd-4a51-a8b3-de92d1b5f72d | |
| 2017-06-23 17:00:08.453 [ip-10-150-96-68] INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user set to root (auth:SIMPLE) | |
| 2017-06-23 17:00:10.124 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobServer - Created BLOB server storage directory /tmp/blobStore-b9a67936-f87e-459e-b32a-a7c352f61ec6 | |
| 2017-06-23 17:00:10.126 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobServer - Started BLOB server at 0.0.0.0:45857 - max concurrent requests: 50 - max backlog: 1000 | |
| 2017-06-23 17:00:10.193 [ip-10-150-96-68] INFO org.apache.flink.runtime.metrics.MetricRegistry - Configuring JMXReporter with {port=8789, class=org.apache.flink.metrics.jmx.JMXReporter}. | |
| 2017-06-23 17:00:10.258 [ip-10-150-96-68] INFO org.apache.flink.runtime.metrics.MetricRegistry - Reporting metrics for reporter jmx of type org.apache.flink.metrics.jmx.JMXReporter. | |
| 2017-06-23 17:00:10.266 [ip-10-150-96-68] INFO org.apache.flink.runtime.jobmanager.MemoryArchivist - Started memory archivist akka://flink/user/$a | |
| 2017-06-23 17:00:10.290 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of JobManager log file: /var/log/hadoop-yarn/containers/application_1498231688207_0004/container_1498231688207_0004_01_000001/jobmanager.log | |
| 2017-06-23 17:00:10.307 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of JobManager stdout file: /var/log/hadoop-yarn/containers/application_1498231688207_0004/container_1498231688207_0004_01_000001/jobmanager.out | |
| 2017-06-23 17:00:10.308 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Using directory /tmp/flink-web-49d7f90f-da93-4298-a1fb-8380cfc1568a for the web interface files | |
| 2017-06-23 17:00:10.308 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Using directory /tmp/wupload for web frontend JAR file uploads | |
| 2017-06-23 17:00:10.556 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Web frontend listening at 0:0:0:0:0:0:0:0:40919 | |
| 2017-06-23 17:00:10.557 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Starting with JobManager akka.tcp://flink@ip-10-150-96-68.ec2.internal:34539/user/jobmanager on port 40919 | |
| 2017-06-23 17:00:10.559 [ip-10-150-96-68] INFO org.apache.flink.runtime.webmonitor.JobManagerRetriever - New leader reachable under akka.tcp://flink@ip-10-150-96-68.ec2.internal:34539/user/jobmanager:00000000-0000-0000-0000-000000000000. | |
| 2017-06-23 17:00:14.092 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.TaskManager - Loading configuration from . | |
| 2017-06-23 17:00:14.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user set to root (auth:SIMPLE) | |
| 2017-06-23 17:00:14.194 [ip-10-150-96-244] INFO org.apache.flink.runtime.util.LeaderRetrievalUtils - Trying to select the network interface and address to use by connecting to the leading JobManager. | |
| 2017-06-23 17:00:14.194 [ip-10-150-96-244] INFO org.apache.flink.runtime.util.LeaderRetrievalUtils - TaskManager will try to connect for 10000 milliseconds before falling back to heuristics | |
| 2017-06-23 17:00:14.196 [ip-10-150-96-244] INFO org.apache.flink.runtime.net.ConnectionUtils - Retrieved new target address ip-10-150-96-68.ec2.internal/10.150.96.68:34539. | |
| 2017-06-23 17:00:14.201 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.TaskManager - TaskManager will use hostname/address 'ip-10-150-96-244' (10.150.96.244) for communication. | |
| 2017-06-23 17:00:14.202 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.TaskManager - Starting TaskManager | |
| 2017-06-23 17:00:14.202 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.TaskManager - Starting TaskManager actor system at ip-10-150-96-244:0. | |
| 2017-06-23 17:00:14.756 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.TaskManager - Starting TaskManager actor | |
| 2017-06-23 17:00:14.764 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.network.netty.NettyConfig - NettyConfig [server address: ip-10-150-96-244/10.150.96.244, server port: 0, ssl enabled: false, memory segment size (bytes): 32768, transport type: NIO, number of server threads: 2 (manual), number of client threads: 2 (manual), server connect backlog: 0 (use Netty's default), client connect timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's default)] | |
| 2017-06-23 17:00:14.773 [ip-10-150-96-244] INFO o.apache.flink.runtime.taskexecutor.TaskManagerConfiguration - Messages have a max timeout of 10000 ms | |
| 2017-06-23 17:00:14.778 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Temporary file directory '/media/flink/tmp0': total 39 GB, usable 39 GB (100.00% usable) | |
| 2017-06-23 17:00:14.778 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Temporary file directory '/media/flink/tmp1': total 39 GB, usable 39 GB (100.00% usable) | |
| 2017-06-23 17:00:14.778 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Temporary file directory '/media/flink/tmp0': total 39 GB, usable 39 GB (100.00% usable) | |
| 2017-06-23 17:00:14.779 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Temporary file directory '/media/flink/tmp1': total 39 GB, usable 39 GB (100.00% usable) | |
| 2017-06-23 17:00:15.009 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.network.buffer.NetworkBufferPool - Allocated 458 MB for network buffer pool (number of memory segments: 14668, bytes per segment: 32768). | |
| 2017-06-23 17:00:15.134 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.network.NetworkEnvironment - Starting the network environment and its components. | |
| 2017-06-23 17:00:15.142 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.network.netty.NettyClient - Successful initialization (took 2 ms). | |
| 2017-06-23 17:00:15.174 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.network.netty.NettyServer - Successful initialization (took 31 ms). Listening on SocketAddress /10.150.96.244:45447. | |
| 2017-06-23 17:00:15.476 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Limiting managed memory to 0.3 of the currently free heap space (1237 MB), memory will be allocated lazily. | |
| 2017-06-23 17:00:15.483 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager uses directory /media/flink/tmp0/flink-io-46e5abc9-d9ac-4dd7-8466-66acb9d2e757 for spill files. | |
| 2017-06-23 17:00:15.483 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager uses directory /media/flink/tmp1/flink-io-2df52e4f-5d79-452b-8c57-b9d3e4561cf1 for spill files. | |
| 2017-06-23 17:00:15.483 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager uses directory /media/flink/tmp0/flink-io-bef99ca3-cc8f-4f71-b584-8a4750c177f6 for spill files. | |
| 2017-06-23 17:00:15.484 [ip-10-150-96-244] INFO org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager uses directory /media/flink/tmp1/flink-io-24bca7b0-47fc-4f06-bcc2-ae484fc76cc5 for spill files. | |
| 2017-06-23 17:00:15.537 [ip-10-150-96-244] INFO org.apache.flink.runtime.metrics.MetricRegistry - Configuring JMXReporter with {port=8789, class=org.apache.flink.metrics.jmx.JMXReporter}. | |
| 2017-06-23 17:00:15.608 [ip-10-150-96-244] INFO org.apache.flink.runtime.metrics.MetricRegistry - Reporting metrics for reporter jmx of type org.apache.flink.metrics.jmx.JMXReporter. | |
| 2017-06-23 17:00:15.622 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp0/flink-dist-cache-6c016a6c-3314-4f23-95dd-3f45b9c67919 | |
| 2017-06-23 17:00:15.623 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp1/flink-dist-cache-1b268566-e1f5-4b82-a4c9-640d28f4ee27 | |
| 2017-06-23 17:00:15.623 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp0/flink-dist-cache-0a0aa9e0-fb3e-419d-9df2-cfb9d6d0f814 | |
| 2017-06-23 17:00:15.624 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp1/flink-dist-cache-bf84cded-944e-4c91-9dc4-72dce79f21a3 | |
| 2017-06-23 17:00:15.632 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp0/flink-dist-cache-4c5009e2-8aa3-4a69-8f04-4c37b1f22a0f | |
| 2017-06-23 17:00:15.634 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp1/flink-dist-cache-921431a4-53c3-48c6-bda8-43e7d5e0e9f4 | |
| 2017-06-23 17:00:15.634 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp0/flink-dist-cache-0889b6e5-985d-40ec-a86a-c529a6a01754 | |
| 2017-06-23 17:00:15.635 [ip-10-150-96-244] INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory /media/flink/tmp1/flink-dist-cache-e2ccb175-51ba-4d1f-b5fe-01c830ea5ea6 | |
| 2017-06-23 17:00:15.755 [ip-10-150-96-68] INFO org.apache.flink.runtime.instance.InstanceManager - Registered TaskManager at ip-10-150-96-244 (akka.tcp://flink@ip-10-150-96-244:41131/user/taskmanager) as 84651f17899ab6d2e91b559461ea2c61. Current number of registered hosts is 1. Current number of alive task slots is 2. | |
| 2017-06-23 17:00:15.772 [ip-10-150-96-244] INFO org.apache.flink.runtime.blob.BlobCache - Created BLOB cache storage directory /tmp/blobStore-f9f9d963-5591-4ab5-80b9-000c9db74c43 | |
| 2017-06-23 17:00:34.935 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:00:35.274 [ip-10-150-96-68] INFO org.apache.flink.runtime.client.JobClient - Checking and uploading JAR files | |
| 2017-06-23 17:00:35.275 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:00:35.622 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job recovers via failover strategy: full graph restart | |
| 2017-06-23 17:00:35.653 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:00:35.653 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:00:35.658 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (05bace2ff9d97b50751a12a8b7297d07) switched from state CREATED to RUNNING. | |
| 2017-06-23 17:00:35.660 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.664 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.665 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.665 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.665 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.665 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.665 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.666 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:00:35.668 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.669 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: S3 File Watcher (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.674 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.674 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: null source -> Sink: Redshift Transmitter (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.675 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.675 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.678 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.678 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.678 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.678 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.680 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.680 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.680 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.681 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.687 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:00:35.688 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:00:35.724 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.724 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) [DEPLOYING] | |
| 2017-06-23 17:00:35.732 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) [DEPLOYING]. | |
| 2017-06-23 17:00:35.732 [ip-10-150-96-244] INFO org.apache.flink.runtime.blob.BlobCache - Downloading 44361bb3236c3a0acd035f3f03cbd7933fdb912f from ip-10-150-96-68.ec2.internal/10.150.96.68:45857 | |
| 2017-06-23 17:00:35.733 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.734 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) [DEPLOYING] | |
| 2017-06-23 17:00:35.735 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) [DEPLOYING]. | |
| 2017-06-23 17:00:35.756 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.756 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) [DEPLOYING] | |
| 2017-06-23 17:00:35.756 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) [DEPLOYING]. | |
| 2017-06-23 17:00:35.763 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.764 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) [DEPLOYING] | |
| 2017-06-23 17:00:35.764 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.779 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) [DEPLOYING]. | |
| 2017-06-23 17:00:35.787 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) [DEPLOYING] | |
| 2017-06-23 17:00:35.787 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) [DEPLOYING]. | |
| 2017-06-23 17:00:35.796 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.796 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) [DEPLOYING] | |
| 2017-06-23 17:00:35.796 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.796 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) [DEPLOYING]. | |
| 2017-06-23 17:00:35.797 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) [DEPLOYING] | |
| 2017-06-23 17:00:35.797 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) [DEPLOYING]. | |
| 2017-06-23 17:00:35.805 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:00:35.806 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) [DEPLOYING] | |
| 2017-06-23 17:00:35.806 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) [DEPLOYING]. | |
| 2017-06-23 17:00:36.164 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) [DEPLOYING]. | |
| 2017-06-23 17:00:36.166 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) [DEPLOYING]. | |
| 2017-06-23 17:00:36.167 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) [DEPLOYING]. | |
| 2017-06-23 17:00:36.168 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) [DEPLOYING]. | |
| 2017-06-23 17:00:36.167 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) [DEPLOYING]. | |
| 2017-06-23 17:00:36.171 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) [DEPLOYING]. | |
| 2017-06-23 17:00:36.174 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) [DEPLOYING]. | |
| 2017-06-23 17:00:36.174 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) [DEPLOYING]. | |
| 2017-06-23 17:00:36.179 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.195 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.196 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.197 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.199 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.184 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.192 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.191 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.194 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.217 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.216 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.218 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.219 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.224 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:36.225 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:00:38.140 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.143 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.140 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.143 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.150 [ip-10-150-96-244] INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Attempting to load RocksDB native library and store it under '/media/flink/tmp0' | |
| 2017-06-23 17:00:38.150 [ip-10-150-96-244] DEBUG org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Attempting to create RocksDB native library folder /media/flink/tmp0/rocksdb-lib-cd1a0327c769d75f452741a81d693798 | |
| 2017-06-23 17:00:38.155 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.155 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.225 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.225 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.269 [ip-10-150-96-244] INFO org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Successfully loaded RocksDB native library | |
| 2017-06-23 17:00:38.274 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.274 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.275 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.275 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.280 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:00:38.280 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: null. | |
| 2017-06-23 17:00:38.305 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.305 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.333 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.333 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.334 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:00:38.334 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: null. | |
| 2017-06-23 17:00:38.673 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:38.673 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.166 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.166 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.208 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.208 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.215 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.215 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.255 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.255 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.293 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.293 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.321 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.321 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.355 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.355 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.372 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.372 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.395 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.395 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.710 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:00:39.710 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:05.760 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 1 @ 1498237265659 | |
| 2017-06-23 17:01:05.760 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 1 @ 1498237265659 | |
| 2017-06-23 17:01:05.787 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:05.787 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:10.092 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:01:10.097 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 31 ms. | |
| 2017-06-23 17:01:10.097 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 31 ms. | |
| 2017-06-23 17:01:10.124 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:01:10.133 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:10.133 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:10.134 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:10.134 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:10.235 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 92 ms. | |
| 2017-06-23 17:01:10.235 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 92 ms. | |
| 2017-06-23 17:01:10.278 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 128 ms. | |
| 2017-06-23 17:01:10.278 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 128 ms. | |
| 2017-06-23 17:01:10.697 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 572 ms. | |
| 2017-06-23 17:01:10.697 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 572 ms. | |
| 2017-06-23 17:01:10.716 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:10.716 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:11.402 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1152 ms. | |
| 2017-06-23 17:01:11.402 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1152 ms. | |
| 2017-06-23 17:01:11.586 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes] | |
| 2017-06-23 17:01:11.587 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes] | |
| 2017-06-23 17:01:11.588 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/30b969f2-0b01-4d99-80d0-8144935ef489'} | |
| 2017-06-23 17:01:11.718 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1436 ms. | |
| 2017-06-23 17:01:11.718 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1436 ms. | |
| 2017-06-23 17:01:11.868 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/66fa2868-df90-43c1-8655-b80a86873655 [8289 bytes] | |
| 2017-06-23 17:01:11.869 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/7770c000-1ce5-408e-8a6b-0ce52c75e66d'} | |
| 2017-06-23 17:01:11.902 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:11.902 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:12.045 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes] | |
| 2017-06-23 17:01:12.045 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes] | |
| 2017-06-23 17:01:12.046 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/d47150ad-9da0-4900-96b4-e6a97099fd78'} | |
| 2017-06-23 17:01:12.226 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 822 ms. | |
| 2017-06-23 17:01:12.226 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 822 ms. | |
| 2017-06-23 17:01:12.430 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/9bcb18b3-16de-4b8a-91bc-814f8617ba63 [8289 bytes] | |
| 2017-06-23 17:01:12.431 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3dbcf9c7-dda7-4efc-89c6-b076f864dd1c'} | |
| 2017-06-23 17:01:12.439 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:12.439 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:12.599 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 880 ms. | |
| 2017-06-23 17:01:12.599 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 880 ms. | |
| 2017-06-23 17:01:12.711 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 485 ms. | |
| 2017-06-23 17:01:12.756 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:12.756 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:12.711 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 485 ms. | |
| 2017-06-23 17:01:13.260 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 661 ms. | |
| 2017-06-23 17:01:13.260 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 661 ms. | |
| 2017-06-23 17:01:13.274 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:13.274 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:13.274 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:01:13.274 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:01:13.274 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes] | |
| 2017-06-23 17:01:13.274 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes] | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1} | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1} | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes] | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes] | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1} | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1} | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}} | |
| 2017-06-23 17:01:13.276 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}} | |
| 2017-06-23 17:01:14.339 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 1 (22212164 bytes in 8679 ms). | |
| 2017-06-23 17:01:14.339 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 1 (22212164 bytes in 8679 ms). | |
| 2017-06-23 17:01:14.339 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 10390085), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 6109065), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 5709152), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:01:14.339 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 10390085), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 6109065), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 5709152), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:01:38.338 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 2 @ 1498237298338 | |
| 2017-06-23 17:01:38.338 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 2 @ 1498237298338 | |
| 2017-06-23 17:01:38.344 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:38.344 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:48.624 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:01:48.624 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:01:48.651 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:01:48.680 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:48.680 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:48.683 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:48.683 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:48.675 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:01:48.711 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 47 ms. | |
| 2017-06-23 17:01:48.711 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 47 ms. | |
| 2017-06-23 17:01:48.748 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 54 ms. | |
| 2017-06-23 17:01:48.748 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 54 ms. | |
| 2017-06-23 17:01:49.474 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 849 ms. | |
| 2017-06-23 17:01:49.474 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 849 ms. | |
| 2017-06-23 17:01:49.486 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:49.486 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:49.957 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1241 ms. | |
| 2017-06-23 17:01:49.957 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1241 ms. | |
| 2017-06-23 17:01:50.139 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1366 ms. | |
| 2017-06-23 17:01:50.139 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1366 ms. | |
| 2017-06-23 17:01:50.420 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes] | |
| 2017-06-23 17:01:50.421 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes] | |
| 2017-06-23 17:01:50.421 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:01:50.422 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/d88d5826-3713-4af0-b7bf-3fbd442f83d4'} | |
| 2017-06-23 17:01:50.439 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 481 ms. | |
| 2017-06-23 17:01:50.439 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 481 ms. | |
| 2017-06-23 17:01:50.724 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b22865dd-7dcc-4eb9-92d8-0c3448533edc [8289 bytes] | |
| 2017-06-23 17:01:50.724 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/0e769d8f-7807-419f-81b2-3214ba02d68c'} | |
| 2017-06-23 17:01:50.733 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:50.733 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:50.846 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes] | |
| 2017-06-23 17:01:50.846 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes] | |
| 2017-06-23 17:01:50.846 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:01:50.846 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/9278f452-b60e-42f6-bff4-7ce0ec2e7835'} | |
| 2017-06-23 17:01:50.938 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 798 ms. | |
| 2017-06-23 17:01:50.938 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 798 ms. | |
| 2017-06-23 17:01:51.226 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/d208acdf-d8ab-46b8-892c-bd8698bdc84b [8289 bytes] | |
| 2017-06-23 17:01:51.227 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/fc252ec9-9f9e-4544-9e58-4ed3cb3c8760'} | |
| 2017-06-23 17:01:51.233 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:51.233 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:51.307 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 867 ms. | |
| 2017-06-23 17:01:51.307 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 867 ms. | |
| 2017-06-23 17:01:51.323 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:51.323 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:52.157 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 1219 ms. | |
| 2017-06-23 17:01:52.157 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 1219 ms. | |
| 2017-06-23 17:01:52.167 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:52.167 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:01:52.167 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}} | |
| 2017-06-23 17:01:52.167 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}} | |
| 2017-06-23 17:01:52.168 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@252420ca | |
| 2017-06-23 17:01:52.168 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@252420ca | |
| 2017-06-23 17:01:52.168 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2} | |
| 2017-06-23 17:01:52.168 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2} | |
| 2017-06-23 17:01:52.168 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes] | |
| 2017-06-23 17:01:52.168 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes] | |
| 2017-06-23 17:01:52.169 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1} | |
| 2017-06-23 17:01:52.169 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1} | |
| 2017-06-23 17:01:52.169 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@211bb0b6 | |
| 2017-06-23 17:01:52.169 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@211bb0b6 | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2} | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2} | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes] | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes] | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1} | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1} | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2}} | |
| 2017-06-23 17:01:52.173 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2}} | |
| 2017-06-23 17:01:53.002 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 2 (55606715 bytes in 14663 ms). | |
| 2017-06-23 17:01:53.002 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 2 (55606715 bytes in 14663 ms). | |
| 2017-06-23 17:01:53.002 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 31161101), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 17211888), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 7229864), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:01:53.002 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 31161101), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 17211888), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 7229864), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:02:17.002 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237337001 | |
| 2017-06-23 17:02:17.002 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237337001 | |
| 2017-06-23 17:02:17.005 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:17.005 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:17.412 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:02:17.412 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:02:17.413 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:02:17.447 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:02:17.452 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:17.452 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:17.454 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:17.454 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:17.531 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 91 ms. | |
| 2017-06-23 17:02:17.531 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 91 ms. | |
| 2017-06-23 17:02:17.542 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 83 ms. | |
| 2017-06-23 17:02:17.542 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 83 ms. | |
| 2017-06-23 17:02:18.377 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 964 ms. | |
| 2017-06-23 17:02:18.377 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 964 ms. | |
| 2017-06-23 17:02:18.383 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:18.383 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:19.068 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1533 ms. | |
| 2017-06-23 17:02:19.068 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1533 ms. | |
| 2017-06-23 17:02:19.208 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes] | |
| 2017-06-23 17:02:19.209 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000013.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes] | |
| 2017-06-23 17:02:19.209 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:19.209 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:19.209 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/8813e1be-975a-484e-9b13-272d90853708'} | |
| 2017-06-23 17:02:19.235 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1689 ms. | |
| 2017-06-23 17:02:19.235 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1689 ms. | |
| 2017-06-23 17:02:19.447 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/c108c524-19ed-427b-9cc8-27892126a576 [8289 bytes] | |
| 2017-06-23 17:02:19.450 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/87c2f5b0-2310-465b-a4cb-bef9cae8391a'} | |
| 2017-06-23 17:02:19.457 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:19.457 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:19.458 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes] | |
| 2017-06-23 17:02:19.458 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000013.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes] | |
| 2017-06-23 17:02:19.458 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:19.458 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:19.458 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/130afcfd-d227-4ca7-9161-6cf9b118aae8'} | |
| 2017-06-23 17:02:19.762 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 694 ms. | |
| 2017-06-23 17:02:19.762 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 694 ms. | |
| 2017-06-23 17:02:19.880 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 645 ms. | |
| 2017-06-23 17:02:19.880 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 645 ms. | |
| 2017-06-23 17:02:19.920 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/df6f6fe0-c2f4-4cb1-8266-1c6c02328ba2 [8289 bytes] | |
| 2017-06-23 17:02:19.920 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/09d4d202-33b3-4960-856b-7d30d3a4f344'} | |
| 2017-06-23 17:02:19.927 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:19.927 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:20.903 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 1139 ms. | |
| 2017-06-23 17:02:20.903 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 1139 ms. | |
| 2017-06-23 17:02:20.931 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:20.931 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:21.621 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 1741 ms. | |
| 2017-06-23 17:02:21.621 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 1741 ms. | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2}} | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2}} | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@7ff89e5b | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@7ff89e5b | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3} | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3} | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@7bc7157c | |
| 2017-06-23 17:02:21.686 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@7bc7157c | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2} | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2} | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes] | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes] | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1} | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1} | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3ec59460 | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3ec59460 | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3} | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3} | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@340042aa | |
| 2017-06-23 17:02:21.687 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@340042aa | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2} | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2} | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes] | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes] | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1} | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1} | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:02:21.688 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:02:22.899 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:02:22.899 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:02:22.903 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2} | |
| 2017-06-23 17:02:22.903 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2} | |
| 2017-06-23 17:02:23.809 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst | |
| 2017-06-23 17:02:23.809 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst | |
| 2017-06-23 17:02:23.809 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2} | |
| 2017-06-23 17:02:23.809 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2} | |
| 2017-06-23 17:02:24.570 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 3 (71387523 bytes in 4909 ms). | |
| 2017-06-23 17:02:24.570 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 3 (71387523 bytes in 4909 ms). | |
| 2017-06-23 17:02:24.570 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 45513069), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 16261124), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 9609468), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:02:24.570 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 45513069), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 16261124), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 9609468), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:02:48.570 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 4 @ 1498237368570 | |
| 2017-06-23 17:02:48.570 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 4 @ 1498237368570 | |
| 2017-06-23 17:02:48.574 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:48.574 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task cf72a4f1906a9509c2df8956e50bd4cf of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:53.759 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:02:53.759 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:02:53.804 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:02:53.822 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:02:53.828 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:53.828 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 2d0ac8997e91fffc16fe4e09d918f9a3 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:53.829 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:53.829 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 70744a96366789923f77ae4bc035a5a5 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:53.883 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 67 ms. | |
| 2017-06-23 17:02:53.883 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 67 ms. | |
| 2017-06-23 17:02:53.901 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 62 ms. | |
| 2017-06-23 17:02:53.901 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 62 ms. | |
| 2017-06-23 17:02:54.359 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 600 ms. | |
| 2017-06-23 17:02:54.359 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-11-thread-1,5,Flink Task Threads] took 600 ms. | |
| 2017-06-23 17:02:54.368 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:54.368 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 4e8169e27914a633db7f6e7d887d5385 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:56.129 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes] | |
| 2017-06-23 17:02:56.131 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000014.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes] | |
| 2017-06-23 17:02:56.146 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000013.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:56.146 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:56.146 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:56.147 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/5eff213f-c77b-44a5-83c0-2ef29c6bf6b0'} | |
| 2017-06-23 17:02:56.190 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes] | |
| 2017-06-23 17:02:56.190 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000014.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes] | |
| 2017-06-23 17:02:56.191 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000013.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:56.191 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:56.191 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:02:56.191 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/2c4e764c-7f10-4c77-9bcd-7e54ddff668d'} | |
| 2017-06-23 17:02:56.327 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 2423 ms. | |
| 2017-06-23 17:02:56.327 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 2423 ms. | |
| 2017-06-23 17:02:56.358 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 2468 ms. | |
| 2017-06-23 17:02:56.358 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 2468 ms. | |
| 2017-06-23 17:02:56.405 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:56.405 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task 32ed163f05f8113b5f3c2e78d9c03675 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:56.369 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/9d47217c-7063-4929-96d0-a479b7f458ec [8289 bytes] | |
| 2017-06-23 17:02:56.383 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/e2da528b-7f4a-4a05-b78d-7a7c99880799 [8289 bytes] | |
| 2017-06-23 17:02:56.395 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/9fa3d2b1-6d29-468f-ace0-9e44c9b4c010'} | |
| 2017-06-23 17:02:56.397 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/16aeab6e-219f-45ed-a30f-8bf74a15e80c'} | |
| 2017-06-23 17:02:56.411 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:56.411 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task d35fea04f5cb26bb59c7e54addd4e6cd of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:56.717 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 389 ms. | |
| 2017-06-23 17:02:56.717 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 389 ms. | |
| 2017-06-23 17:02:57.183 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 825 ms. | |
| 2017-06-23 17:02:57.183 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 825 ms. | |
| 2017-06-23 17:02:58.890 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 2172 ms. | |
| 2017-06-23 17:02:58.890 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-8-thread-1,5,Flink Task Threads] took 2172 ms. | |
| 2017-06-23 17:02:58.917 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:02:58.917 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task faa8faf188f8e1bcc455d16854189378 of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:03:00.375 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 3171 ms. | |
| 2017-06-23 17:03:00.375 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07, asynchronous part) in thread Thread[pool-12-thread-1,5,Flink Task Threads] took 3171 ms. | |
| 2017-06-23 17:03:00.386 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:03:00.386 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 4 from task de0ca8d35dfae1bd6fd14fc628bcc56d of job 05bace2ff9d97b50751a12a8b7297d07. | |
| 2017-06-23 17:03:00.386 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:03:00.386 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:03:00.386 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@1cb2f9c1 | |
| 2017-06-23 17:03:00.386 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@1cb2f9c1 | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@188d6e8f | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@188d6e8f | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes] | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes] | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@5d8ff3fa | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@5d8ff3fa | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=2} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=2} | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@676f4f49 | |
| 2017-06-23 17:03:00.387 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@676f4f49 | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@6c18ce94 | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@6c18ce94 | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=3} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes] | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes] | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@1f122c8e | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@1f122c8e | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=2} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=2} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=3}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=3}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=2}} | |
| 2017-06-23 17:03:00.388 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=2}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=3}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=3}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=3}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=2}} | |
| 2017-06-23 17:03:01.530 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:03:01.530 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:03:01.530 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2} | |
| 2017-06-23 17:03:01.530 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=2} | |
| 2017-06-23 17:03:01.531 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst | |
| 2017-06-23 17:03:01.531 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst | |
| 2017-06-23 17:03:01.531 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2} | |
| 2017-06-23 17:03:01.531 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=2} | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2} | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=2} | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2} | |
| 2017-06-23 17:03:02.030 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=2} | |
| 2017-06-23 17:03:02.770 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 4 (94345745 bytes in 12096 ms). | |
| 2017-06-23 17:03:02.770 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 4 (94345745 bytes in 12096 ms). | |
| 2017-06-23 17:03:02.770 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 64374795), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 20797012), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 9170076), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:03:02.770 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 64374795), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 20797012), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 9170076), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:03:16.078 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (05bace2ff9d97b50751a12a8b7297d07) switched from state RUNNING to CANCELLING. | |
| 2017-06-23 17:03:16.078 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.080 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.080 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.081 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.081 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.081 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.081 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.081 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.080 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385). | |
| 2017-06-23 17:03:16.081 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.094 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385). | |
| 2017-06-23 17:03:16.096 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf). | |
| 2017-06-23 17:03:16.096 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.097 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf). | |
| 2017-06-23 17:03:16.097 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378). | |
| 2017-06-23 17:03:16.097 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.098 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378). | |
| 2017-06-23 17:03:16.098 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d). | |
| 2017-06-23 17:03:16.098 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.099 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d). | |
| 2017-06-23 17:03:16.105 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675). | |
| 2017-06-23 17:03:16.105 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.110 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.110 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675). | |
| 2017-06-23 17:03:16.111 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf). | |
| 2017-06-23 17:03:16.111 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd). | |
| 2017-06-23 17:03:16.111 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.111 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd). | |
| 2017-06-23 17:03:16.112 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3). | |
| 2017-06-23 17:03:16.112 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.112 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3). | |
| 2017-06-23 17:03:16.112 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5). | |
| 2017-06-23 17:03:16.112 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:03:16.113 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5). | |
| 2017-06-23 17:03:16.127 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.127 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3). | |
| 2017-06-23 17:03:16.127 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.128 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5). | |
| 2017-06-23 17:03:16.137 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.137 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378). | |
| 2017-06-23 17:03:16.137 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.137 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d). | |
| 2017-06-23 17:03:16.139 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) [CANCELED] | |
| 2017-06-23 17:03:16.141 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) [CANCELED] | |
| 2017-06-23 17:03:16.141 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) [CANCELED] | |
| 2017-06-23 17:03:16.142 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) [CANCELED] | |
| 2017-06-23 17:03:16.142 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) [CANCELED] | |
| 2017-06-23 17:03:16.151 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (cf72a4f1906a9509c2df8956e50bd4cf) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.151 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (de0ca8d35dfae1bd6fd14fc628bcc56d) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.153 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (faa8faf188f8e1bcc455d16854189378) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.154 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (70744a96366789923f77ae4bc035a5a5) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.154 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (2d0ac8997e91fffc16fe4e09d918f9a3) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.187 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-24bca7b0-47fc-4f06-bcc2-ae484fc76cc5/job-05bace2ff9d97b50751a12a8b7297d07_op-WindowOperator_8_0_uuid-e407b528-a913-4191-95c6-18e8191a0b0c | |
| 2017-06-23 17:03:16.191 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.191 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675). | |
| 2017-06-23 17:03:16.191 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) [CANCELED] | |
| 2017-06-23 17:03:16.197 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (32ed163f05f8113b5f3c2e78d9c03675) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.233 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-24bca7b0-47fc-4f06-bcc2-ae484fc76cc5/job-05bace2ff9d97b50751a12a8b7297d07_op-WindowOperator_8_1_uuid-fd5758ea-be63-4e56-8f7a-93a22b0af12b | |
| 2017-06-23 17:03:16.235 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:16.235 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd). | |
| 2017-06-23 17:03:16.235 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) [CANCELED] | |
| 2017-06-23 17:03:16.241 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (d35fea04f5cb26bb59c7e54addd4e6cd) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:18.883 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:18.883 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385). | |
| 2017-06-23 17:03:18.883 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) [CANCELED] | |
| 2017-06-23 17:03:18.890 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (4e8169e27914a633db7f6e7d887d5385) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:03:18.890 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (05bace2ff9d97b50751a12a8b7297d07) switched from state CANCELLING to CANCELED. | |
| 2017-06-23 17:03:18.891 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 05bace2ff9d97b50751a12a8b7297d07 | |
| 2017-06-23 17:03:18.891 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 05bace2ff9d97b50751a12a8b7297d07 | |
| 2017-06-23 17:03:18.891 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:03:18.891 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:03:18.892 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 3 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-fb24c446ecd8' not discarded. | |
| 2017-06-23 17:03:18.892 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 3 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-fb24c446ecd8' not discarded. | |
| 2017-06-23 17:03:18.892 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 4 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e' not discarded. | |
| 2017-06-23 17:03:18.892 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 4 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e' not discarded. | |
| 2017-06-23 17:03:52.008 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:03:52.342 [ip-10-150-96-68] INFO org.apache.flink.runtime.client.JobClient - Checking and uploading JAR files | |
| 2017-06-23 17:03:52.343 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:03:52.678 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job recovers via failover strategy: full graph restart | |
| 2017-06-23 17:03:52.680 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:03:52.680 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:03:52.681 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Starting job from savepoint s3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e () | |
| 2017-06-23 17:03:52.681 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Starting job from savepoint s3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e () | |
| 2017-06-23 17:03:52.682 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.savepoint.SavepointStore - Loading savepoint from s3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e | |
| 2017-06-23 17:03:52.682 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.savepoint.SavepointStore - Loading savepoint from s3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes] | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes] | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes] | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes] | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.761 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes] | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes] | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes] | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes] | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes] | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes] | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes] | |
| 2017-06-23 17:03:54.762 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes] | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes] | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes] | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes] | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes] | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1} | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:03:54.763 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:03:54.764 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Reset the checkpoint ID to 5. | |
| 2017-06-23 17:03:54.764 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Reset the checkpoint ID to 5. | |
| 2017-06-23 17:03:54.764 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Restoring from latest valid checkpoint: Checkpoint 4 @ 0 for e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:54.764 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Restoring from latest valid checkpoint: Checkpoint 4 @ 0 for e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:54.767 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore | |
| 2017-06-23 17:03:54.767 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore | |
| 2017-06-23 17:03:54.767 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (e9720dd241950b6a0222ad03ada0d8ec) switched from state CREATED to RUNNING. | |
| 2017-06-23 17:03:54.768 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.768 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.768 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.768 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.768 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.769 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.770 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.770 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:03:54.770 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.771 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: null source -> Sink: Redshift Transmitter (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.771 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.771 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: S3 File Watcher (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.772 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.772 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.772 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.773 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.773 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.773 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.772 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.772 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) [DEPLOYING] | |
| 2017-06-23 17:03:54.788 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.788 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.789 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.789 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.790 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:03:54.790 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:03:54.772 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) [DEPLOYING]. | |
| 2017-06-23 17:03:54.776 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.776 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) [DEPLOYING] | |
| 2017-06-23 17:03:54.776 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) [DEPLOYING]. | |
| 2017-06-23 17:03:54.819 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.820 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.826 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.813 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.813 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) [DEPLOYING]. | |
| 2017-06-23 17:03:54.813 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) [DEPLOYING]. | |
| 2017-06-23 17:03:54.813 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) [DEPLOYING] | |
| 2017-06-23 17:03:54.814 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.815 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.813 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) [DEPLOYING]. | |
| 2017-06-23 17:03:54.819 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) [DEPLOYING]. | |
| 2017-06-23 17:03:54.821 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.845 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.837 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.836 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.845 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) [DEPLOYING] | |
| 2017-06-23 17:03:54.837 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) [DEPLOYING] | |
| 2017-06-23 17:03:54.846 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) [DEPLOYING] | |
| 2017-06-23 17:03:54.845 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) [DEPLOYING]. | |
| 2017-06-23 17:03:54.837 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) [DEPLOYING]. | |
| 2017-06-23 17:03:54.846 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) [DEPLOYING]. | |
| 2017-06-23 17:03:54.845 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) [DEPLOYING]. | |
| 2017-06-23 17:03:54.838 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) [DEPLOYING]. | |
| 2017-06-23 17:03:54.846 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) [DEPLOYING]. | |
| 2017-06-23 17:03:54.845 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.852 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.852 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) [DEPLOYING] | |
| 2017-06-23 17:03:54.852 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) [DEPLOYING]. | |
| 2017-06-23 17:03:54.853 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) [DEPLOYING]. | |
| 2017-06-23 17:03:54.858 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.853 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:03:54.853 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) [DEPLOYING] | |
| 2017-06-23 17:03:54.853 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) [DEPLOYING]. | |
| 2017-06-23 17:03:54.865 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.854 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) [DEPLOYING]. | |
| 2017-06-23 17:03:54.870 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.874 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.881 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.879 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.884 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.882 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:54.887 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:03:55.158 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.158 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.213 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.213 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.222 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.222 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.245 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.245 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.246 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:03:55.246 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: [org.apache.flink.runtime.state.IncrementalKeyedStateHandle@20e2b220]. | |
| 2017-06-23 17:03:55.248 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.248 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.248 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - restoring instance org.apache.flink.runtime.state.IncrementalKeyedStateHandle@20e2b220, hasExtrayKeys: false | |
| 2017-06-23 17:03:55.249 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes] | |
| 2017-06-23 17:03:55.270 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.270 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.272 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.272 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.272 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:03:55.273 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: [org.apache.flink.runtime.state.IncrementalKeyedStateHandle@412b3110]. | |
| 2017-06-23 17:03:55.273 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - restoring instance org.apache.flink.runtime.state.IncrementalKeyedStateHandle@412b3110, hasExtrayKeys: false | |
| 2017-06-23 17:03:55.273 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes] | |
| 2017-06-23 17:03:55.283 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.283 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.305 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.305 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.328 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes] | |
| 2017-06-23 17:03:55.352 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes] | |
| 2017-06-23 17:03:55.388 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.388 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.522 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.522 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:55.562 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000014.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes] | |
| 2017-06-23 17:03:55.655 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000014.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes] | |
| 2017-06-23 17:03:55.712 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000013.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes] | |
| 2017-06-23 17:03:55.794 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000013.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes] | |
| 2017-06-23 17:03:56.014 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading MANIFEST-000006 -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/5eff213f-c77b-44a5-83c0-2ef29c6bf6b0'} | |
| 2017-06-23 17:03:56.017 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading MANIFEST-000006 -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/2c4e764c-7f10-4c77-9bcd-7e54ddff668d'} | |
| 2017-06-23 17:03:56.017 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading OPTIONS-000010 -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/e2da528b-7f4a-4a05-b78d-7a7c99880799 [8289 bytes] | |
| 2017-06-23 17:03:56.028 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading OPTIONS-000010 -> File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/9d47217c-7063-4929-96d0-a479b7f458ec [8289 bytes] | |
| 2017-06-23 17:03:56.109 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading CURRENT -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/9fa3d2b1-6d29-468f-ace0-9e44c9b4c010'} | |
| 2017-06-23 17:03:56.185 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading CURRENT -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/16aeab6e-219f-45ed-a30f-8bf74a15e80c'} | |
| 2017-06-23 17:03:56.294 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:56.294 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:56.387 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:56.387 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.228 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.228 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.228 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.228 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.303 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.303 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.361 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.361 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.394 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.394 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.434 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:03:57.434 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:24.768 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 5 @ 1498237464768 | |
| 2017-06-23 17:04:24.768 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 5 @ 1498237464768 | |
| 2017-06-23 17:04:24.776 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 6d9bdf6f81477380a1ffdadb1852f0ba of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:24.776 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 6d9bdf6f81477380a1ffdadb1852f0ba of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:34.668 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 2 ms. | |
| 2017-06-23 17:04:34.668 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 2 ms. | |
| 2017-06-23 17:04:34.689 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:04:34.692 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:04:34.692 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:04:35.166 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-14-thread-1,5,Flink Task Threads] took 482 ms. | |
| 2017-06-23 17:04:35.166 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-14-thread-1,5,Flink Task Threads] took 482 ms. | |
| 2017-06-23 17:04:35.172 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task e1951fe0082944d6da41b97577933d11 of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:35.172 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task e1951fe0082944d6da41b97577933d11 of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:35.238 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task d447904f3de0794ba6d40ebb4f42f302 of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:35.238 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task d447904f3de0794ba6d40ebb4f42f302 of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:35.248 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 06c77b71fbbe43c8355e0227d78bd8ae of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:35.248 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 06c77b71fbbe43c8355e0227d78bd8ae of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:35.229 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:04:35.250 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:04:35.250 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:04:36.828 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes] | |
| 2017-06-23 17:04:36.828 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000021.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes] | |
| 2017-06-23 17:04:37.182 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 2487 ms. | |
| 2017-06-23 17:04:37.182 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 2487 ms. | |
| 2017-06-23 17:04:37.361 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes] | |
| 2017-06-23 17:04:37.361 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000021.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes] | |
| 2017-06-23 17:04:37.363 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes] | |
| 2017-06-23 17:04:37.363 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000019.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes] | |
| 2017-06-23 17:04:37.563 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-16-thread-1,5,Flink Task Threads] took 380 ms. | |
| 2017-06-23 17:04:37.563 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-16-thread-1,5,Flink Task Threads] took 380 ms. | |
| 2017-06-23 17:04:37.580 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/cfeeed8e-1efd-463a-8159-8adfca1d8770 [1273 bytes] | |
| 2017-06-23 17:04:37.852 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/1c97ad5d-dbeb-46b8-a13d-e93f420b4e0c [8290 bytes] | |
| 2017-06-23 17:04:37.853 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/91dfcc7f-a84a-4ed2-86fb-887b9fff65b3'} | |
| 2017-06-23 17:04:37.859 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 7739f9af4fa71ff280f3a3f3c8d4ba16 of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:37.859 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 7739f9af4fa71ff280f3a3f3c8d4ba16 of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:37.947 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 2694 ms. | |
| 2017-06-23 17:04:37.953 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes] | |
| 2017-06-23 17:04:37.947 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 2694 ms. | |
| 2017-06-23 17:04:37.954 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000019.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes] | |
| 2017-06-23 17:04:38.325 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-17-thread-1,5,Flink Task Threads] took 345 ms. | |
| 2017-06-23 17:04:38.325 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-17-thread-1,5,Flink Task Threads] took 345 ms. | |
| 2017-06-23 17:04:38.487 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/38d5fc35-1925-497a-ae2d-be78996a1d80 [1273 bytes] | |
| 2017-06-23 17:04:38.909 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/e4967e43-3113-4322-ae8a-90c8b38e4f70 [8290 bytes] | |
| 2017-06-23 17:04:38.909 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/0fec053a-13ce-4a0e-b0db-c21a126eed79'} | |
| 2017-06-23 17:04:38.919 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 982e496d28b4a04a7c4d546927ad630f of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:38.919 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 982e496d28b4a04a7c4d546927ad630f of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:39.221 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-16-thread-1,5,Flink Task Threads] took 1657 ms. | |
| 2017-06-23 17:04:39.221 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-16-thread-1,5,Flink Task Threads] took 1657 ms. | |
| 2017-06-23 17:04:39.235 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 5e72043eec262aa475588e0dd1877f7b of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:39.235 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 5e72043eec262aa475588e0dd1877f7b of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:44.445 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-17-thread-1,5,Flink Task Threads] took 6117 ms. | |
| 2017-06-23 17:04:44.445 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec, asynchronous part) in thread Thread[pool-17-thread-1,5,Flink Task Threads] took 6117 ms. | |
| 2017-06-23 17:04:44.455 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 3b6535cd14bd51f7c03445318ab7bacf of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:44.455 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 5 from task 3b6535cd14bd51f7c03445318ab7bacf of job e9720dd241950b6a0222ad03ada0d8ec. | |
| 2017-06-23 17:04:44.455 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:04:44.455 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:04:44.455 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000019.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes] | |
| 2017-06-23 17:04:44.455 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000019.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes] | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000019.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000019.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000021.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes] | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000021.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes] | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000021.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000021.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000019.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes] | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000019.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes] | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000019.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000019.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000021.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes] | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000021.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes] | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000021.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000021.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes], referenceCount=1} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000019.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes], referenceCount=1}, WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000021.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000021.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000019.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:04:44.456 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/e9f6ffb4-2918-45d8-812e-e74e8153f219 [2209934 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/3f7ac07f-24a3-4fb8-8c32-381a9b021882 [1340725 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/b6e92b0e-e555-47c9-8001-a3df55ffd891 [3624276 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/63e9b6bd-42e6-42cb-b458-1bd781beb593 [3202732 bytes], referenceCount=1}, WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000019.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/457f645e-1321-4487-bfce-982779883703 [5185743 bytes], referenceCount=1}, WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000021.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/f718b280-0cdd-473e-be4a-4664bb9c5782 [2019993 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-2/40206d4f-cc2a-45c2-b923-f417f3c4d1f2 [3621843 bytes], referenceCount=1}, WindowOperator_26_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000021.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/3f5c585d-18fe-491f-9b9c-c3231f4dbcb1 [2024787 bytes], referenceCount=1}, WindowOperator_8_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-1/618f631b-2209-4351-8da2-b596b17dd8ac [1346175 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000014.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-4/bc47fdeb-06fe-4da6-a0ce-c1b0bb5a47a7 [3185291 bytes], referenceCount=1}, WindowOperator_26_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000019.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/e9720dd241950b6a0222ad03ada0d8ec/chk-5/8addef07-35b1-43b1-af57-5a6a08acd785 [5186453 bytes], referenceCount=1}, WindowOperator_8_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/05bace2ff9d97b50751a12a8b7297d07/chk-3/6d80d688-6c7e-4e07-a660-8d811fe84679 [2220835 bytes], referenceCount=1}} | |
| 2017-06-23 17:04:45.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 5 (96281494 bytes in 20264 ms). | |
| 2017-06-23 17:04:45.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 5 (96281494 bytes in 20264 ms). | |
| 2017-06-23 17:04:45.032 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 66444284), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 29767431), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 65917), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:04:45.032 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 66444284), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 29767431), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 65917), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:05:04.142 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (e9720dd241950b6a0222ad03ada0d8ec) switched from state RUNNING to CANCELLING. | |
| 2017-06-23 17:05:04.143 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.143 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.143 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.143 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.144 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.144 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.145 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.145 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.141 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba). | |
| 2017-06-23 17:05:04.141 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.142 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba). | |
| 2017-06-23 17:05:04.142 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11). | |
| 2017-06-23 17:05:04.142 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.142 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11). | |
| 2017-06-23 17:05:04.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b). | |
| 2017-06-23 17:05:04.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b). | |
| 2017-06-23 17:05:04.145 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf). | |
| 2017-06-23 17:05:04.145 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.145 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf). | |
| 2017-06-23 17:05:04.145 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16). | |
| 2017-06-23 17:05:04.145 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16). | |
| 2017-06-23 17:05:04.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f). | |
| 2017-06-23 17:05:04.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f). | |
| 2017-06-23 17:05:04.147 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302). | |
| 2017-06-23 17:05:04.147 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.148 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.148 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba). | |
| 2017-06-23 17:05:04.152 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) [CANCELED] | |
| 2017-06-23 17:05:04.152 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302). | |
| 2017-06-23 17:05:04.156 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae). | |
| 2017-06-23 17:05:04.156 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:05:04.160 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae). | |
| 2017-06-23 17:05:04.167 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.168 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302). | |
| 2017-06-23 17:05:04.168 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) [CANCELED] | |
| 2017-06-23 17:05:04.172 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.172 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b). | |
| 2017-06-23 17:05:04.172 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) [CANCELED] | |
| 2017-06-23 17:05:04.176 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.176 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae). | |
| 2017-06-23 17:05:04.176 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) [CANCELED] | |
| 2017-06-23 17:05:04.181 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (d447904f3de0794ba6d40ebb4f42f302) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.181 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (6d9bdf6f81477380a1ffdadb1852f0ba) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.182 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (5e72043eec262aa475588e0dd1877f7b) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.183 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (06c77b71fbbe43c8355e0227d78bd8ae) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.197 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.197 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf). | |
| 2017-06-23 17:05:04.197 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) [CANCELED] | |
| 2017-06-23 17:05:04.203 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b6535cd14bd51f7c03445318ab7bacf) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.212 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-24bca7b0-47fc-4f06-bcc2-ae484fc76cc5/job-e9720dd241950b6a0222ad03ada0d8ec_op-WindowOperator_26_1_uuid-5c11403e-0894-4ebe-9e81-2666a8ddad6b | |
| 2017-06-23 17:05:04.214 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.214 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f). | |
| 2017-06-23 17:05:04.214 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) [CANCELED] | |
| 2017-06-23 17:05:04.220 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (982e496d28b4a04a7c4d546927ad630f) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.269 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-24bca7b0-47fc-4f06-bcc2-ae484fc76cc5/job-e9720dd241950b6a0222ad03ada0d8ec_op-WindowOperator_26_0_uuid-29d2bf7d-9de2-4ab0-9b96-ac224ce096d6 | |
| 2017-06-23 17:05:04.271 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:04.271 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16). | |
| 2017-06-23 17:05:04.272 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) [CANCELED] | |
| 2017-06-23 17:05:04.278 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (7739f9af4fa71ff280f3a3f3c8d4ba16) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:08.024 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:08.024 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11). | |
| 2017-06-23 17:05:08.025 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) [CANCELED] | |
| 2017-06-23 17:05:08.031 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (e1951fe0082944d6da41b97577933d11) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (e9720dd241950b6a0222ad03ada0d8ec) switched from state CANCELLING to CANCELED. | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job e9720dd241950b6a0222ad03ada0d8ec | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job e9720dd241950b6a0222ad03ada0d8ec | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 4 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e' not discarded. | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 4 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-981050d24a8e' not discarded. | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 5 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-00addb3d694c' not discarded. | |
| 2017-06-23 17:05:08.032 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 5 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-00addb3d694c' not discarded. | |
| 2017-06-23 17:05:12.396 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:05:12.869 [ip-10-150-96-68] INFO org.apache.flink.runtime.client.JobClient - Checking and uploading JAR files | |
| 2017-06-23 17:05:12.870 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:05:13.299 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job recovers via failover strategy: full graph restart | |
| 2017-06-23 17:05:13.301 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:05:13.301 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:05:13.302 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (2a4bf3e1e3a65bfd5bdb7197399e8dae) switched from state CREATED to RUNNING. | |
| 2017-06-23 17:05:13.303 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.307 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.309 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.307 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.309 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: null source -> Sink: Redshift Transmitter (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.308 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.308 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.308 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.308 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.308 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:05:13.310 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.310 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: S3 File Watcher (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.311 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.311 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.312 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.312 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.313 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.313 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.310 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.313 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) [DEPLOYING] | |
| 2017-06-23 17:05:13.313 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.316 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) [DEPLOYING] | |
| 2017-06-23 17:05:13.313 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) [DEPLOYING]. | |
| 2017-06-23 17:05:13.316 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) [DEPLOYING]. | |
| 2017-06-23 17:05:13.320 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.320 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.320 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.320 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.321 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:05:13.321 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:05:13.324 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.324 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.328 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) [DEPLOYING] | |
| 2017-06-23 17:05:13.324 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) [DEPLOYING] | |
| 2017-06-23 17:05:13.328 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) [DEPLOYING]. | |
| 2017-06-23 17:05:13.324 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) [DEPLOYING]. | |
| 2017-06-23 17:05:13.336 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) [DEPLOYING]. | |
| 2017-06-23 17:05:13.333 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.331 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.336 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) [DEPLOYING]. | |
| 2017-06-23 17:05:13.336 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) [DEPLOYING] | |
| 2017-06-23 17:05:13.350 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.333 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) [DEPLOYING] | |
| 2017-06-23 17:05:13.343 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) [DEPLOYING]. | |
| 2017-06-23 17:05:13.343 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) [DEPLOYING]. | |
| 2017-06-23 17:05:13.344 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.348 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.336 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) [DEPLOYING]. | |
| 2017-06-23 17:05:13.333 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) [DEPLOYING]. | |
| 2017-06-23 17:05:13.348 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.348 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.344 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) [DEPLOYING]. | |
| 2017-06-23 17:05:13.343 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) [DEPLOYING]. | |
| 2017-06-23 17:05:13.349 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.356 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.356 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.353 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.355 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.356 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) [DEPLOYING] | |
| 2017-06-23 17:05:13.356 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) [DEPLOYING]. | |
| 2017-06-23 17:05:13.358 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.358 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.360 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.356 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) [DEPLOYING]. | |
| 2017-06-23 17:05:13.357 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.365 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.383 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:05:13.393 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) [DEPLOYING] | |
| 2017-06-23 17:05:13.393 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) [DEPLOYING]. | |
| 2017-06-23 17:05:13.393 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) [DEPLOYING]. | |
| 2017-06-23 17:05:13.408 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.400 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:05:13.698 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.698 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.722 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.722 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.761 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.761 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.763 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.763 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.763 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:05:13.764 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: null. | |
| 2017-06-23 17:05:13.781 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.781 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.798 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.798 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.799 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.799 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.809 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.809 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.817 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.817 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.817 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:05:13.817 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: null. | |
| 2017-06-23 17:05:13.823 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.823 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.829 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.829 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.846 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.846 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.857 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.857 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.887 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.887 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.892 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.892 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.932 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.932 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.969 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.969 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.994 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:13.994 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:14.019 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:14.019 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:43.303 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 1 @ 1498237543303 | |
| 2017-06-23 17:05:43.303 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 1 @ 1498237543303 | |
| 2017-06-23 17:05:43.317 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 68e09cabc8c52b1d14b37b9c132cb4c2 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:43.317 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 68e09cabc8c52b1d14b37b9c132cb4c2 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:47.656 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 8 ms. | |
| 2017-06-23 17:05:47.656 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 8 ms. | |
| 2017-06-23 17:05:47.667 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:05:47.704 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 28 ms. | |
| 2017-06-23 17:05:47.704 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 28 ms. | |
| 2017-06-23 17:05:47.809 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:05:47.836 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task ba26e765c9e09a827fd878bb193c1545 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:47.836 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task ba26e765c9e09a827fd878bb193c1545 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:47.836 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 51fd1783551ea45fa2a3f40d6ea4cdc0 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:47.836 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 51fd1783551ea45fa2a3f40d6ea4cdc0 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:47.903 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 67 ms. | |
| 2017-06-23 17:05:47.903 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 67 ms. | |
| 2017-06-23 17:05:48.161 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-22-thread-1,5,Flink Task Threads] took 504 ms. | |
| 2017-06-23 17:05:48.161 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-22-thread-1,5,Flink Task Threads] took 504 ms. | |
| 2017-06-23 17:05:48.167 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task a1f5deae27a40d11e1bf24554ca7adc3 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:48.167 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task a1f5deae27a40d11e1bf24554ca7adc3 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:48.353 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 648 ms. | |
| 2017-06-23 17:05:48.353 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 648 ms. | |
| 2017-06-23 17:05:48.479 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 566 ms. | |
| 2017-06-23 17:05:48.479 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 566 ms. | |
| 2017-06-23 17:05:49.022 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 668 ms. | |
| 2017-06-23 17:05:49.022 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 668 ms. | |
| 2017-06-23 17:05:49.241 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes] | |
| 2017-06-23 17:05:49.242 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes] | |
| 2017-06-23 17:05:49.242 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/aaefd9d4-a943-46fc-af8c-99f1f5d6d103'} | |
| 2017-06-23 17:05:49.542 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/afac1b57-dd58-461f-bc71-03f726c809d3 [8290 bytes] | |
| 2017-06-23 17:05:49.543 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/e10c7d9d-02a5-494b-8460-c61f763e0366'} | |
| 2017-06-23 17:05:49.551 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 0f33835d7b5dfe2fe6f250151b54a06c of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:49.551 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 0f33835d7b5dfe2fe6f250151b54a06c of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:49.667 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 644 ms. | |
| 2017-06-23 17:05:49.667 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 644 ms. | |
| 2017-06-23 17:05:49.676 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 3b312e39b47b3af96de15fb50e1fc4cb of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:49.676 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 3b312e39b47b3af96de15fb50e1fc4cb of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:52.130 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes] | |
| 2017-06-23 17:05:52.130 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes] | |
| 2017-06-23 17:05:52.130 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/b9a5f49a-8b30-4f64-bab0-780b311aabc5'} | |
| 2017-06-23 17:05:52.531 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/18031e0a-3678-43f3-bf59-b35eb22991d6 [8290 bytes] | |
| 2017-06-23 17:05:52.531 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa3fdbfd-438c-46c4-a6af-92123bf07549'} | |
| 2017-06-23 17:05:52.538 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 13abc21bafff040142fa7f0dc76df036 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:52.538 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 13abc21bafff040142fa7f0dc76df036 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:54.334 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 5854 ms. | |
| 2017-06-23 17:05:54.334 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 5854 ms. | |
| 2017-06-23 17:05:55.030 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 695 ms. | |
| 2017-06-23 17:05:55.030 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 695 ms. | |
| 2017-06-23 17:05:55.039 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task fb78a0551b3b3590fbfdf58aa2aae549 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:55.039 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task fb78a0551b3b3590fbfdf58aa2aae549 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes] | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes] | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1} | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1} | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes] | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes] | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1} | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1} | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:05:55.040 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:05:55.290 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 1 (25619355 bytes in 11987 ms). | |
| 2017-06-23 17:05:55.290 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 1 (25619355 bytes in 11987 ms). | |
| 2017-06-23 17:05:55.291 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 12480097), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 6107988), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 7027408), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:05:55.291 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 12480097), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 6107988), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 7027408), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:06:19.290 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 2 @ 1498237579290 | |
| 2017-06-23 17:06:19.290 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 2 @ 1498237579290 | |
| 2017-06-23 17:06:19.297 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 68e09cabc8c52b1d14b37b9c132cb4c2 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:19.297 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 68e09cabc8c52b1d14b37b9c132cb4c2 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:22.383 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:06:22.383 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:06:22.386 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:06:22.417 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task ba26e765c9e09a827fd878bb193c1545 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:22.417 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task ba26e765c9e09a827fd878bb193c1545 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:22.418 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 51fd1783551ea45fa2a3f40d6ea4cdc0 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:22.418 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 51fd1783551ea45fa2a3f40d6ea4cdc0 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:22.412 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:06:22.416 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 26 ms. | |
| 2017-06-23 17:06:22.416 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 26 ms. | |
| 2017-06-23 17:06:22.450 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 26 ms. | |
| 2017-06-23 17:06:22.450 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 26 ms. | |
| 2017-06-23 17:06:22.843 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-22-thread-1,5,Flink Task Threads] took 460 ms. | |
| 2017-06-23 17:06:22.843 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-22-thread-1,5,Flink Task Threads] took 460 ms. | |
| 2017-06-23 17:06:22.848 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task a1f5deae27a40d11e1bf24554ca7adc3 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:22.848 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task a1f5deae27a40d11e1bf24554ca7adc3 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:23.591 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1139 ms. | |
| 2017-06-23 17:06:23.591 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1139 ms. | |
| 2017-06-23 17:06:23.613 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1196 ms. | |
| 2017-06-23 17:06:23.613 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1196 ms. | |
| 2017-06-23 17:06:23.968 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes] | |
| 2017-06-23 17:06:23.969 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes] | |
| 2017-06-23 17:06:23.969 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:06:23.969 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/23944752-bb6c-4900-8f14-5044365ff966'} | |
| 2017-06-23 17:06:24.032 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 441 ms. | |
| 2017-06-23 17:06:24.032 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 441 ms. | |
| 2017-06-23 17:06:24.056 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 442 ms. | |
| 2017-06-23 17:06:24.056 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 442 ms. | |
| 2017-06-23 17:06:24.101 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes] | |
| 2017-06-23 17:06:24.101 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes] | |
| 2017-06-23 17:06:24.101 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:06:24.101 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/54421ec0-a0fe-418d-92cd-a9db88d92f5a'} | |
| 2017-06-23 17:06:24.194 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/80951c19-a8d7-4122-9113-52db3867cf77 [8290 bytes] | |
| 2017-06-23 17:06:24.194 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/e6c39816-0944-4a9a-9fbc-0d9638d41802'} | |
| 2017-06-23 17:06:24.201 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 13abc21bafff040142fa7f0dc76df036 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:24.201 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 13abc21bafff040142fa7f0dc76df036 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:24.433 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/e054aa2c-e8bf-4c47-b4c1-31dbb3f5b6b3 [8290 bytes] | |
| 2017-06-23 17:06:24.434 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/c5902028-d468-41c7-b5f7-2f0ce990ead8'} | |
| 2017-06-23 17:06:24.440 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 0f33835d7b5dfe2fe6f250151b54a06c of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:24.440 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 0f33835d7b5dfe2fe6f250151b54a06c of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:24.888 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 856 ms. | |
| 2017-06-23 17:06:24.888 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 856 ms. | |
| 2017-06-23 17:06:24.897 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 3b312e39b47b3af96de15fb50e1fc4cb of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:24.897 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 3b312e39b47b3af96de15fb50e1fc4cb of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:25.485 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 1417 ms. | |
| 2017-06-23 17:06:25.485 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 1417 ms. | |
| 2017-06-23 17:06:25.494 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task fb78a0551b3b3590fbfdf58aa2aae549 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:25.494 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task fb78a0551b3b3590fbfdf58aa2aae549 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:25.494 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:06:25.494 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@740e193e | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@740e193e | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2} | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2} | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes] | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes] | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1} | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1} | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@7651dcdc | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@7651dcdc | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2} | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2} | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes] | |
| 2017-06-23 17:06:25.495 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes] | |
| 2017-06-23 17:06:25.496 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1} | |
| 2017-06-23 17:06:25.496 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1} | |
| 2017-06-23 17:06:25.496 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2}} | |
| 2017-06-23 17:06:25.496 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2}} | |
| 2017-06-23 17:06:25.726 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 2 (56213135 bytes in 6436 ms). | |
| 2017-06-23 17:06:25.726 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 2 (56213135 bytes in 6436 ms). | |
| 2017-06-23 17:06:25.726 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 33234555), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 16984637), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 5990081), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:06:25.726 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 33234555), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 16984637), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 5990081), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:06:49.725 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237609725 | |
| 2017-06-23 17:06:49.725 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237609725 | |
| 2017-06-23 17:06:49.728 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 68e09cabc8c52b1d14b37b9c132cb4c2 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:49.728 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 68e09cabc8c52b1d14b37b9c132cb4c2 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:50.165 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:06:50.165 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 1 ms. | |
| 2017-06-23 17:06:50.209 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task ba26e765c9e09a827fd878bb193c1545 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:50.209 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task ba26e765c9e09a827fd878bb193c1545 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:50.209 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 51fd1783551ea45fa2a3f40d6ea4cdc0 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:50.209 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 51fd1783551ea45fa2a3f40d6ea4cdc0 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:50.201 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:06:50.203 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:06:50.253 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 38 ms. | |
| 2017-06-23 17:06:50.253 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 38 ms. | |
| 2017-06-23 17:06:50.257 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 44 ms. | |
| 2017-06-23 17:06:50.257 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 44 ms. | |
| 2017-06-23 17:06:50.990 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-22-thread-1,5,Flink Task Threads] took 825 ms. | |
| 2017-06-23 17:06:50.990 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-22-thread-1,5,Flink Task Threads] took 825 ms. | |
| 2017-06-23 17:06:50.996 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task a1f5deae27a40d11e1bf24554ca7adc3 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:50.996 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task a1f5deae27a40d11e1bf24554ca7adc3 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:51.834 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1579 ms. | |
| 2017-06-23 17:06:51.834 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1579 ms. | |
| 2017-06-23 17:06:51.838 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1580 ms. | |
| 2017-06-23 17:06:51.838 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1580 ms. | |
| 2017-06-23 17:06:52.181 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes] | |
| 2017-06-23 17:06:52.182 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000013.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes] | |
| 2017-06-23 17:06:52.182 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:06:52.182 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:06:52.182 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/eff08053-e17b-4b4a-8e6c-a221ddcfc89e'} | |
| 2017-06-23 17:06:52.218 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes] | |
| 2017-06-23 17:06:52.218 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000013.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes] | |
| 2017-06-23 17:06:52.218 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:06:52.219 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:06:52.233 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/16dd0b86-2e3c-4f03-ba50-4561d806bc7e'} | |
| 2017-06-23 17:06:52.413 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/5491adf7-be5e-4525-be85-bab746a67465 [8290 bytes] | |
| 2017-06-23 17:06:52.414 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/b4183068-337b-4a9d-ab58-1de3efb6b128'} | |
| 2017-06-23 17:06:52.420 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 0f33835d7b5dfe2fe6f250151b54a06c of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:52.420 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 0f33835d7b5dfe2fe6f250151b54a06c of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:52.432 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 597 ms. | |
| 2017-06-23 17:06:52.432 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 597 ms. | |
| 2017-06-23 17:06:52.462 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/1b7e9c81-ce31-4d15-ab24-0f9cefc782f3 [8290 bytes] | |
| 2017-06-23 17:06:52.463 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/25dd05af-b7ec-4825-8e24-6aaf33fbd596'} | |
| 2017-06-23 17:06:52.469 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 13abc21bafff040142fa7f0dc76df036 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:52.469 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 13abc21bafff040142fa7f0dc76df036 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:52.941 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 1103 ms. | |
| 2017-06-23 17:06:52.941 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 1103 ms. | |
| 2017-06-23 17:06:54.391 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 1448 ms. | |
| 2017-06-23 17:06:54.391 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-26-thread-1,5,Flink Task Threads] took 1448 ms. | |
| 2017-06-23 17:06:54.402 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 3b312e39b47b3af96de15fb50e1fc4cb of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:54.402 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 3b312e39b47b3af96de15fb50e1fc4cb of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:54.792 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 2359 ms. | |
| 2017-06-23 17:06:54.792 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae, asynchronous part) in thread Thread[pool-23-thread-1,5,Flink Task Threads] took 2359 ms. | |
| 2017-06-23 17:06:54.802 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task fb78a0551b3b3590fbfdf58aa2aae549 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:54.802 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task fb78a0551b3b3590fbfdf58aa2aae549 of job 2a4bf3e1e3a65bfd5bdb7197399e8dae. | |
| 2017-06-23 17:06:54.802 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2}} | |
| 2017-06-23 17:06:54.802 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2}} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@2008a355 | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@2008a355 | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=3} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=3} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@2437238c | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@2437238c | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=2} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=2} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes] | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes] | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes], referenceCount=1} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes], referenceCount=1} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@6135a96e | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@6135a96e | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=3} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=3} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3569851c | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3569851c | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=2} | |
| 2017-06-23 17:06:54.803 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=2} | |
| 2017-06-23 17:06:54.804 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes] | |
| 2017-06-23 17:06:54.804 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes] | |
| 2017-06-23 17:06:54.804 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes], referenceCount=1} | |
| 2017-06-23 17:06:54.804 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes], referenceCount=1} | |
| 2017-06-23 17:06:54.804 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=2}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=3}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=2}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=3}} | |
| 2017-06-23 17:06:54.804 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/78c5117b-baca-4b9e-90a2-a97eea54f25c [2222211 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=2}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=3}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000013.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-3/bc61def9-15db-4b52-ab15-d5c0c0c69c8f [2219930 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=2}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=3}} | |
| 2017-06-23 17:06:57.147 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:06:57.147 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:06:57.147 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2} | |
| 2017-06-23 17:06:57.147 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=2} | |
| 2017-06-23 17:06:57.537 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst | |
| 2017-06-23 17:06:57.537 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst | |
| 2017-06-23 17:06:57.537 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2} | |
| 2017-06-23 17:06:57.537 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - UNRegistration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result: Result{reference=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=2} | |
| 2017-06-23 17:06:58.427 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 3 (74709441 bytes in 5284 ms). | |
| 2017-06-23 17:06:58.427 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 3 (74709441 bytes in 5284 ms). | |
| 2017-06-23 17:06:58.427 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 49705655), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 16131178), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 8868746), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:06:58.427 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 49705655), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 16131178), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 8868746), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 2, maxParallelism: 128, sub task states: 2, total size (bytes): 0) | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (2a4bf3e1e3a65bfd5bdb7197399e8dae) switched from state RUNNING to CANCELLING. | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.220 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2). | |
| 2017-06-23 17:07:01.220 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.221 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2). | |
| 2017-06-23 17:07:01.221 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3). | |
| 2017-06-23 17:07:01.221 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3). | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549). | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.222 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549). | |
| 2017-06-23 17:07:01.223 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb). | |
| 2017-06-23 17:07:01.228 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.228 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.230 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb). | |
| 2017-06-23 17:07:01.236 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2). | |
| 2017-06-23 17:07:01.230 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036). | |
| 2017-06-23 17:07:01.230 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.237 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) [CANCELED] | |
| 2017-06-23 17:07:01.237 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036). | |
| 2017-06-23 17:07:01.238 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c). | |
| 2017-06-23 17:07:01.238 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.239 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c). | |
| 2017-06-23 17:07:01.239 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545). | |
| 2017-06-23 17:07:01.239 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.240 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545). | |
| 2017-06-23 17:07:01.244 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0). | |
| 2017-06-23 17:07:01.244 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:07:01.244 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0). | |
| 2017-06-23 17:07:01.257 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (68e09cabc8c52b1d14b37b9c132cb4c2) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.261 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.261 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545). | |
| 2017-06-23 17:07:01.261 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) [CANCELED] | |
| 2017-06-23 17:07:01.268 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (ba26e765c9e09a827fd878bb193c1545) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.268 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.268 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0). | |
| 2017-06-23 17:07:01.268 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) [CANCELED] | |
| 2017-06-23 17:07:01.276 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (51fd1783551ea45fa2a3f40d6ea4cdc0) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.279 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.279 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb). | |
| 2017-06-23 17:07:01.279 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) [CANCELED] | |
| 2017-06-23 17:07:01.285 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (3b312e39b47b3af96de15fb50e1fc4cb) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.304 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.304 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549). | |
| 2017-06-23 17:07:01.305 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) [CANCELED] | |
| 2017-06-23 17:07:01.310 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (fb78a0551b3b3590fbfdf58aa2aae549) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.311 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-2df52e4f-5d79-452b-8c57-b9d3e4561cf1/job-2a4bf3e1e3a65bfd5bdb7197399e8dae_op-WindowOperator_44_1_uuid-8ae76b0a-cf15-4d5c-952d-faeb18300a57 | |
| 2017-06-23 17:07:01.314 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.314 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c). | |
| 2017-06-23 17:07:01.314 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) [CANCELED] | |
| 2017-06-23 17:07:01.320 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (0f33835d7b5dfe2fe6f250151b54a06c) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.349 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-24bca7b0-47fc-4f06-bcc2-ae484fc76cc5/job-2a4bf3e1e3a65bfd5bdb7197399e8dae_op-WindowOperator_44_0_uuid-ad336035-93f5-4567-b682-5f50721e5a49 | |
| 2017-06-23 17:07:01.352 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:01.352 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036). | |
| 2017-06-23 17:07:01.352 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) [CANCELED] | |
| 2017-06-23 17:07:01.358 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (13abc21bafff040142fa7f0dc76df036) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:13.125 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:13.125 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3). | |
| 2017-06-23 17:07:13.127 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) [CANCELED] | |
| 2017-06-23 17:07:13.133 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (a1f5deae27a40d11e1bf24554ca7adc3) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:07:13.133 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (2a4bf3e1e3a65bfd5bdb7197399e8dae) switched from state CANCELLING to CANCELED. | |
| 2017-06-23 17:07:13.133 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 2a4bf3e1e3a65bfd5bdb7197399e8dae | |
| 2017-06-23 17:07:13.133 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 2a4bf3e1e3a65bfd5bdb7197399e8dae | |
| 2017-06-23 17:07:13.133 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:07:13.133 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:07:13.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 2 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e' not discarded. | |
| 2017-06-23 17:07:13.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 2 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e' not discarded. | |
| 2017-06-23 17:07:13.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 3 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-a59c2079f016' not discarded. | |
| 2017-06-23 17:07:13.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 3 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-a59c2079f016' not discarded. | |
| 2017-06-23 17:07:18.556 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:07:18.893 [ip-10-150-96-68] INFO org.apache.flink.runtime.client.JobClient - Checking and uploading JAR files | |
| 2017-06-23 17:07:18.895 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:07:19.221 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job recovers via failover strategy: full graph restart | |
| 2017-06-23 17:07:19.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:07:19.223 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:07:19.224 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Starting job from savepoint s3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e () | |
| 2017-06-23 17:07:19.224 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Starting job from savepoint s3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e () | |
| 2017-06-23 17:07:19.224 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.savepoint.SavepointStore - Loading savepoint from s3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e | |
| 2017-06-23 17:07:19.224 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.savepoint.SavepointStore - Loading savepoint from s3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e | |
| 2017-06-23 17:07:19.549 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:07:19.549 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes] | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes] | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes] | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes] | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes] | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes] | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes] | |
| 2017-06-23 17:07:19.550 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes] | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1} | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Reset the checkpoint ID to 3. | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Reset the checkpoint ID to 3. | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Restoring from latest valid checkpoint: Checkpoint 2 @ 0 for 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:19.551 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Restoring from latest valid checkpoint: Checkpoint 2 @ 0 for 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:19.552 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore | |
| 2017-06-23 17:07:19.552 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore | |
| 2017-06-23 17:07:19.552 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (5d6607746c6f40f45be3144ca4df4d93) switched from state CREATED to RUNNING. | |
| 2017-06-23 17:07:19.552 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.552 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.553 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.553 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.553 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.553 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.553 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.553 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:07:19.554 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.554 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: null source -> Sink: Redshift Transmitter (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.554 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.555 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: S3 File Watcher (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.555 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.555 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.557 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.557 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.555 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.556 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) [DEPLOYING] | |
| 2017-06-23 17:07:19.556 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) [DEPLOYING]. | |
| 2017-06-23 17:07:19.557 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.557 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.558 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.558 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.558 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.558 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (1/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.560 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:07:19.560 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (2/2) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:07:19.558 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.567 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) [DEPLOYING]. | |
| 2017-06-23 17:07:19.568 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) [DEPLOYING] | |
| 2017-06-23 17:07:19.568 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) [DEPLOYING]. | |
| 2017-06-23 17:07:19.569 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) [DEPLOYING]. | |
| 2017-06-23 17:07:19.568 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.569 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) [DEPLOYING] | |
| 2017-06-23 17:07:19.570 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.572 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.576 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.578 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.579 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.581 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.582 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:07:19.570 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) [DEPLOYING]. | |
| 2017-06-23 17:07:19.583 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) [DEPLOYING] | |
| 2017-06-23 17:07:19.581 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) [DEPLOYING] | |
| 2017-06-23 17:07:19.576 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) [DEPLOYING] | |
| 2017-06-23 17:07:19.578 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) [DEPLOYING] | |
| 2017-06-23 17:07:19.580 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) [DEPLOYING] | |
| 2017-06-23 17:07:19.570 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) [DEPLOYING]. | |
| 2017-06-23 17:07:19.583 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) [DEPLOYING]. | |
| 2017-06-23 17:07:19.581 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) [DEPLOYING]. | |
| 2017-06-23 17:07:19.576 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) [DEPLOYING]. | |
| 2017-06-23 17:07:19.578 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) [DEPLOYING]. | |
| 2017-06-23 17:07:19.580 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) [DEPLOYING]. | |
| 2017-06-23 17:07:19.576 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.583 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) [DEPLOYING]. | |
| 2017-06-23 17:07:19.581 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) [DEPLOYING]. | |
| 2017-06-23 17:07:19.577 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) [DEPLOYING]. | |
| 2017-06-23 17:07:19.578 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) [DEPLOYING]. | |
| 2017-06-23 17:07:19.580 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) [DEPLOYING]. | |
| 2017-06-23 17:07:19.591 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.591 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.592 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.593 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.593 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.594 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.587 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.587 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.589 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.589 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.595 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.600 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:19.593 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:07:20.152 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.152 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.153 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.153 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.180 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.180 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.300 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.300 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.307 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.307 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.312 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:07:20.312 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: [org.apache.flink.runtime.state.IncrementalKeyedStateHandle@dcce86ae]. | |
| 2017-06-23 17:07:20.312 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - restoring instance org.apache.flink.runtime.state.IncrementalKeyedStateHandle@dcce86ae, hasExtrayKeys: false | |
| 2017-06-23 17:07:20.312 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes] | |
| 2017-06-23 17:07:20.333 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.333 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.339 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.339 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.359 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.359 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.401 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes] | |
| 2017-06-23 17:07:20.443 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.443 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.452 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.452 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.453 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:07:20.453 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: [org.apache.flink.runtime.state.IncrementalKeyedStateHandle@95a7f87]. | |
| 2017-06-23 17:07:20.453 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - restoring instance org.apache.flink.runtime.state.IncrementalKeyedStateHandle@95a7f87, hasExtrayKeys: false | |
| 2017-06-23 17:07:20.453 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes] | |
| 2017-06-23 17:07:20.472 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.472 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.501 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading MANIFEST-000006 -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/54421ec0-a0fe-418d-92cd-a9db88d92f5a'} | |
| 2017-06-23 17:07:20.502 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading OPTIONS-000010 -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/e054aa2c-e8bf-4c47-b4c1-31dbb3f5b6b3 [8290 bytes] | |
| 2017-06-23 17:07:20.506 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.506 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.514 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes] | |
| 2017-06-23 17:07:20.538 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading CURRENT -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/c5902028-d468-41c7-b5f7-2f0ce990ead8'} | |
| 2017-06-23 17:07:20.689 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.689 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.740 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading MANIFEST-000006 -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/23944752-bb6c-4900-8f14-5044365ff966'} | |
| 2017-06-23 17:07:20.741 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading OPTIONS-000010 -> File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/80951c19-a8d7-4122-9113-52db3867cf77 [8290 bytes] | |
| 2017-06-23 17:07:20.785 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading CURRENT -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/e6c39816-0944-4a9a-9fbc-0d9638d41802'} | |
| 2017-06-23 17:07:20.953 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:20.953 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:21.293 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:21.293 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:21.334 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:21.334 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:22.264 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:22.264 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:22.294 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:22.294 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:22.327 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:22.327 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:49.552 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237669552 | |
| 2017-06-23 17:07:49.552 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237669552 | |
| 2017-06-23 17:07:49.567 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task fea30a9bbaadb42ded86af0b736e1384 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:49.567 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task fea30a9bbaadb42ded86af0b736e1384 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:52.622 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 2 ms. | |
| 2017-06-23 17:07:52.622 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 2 ms. | |
| 2017-06-23 17:07:52.639 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 1 | |
| 2017-06-23 17:07:52.660 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:07:52.668 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 468379a5ad983d0529755d9b3165e97d of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:52.668 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 468379a5ad983d0529755d9b3165e97d of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:52.674 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 271b2171285e15491396af991501e6c3 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:52.674 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 271b2171285e15491396af991501e6c3 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:52.687 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 19 ms. | |
| 2017-06-23 17:07:52.687 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 19 ms. | |
| 2017-06-23 17:07:52.691 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 40 ms. | |
| 2017-06-23 17:07:52.691 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 40 ms. | |
| 2017-06-23 17:07:53.217 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-31-thread-1,5,Flink Task Threads] took 590 ms. | |
| 2017-06-23 17:07:53.217 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-31-thread-1,5,Flink Task Threads] took 590 ms. | |
| 2017-06-23 17:07:53.229 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 180c1cb27ee98db1589be6ef216de811 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:53.229 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 180c1cb27ee98db1589be6ef216de811 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:54.098 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1405 ms. | |
| 2017-06-23 17:07:54.098 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2),5,Flink Task Threads] took 1405 ms. | |
| 2017-06-23 17:07:54.254 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1565 ms. | |
| 2017-06-23 17:07:54.254 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2),5,Flink Task Threads] took 1565 ms. | |
| 2017-06-23 17:07:54.298 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/8f80d77d-e0aa-4e54-8296-f69486a97e28 [2009487 bytes] | |
| 2017-06-23 17:07:54.298 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000017.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/8f80d77d-e0aa-4e54-8296-f69486a97e28 [2009487 bytes] | |
| 2017-06-23 17:07:54.298 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:07:54.298 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:07:54.299 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/3b324fd0-ef32-4f7e-9aab-a6bc6bc14a54'} | |
| 2017-06-23 17:07:54.373 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/9538d48a-7a11-4880-8132-3ef8bcde789d [2006594 bytes] | |
| 2017-06-23 17:07:54.385 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000017.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/9538d48a-7a11-4880-8132-3ef8bcde789d [2006594 bytes] | |
| 2017-06-23 17:07:54.385 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:07:54.385 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:07:54.385 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/b7992c85-9d09-45fc-9ccf-7c8e2a24a323'} | |
| 2017-06-23 17:07:54.515 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/494f5ad3-ed5d-4da3-812e-49149ff45bb0 [8290 bytes] | |
| 2017-06-23 17:07:54.520 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/ff0901c7-106b-4d6c-a70c-6ede13bc0b2d'} | |
| 2017-06-23 17:07:54.526 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 001562689e50727a0b2ec197d92e4077 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:54.526 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 001562689e50727a0b2ec197d92e4077 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:54.779 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-34-thread-1,5,Flink Task Threads] took 680 ms. | |
| 2017-06-23 17:07:54.779 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-34-thread-1,5,Flink Task Threads] took 680 ms. | |
| 2017-06-23 17:07:54.858 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-32-thread-1,5,Flink Task Threads] took 602 ms. | |
| 2017-06-23 17:07:54.858 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-32-thread-1,5,Flink Task Threads] took 602 ms. | |
| 2017-06-23 17:07:54.973 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/c04bef81-54fd-462a-8cbb-887a91605857 [8290 bytes] | |
| 2017-06-23 17:07:54.980 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93/chk-3/8791b049-f998-440e-8c17-46b3b0eb0bf3'} | |
| 2017-06-23 17:07:54.992 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2f67f032e91e39a44bd242612ad8901e of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:54.992 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2f67f032e91e39a44bd242612ad8901e of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:56.278 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-34-thread-1,5,Flink Task Threads] took 1498 ms. | |
| 2017-06-23 17:07:56.278 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-34-thread-1,5,Flink Task Threads] took 1498 ms. | |
| 2017-06-23 17:07:56.311 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task f6b7da51c78a783c40d2f831e4041676 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:56.311 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task f6b7da51c78a783c40d2f831e4041676 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:56.304 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-32-thread-1,5,Flink Task Threads] took 1445 ms. | |
| 2017-06-23 17:07:56.304 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/5d6607746c6f40f45be3144ca4df4d93, asynchronous part) in thread Thread[pool-32-thread-1,5,Flink Task Threads] took 1445 ms. | |
| 2017-06-23 17:07:56.314 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2da2dd9253bfecf0532d1d260f7fc678 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:56.314 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2da2dd9253bfecf0532d1d260f7fc678 of job 5d6607746c6f40f45be3144ca4df4d93. | |
| 2017-06-23 17:07:56.314 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:07:56.314 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/826f4e5c-4590-4c8b-912c-ba04aff88482 [3575657 bytes], referenceCount=1}, WindowOperator_44_1-KeyGroupRange{startKeyGroup=64, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/92c6d08b-1848-4322-a790-4aef6e1936a0 [1340143 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-2/195b610a-0509-49b3-b53e-64f73b95af27 [3566947 bytes], referenceCount=1}, WindowOperator_44_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/2a4bf3e1e3a65bfd5bdb7197399e8dae/chk-1/fa7ba575-3e22-41cd-87db-21579940c0b7 [1345530 bytes], referenceCount=1}} | |
| 2017-06-23 17:07:56.315 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_62_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@40717c66 | |
| 2017-06-23 17:07:56.315 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_62_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@40717c66 | |
| 2017-06-23 17:09:06.735 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (5d6607746c6f40f45be3144ca4df4d93) switched from state RUNNING to CANCELLING. | |
| 2017-06-23 17:09:06.737 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.737 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.737 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.736 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384). | |
| 2017-06-23 17:09:06.736 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.737 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384). | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811). | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811). | |
| 2017-06-23 17:09:06.738 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678). | |
| 2017-06-23 17:09:06.739 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.740 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678). | |
| 2017-06-23 17:09:06.740 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.740 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384). | |
| 2017-06-23 17:09:06.740 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676). | |
| 2017-06-23 17:09:06.741 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) [CANCELED] | |
| 2017-06-23 17:09:06.741 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.741 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676). | |
| 2017-06-23 17:09:06.747 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077). | |
| 2017-06-23 17:09:06.748 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.748 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077). | |
| 2017-06-23 17:09:06.748 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e). | |
| 2017-06-23 17:09:06.748 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.749 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e). | |
| 2017-06-23 17:09:06.750 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d). | |
| 2017-06-23 17:09:06.750 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.750 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d). | |
| 2017-06-23 17:09:06.750 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3). | |
| 2017-06-23 17:09:06.750 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:09:06.750 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3). | |
| 2017-06-23 17:09:06.757 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.757 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3). | |
| 2017-06-23 17:09:06.757 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) [CANCELED] | |
| 2017-06-23 17:09:06.758 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.759 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d). | |
| 2017-06-23 17:09:06.759 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) [CANCELED] | |
| 2017-06-23 17:09:06.768 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.768 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676). | |
| 2017-06-23 17:09:06.768 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) [CANCELED] | |
| 2017-06-23 17:09:06.776 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.776 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678). | |
| 2017-06-23 17:09:06.776 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) [CANCELED] | |
| 2017-06-23 17:09:06.781 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (fea30a9bbaadb42ded86af0b736e1384) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.782 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/2) (468379a5ad983d0529755d9b3165e97d) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.781 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (2/2) (271b2171285e15491396af991501e6c3) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.782 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (2/2) (f6b7da51c78a783c40d2f831e4041676) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.785 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/2) (2da2dd9253bfecf0532d1d260f7fc678) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.883 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-24bca7b0-47fc-4f06-bcc2-ae484fc76cc5/job-5d6607746c6f40f45be3144ca4df4d93_op-WindowOperator_62_0_uuid-8c5ffc0a-1d9b-4efa-9423-b5dcd020a9e5 | |
| 2017-06-23 17:09:06.886 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.886 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077). | |
| 2017-06-23 17:09:06.886 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) [CANCELED] | |
| 2017-06-23 17:09:06.892 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/2) (001562689e50727a0b2ec197d92e4077) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:06.997 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-2df52e4f-5d79-452b-8c57-b9d3e4561cf1/job-5d6607746c6f40f45be3144ca4df4d93_op-WindowOperator_62_1_uuid-94f23628-4b11-4836-a556-22c818e6a530 | |
| 2017-06-23 17:09:07.000 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:07.000 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e). | |
| 2017-06-23 17:09:07.000 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) [CANCELED] | |
| 2017-06-23 17:09:07.006 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (2/2) (2f67f032e91e39a44bd242612ad8901e) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:07.811 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_62_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:09:07.811 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to UNregister for key WindowOperator_62_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=63}-000011.sst | |
| 2017-06-23 17:09:07.922 [ip-10-150-96-68] WARN org.apache.flink.runtime.checkpoint.OperatorSubtaskState - Error while discarding operator states. | |
| java.lang.IllegalStateException: Cannot unregister a state that is not registered. | |
| at org.apache.flink.util.Preconditions.checkState(Preconditions.java:195) | |
| at org.apache.flink.runtime.state.SharedStateRegistry.unregisterReference(SharedStateRegistry.java:132) | |
| at org.apache.flink.runtime.state.IncrementalKeyedStateHandle.discardState(IncrementalKeyedStateHandle.java:178) | |
| at org.apache.flink.runtime.state.StateUtil.bestEffortDiscardAllStateObjects(StateUtil.java:61) | |
| at org.apache.flink.runtime.checkpoint.OperatorSubtaskState.discardState(OperatorSubtaskState.java:137) | |
| at org.apache.flink.runtime.checkpoint.OperatorState.discardState(OperatorState.java:117) | |
| at org.apache.flink.runtime.state.StateUtil.bestEffortDiscardAllStateObjects(StateUtil.java:61) | |
| at org.apache.flink.runtime.checkpoint.PendingCheckpoint$1.run(PendingCheckpoint.java:545) | |
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) | |
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) | |
| at java.lang.Thread.run(Thread.java:748) | |
| 2017-06-23 17:09:07.922 [ip-10-150-96-68] WARN org.apache.flink.runtime.checkpoint.OperatorSubtaskState - Error while discarding operator states. | |
| java.lang.IllegalStateException: Cannot unregister a state that is not registered. | |
| at org.apache.flink.util.Preconditions.checkState(Preconditions.java:195) | |
| at org.apache.flink.runtime.state.SharedStateRegistry.unregisterReference(SharedStateRegistry.java:132) | |
| at org.apache.flink.runtime.state.IncrementalKeyedStateHandle.discardState(IncrementalKeyedStateHandle.java:178) | |
| at org.apache.flink.runtime.state.StateUtil.bestEffortDiscardAllStateObjects(StateUtil.java:61) | |
| at org.apache.flink.runtime.checkpoint.OperatorSubtaskState.discardState(OperatorSubtaskState.java:137) | |
| at org.apache.flink.runtime.checkpoint.OperatorState.discardState(OperatorState.java:117) | |
| at org.apache.flink.runtime.state.StateUtil.bestEffortDiscardAllStateObjects(StateUtil.java:61) | |
| at org.apache.flink.runtime.checkpoint.PendingCheckpoint$1.run(PendingCheckpoint.java:545) | |
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) | |
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) | |
| at java.lang.Thread.run(Thread.java:748) | |
| 2017-06-23 17:09:08.627 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:08.627 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811). | |
| 2017-06-23 17:09:08.628 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) [CANCELED] | |
| 2017-06-23 17:09:08.634 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (180c1cb27ee98db1589be6ef216de811) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:09:08.634 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (5d6607746c6f40f45be3144ca4df4d93) switched from state CANCELLING to CANCELED. | |
| 2017-06-23 17:09:08.635 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 5d6607746c6f40f45be3144ca4df4d93 | |
| 2017-06-23 17:09:08.635 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 5d6607746c6f40f45be3144ca4df4d93 | |
| 2017-06-23 17:09:08.635 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:09:08.635 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:09:08.635 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 2 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e' not discarded. | |
| 2017-06-23 17:09:08.635 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 2 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-f111883e549e' not discarded. | |
| 2017-06-23 17:09:21.471 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:09:21.788 [ip-10-150-96-68] INFO org.apache.flink.runtime.client.JobClient - Checking and uploading JAR files | |
| 2017-06-23 17:09:21.789 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:09:22.130 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job recovers via failover strategy: full graph restart | |
| 2017-06-23 17:09:22.132 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:09:22.132 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:09:22.133 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (931639851a3e82c83576b702a95573ad) switched from state CREATED to RUNNING. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: null source -> Sink: Redshift Transmitter (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:09:22.135 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:09:22.135 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: S3 File Watcher (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) [DEPLOYING] | |
| 2017-06-23 17:09:22.134 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) [DEPLOYING]. | |
| 2017-06-23 17:09:22.138 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:09:22.138 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:09:22.139 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:09:22.139 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:09:22.140 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:09:22.140 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:09:22.139 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:09:22.143 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:09:22.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:09:22.145 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) [DEPLOYING]. | |
| 2017-06-23 17:09:22.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:09:22.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) [DEPLOYING] | |
| 2017-06-23 17:09:22.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) [DEPLOYING] | |
| 2017-06-23 17:09:22.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) [DEPLOYING]. | |
| 2017-06-23 17:09:22.144 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) [DEPLOYING]. | |
| 2017-06-23 17:09:22.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) [DEPLOYING]. | |
| 2017-06-23 17:09:22.145 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) [DEPLOYING]. | |
| 2017-06-23 17:09:22.146 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.152 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.152 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.152 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.147 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) [DEPLOYING] | |
| 2017-06-23 17:09:22.147 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) [DEPLOYING] | |
| 2017-06-23 17:09:22.148 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.148 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) [DEPLOYING]. | |
| 2017-06-23 17:09:22.148 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) [DEPLOYING]. | |
| 2017-06-23 17:09:22.149 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) [DEPLOYING]. | |
| 2017-06-23 17:09:22.149 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) [DEPLOYING]. | |
| 2017-06-23 17:09:22.153 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.154 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.159 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.159 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:09:22.798 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.798 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.802 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.802 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.803 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.803 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.811 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.811 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.812 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:09:22.812 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: null. | |
| 2017-06-23 17:09:22.859 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.859 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.888 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.888 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.897 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.897 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.999 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:22.999 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:23.026 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:23.026 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:23.036 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:23.036 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:23.060 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:23.060 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:52.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 1 @ 1498237792133 | |
| 2017-06-23 17:09:52.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 1 @ 1498237792133 | |
| 2017-06-23 17:09:52.137 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 9db0de4e61d53217185ed5a099a91ee8 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:52.137 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 9db0de4e61d53217185ed5a099a91ee8 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:53.607 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 5 ms. | |
| 2017-06-23 17:09:53.607 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 5 ms. | |
| 2017-06-23 17:09:53.664 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:09:53.671 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 0ea0c6df7951b593c2ef006757d5df97 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:53.671 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 0ea0c6df7951b593c2ef006757d5df97 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:53.712 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 38 ms. | |
| 2017-06-23 17:09:53.712 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 38 ms. | |
| 2017-06-23 17:09:54.033 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-42-thread-1,5,Flink Task Threads] took 417 ms. | |
| 2017-06-23 17:09:54.033 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-42-thread-1,5,Flink Task Threads] took 417 ms. | |
| 2017-06-23 17:09:54.038 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task dbe1f5bbd22669eebb77208158f28f68 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:54.038 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task dbe1f5bbd22669eebb77208158f28f68 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:54.350 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 637 ms. | |
| 2017-06-23 17:09:54.350 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 637 ms. | |
| 2017-06-23 17:09:54.875 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 525 ms. | |
| 2017-06-23 17:09:54.875 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 525 ms. | |
| 2017-06-23 17:09:55.205 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes] | |
| 2017-06-23 17:09:55.212 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes] | |
| 2017-06-23 17:09:55.212 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/c0522697-d4dc-4624-9dac-b952595bf2eb'} | |
| 2017-06-23 17:09:55.452 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/6ce7dd18-b87e-4f27-9bbc-e6308983b4ac [8290 bytes] | |
| 2017-06-23 17:09:55.453 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/7c2cf4f0-54a6-46b8-9fe4-f33ed9c2a29f'} | |
| 2017-06-23 17:09:55.459 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task f3817edcc1773b47abe08c666b91dc77 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:55.459 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task f3817edcc1773b47abe08c666b91dc77 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:55.710 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 834 ms. | |
| 2017-06-23 17:09:55.710 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 834 ms. | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 0933645d2e4e6d5dd924853440e7540e of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 1 from task 0933645d2e4e6d5dd924853440e7540e of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes] | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes] | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1} | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1} | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:09:55.718 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:09:55.922 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 1 (21045466 bytes in 3789 ms). | |
| 2017-06-23 17:09:55.922 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 1 (21045466 bytes in 3789 ms). | |
| 2017-06-23 17:09:55.922 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 12488905), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 5305005), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3247694), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0) | |
| 2017-06-23 17:09:55.922 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 12488905), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 5305005), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3247694), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0) | |
| 2017-06-23 17:10:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 2 @ 1498237822133 | |
| 2017-06-23 17:10:22.134 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 2 @ 1498237822133 | |
| 2017-06-23 17:10:22.137 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 9db0de4e61d53217185ed5a099a91ee8 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:22.137 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 9db0de4e61d53217185ed5a099a91ee8 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:31.593 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:10:31.593 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 0 ms. | |
| 2017-06-23 17:10:31.653 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:10:31.658 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 0ea0c6df7951b593c2ef006757d5df97 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:31.658 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 0ea0c6df7951b593c2ef006757d5df97 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:31.681 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 24 ms. | |
| 2017-06-23 17:10:31.681 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 24 ms. | |
| 2017-06-23 17:10:31.976 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-42-thread-1,5,Flink Task Threads] took 382 ms. | |
| 2017-06-23 17:10:31.976 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-42-thread-1,5,Flink Task Threads] took 382 ms. | |
| 2017-06-23 17:10:31.981 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task dbe1f5bbd22669eebb77208158f28f68 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:31.981 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task dbe1f5bbd22669eebb77208158f28f68 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:33.281 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 1598 ms. | |
| 2017-06-23 17:10:33.281 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 1598 ms. | |
| 2017-06-23 17:10:33.872 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes] | |
| 2017-06-23 17:10:33.872 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes] | |
| 2017-06-23 17:10:33.872 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:10:33.872 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/f77abab5-8d9b-4958-a35a-962620f7528a'} | |
| 2017-06-23 17:10:33.879 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 598 ms. | |
| 2017-06-23 17:10:33.879 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 598 ms. | |
| 2017-06-23 17:10:34.165 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a9f7a266-5df6-4320-a2b1-90c496f12566 [8290 bytes] | |
| 2017-06-23 17:10:34.165 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a9e7be0b-038f-4fe1-9e6d-d421e3034bef'} | |
| 2017-06-23 17:10:34.172 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task f3817edcc1773b47abe08c666b91dc77 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:34.172 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task f3817edcc1773b47abe08c666b91dc77 of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:35.896 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 2016 ms. | |
| 2017-06-23 17:10:35.896 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad, asynchronous part) in thread Thread[pool-39-thread-1,5,Flink Task Threads] took 2016 ms. | |
| 2017-06-23 17:10:35.906 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 0933645d2e4e6d5dd924853440e7540e of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:35.906 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 2 from task 0933645d2e4e6d5dd924853440e7540e of job 931639851a3e82c83576b702a95573ad. | |
| 2017-06-23 17:10:35.906 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:10:35.906 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3c74d58d | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3c74d58d | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=2} | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=2} | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes] | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes] | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1} | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1} | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1}, WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=2}} | |
| 2017-06-23 17:10:35.907 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1}, WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=2}} | |
| 2017-06-23 17:10:36.389 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 2 (46299511 bytes in 14256 ms). | |
| 2017-06-23 17:10:36.389 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed checkpoint 2 (46299511 bytes in 14256 ms). | |
| 2017-06-23 17:10:36.389 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 31188457), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 11736094), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3371098), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0) | |
| 2017-06-23 17:10:36.389 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Checkpoint state: OperatorState(operatorID: 3c249f46784e7918941b8254a455626e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: 2507a43fb0c9c91dd4f7e60bcfef8d4e, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0), OperatorState(operatorID: bb0199e122f4970209706820e4def95b, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 31188457), OperatorState(operatorID: d1c48e48788f3367ffb96f7809bcfb9d, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 11736094), OperatorState(operatorID: 44aee4921a91b9a2911691cc10171a24, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3371098), OperatorState(operatorID: cc6f3a804a6c2964c83f3008635bbb25, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 3862), OperatorState(operatorID: 70d8f5f0f7af92550fed76e829eeea46, parallelism: 1, maxParallelism: 128, sub task states: 1, total size (bytes): 0) | |
| 2017-06-23 17:10:45.907 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (931639851a3e82c83576b702a95573ad) switched from state RUNNING to CANCELLING. | |
| 2017-06-23 17:10:45.908 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.908 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.908 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.908 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.908 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.906 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8). | |
| 2017-06-23 17:10:45.906 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.906 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8). | |
| 2017-06-23 17:10:45.907 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68). | |
| 2017-06-23 17:10:45.907 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.907 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68). | |
| 2017-06-23 17:10:45.907 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e). | |
| 2017-06-23 17:10:45.907 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.910 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:45.910 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8). | |
| 2017-06-23 17:10:45.910 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e). | |
| 2017-06-23 17:10:45.910 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) [CANCELED] | |
| 2017-06-23 17:10:45.911 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77). | |
| 2017-06-23 17:10:45.911 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.911 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77). | |
| 2017-06-23 17:10:45.916 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Attempting to cancel task To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97). | |
| 2017-06-23 17:10:45.916 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from RUNNING to CANCELING. | |
| 2017-06-23 17:10:45.917 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Triggering cancellation of task code To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97). | |
| 2017-06-23 17:10:45.922 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:45.922 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97). | |
| 2017-06-23 17:10:45.923 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) [CANCELED] | |
| 2017-06-23 17:10:45.927 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (9db0de4e61d53217185ed5a099a91ee8) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:45.927 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (0ea0c6df7951b593c2ef006757d5df97) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:45.982 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - deleting instance base path /media/flink/tmp1/flink-io-2df52e4f-5d79-452b-8c57-b9d3e4561cf1/job-931639851a3e82c83576b702a95573ad_op-WindowOperator_80_0_uuid-146601bf-8d97-4864-b5e3-e49f409272d9 | |
| 2017-06-23 17:10:45.985 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:45.985 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77). | |
| 2017-06-23 17:10:45.985 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) [CANCELED] | |
| 2017-06-23 17:10:45.990 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (f3817edcc1773b47abe08c666b91dc77) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:46.862 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:46.863 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e). | |
| 2017-06-23 17:10:46.863 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) [CANCELED] | |
| 2017-06-23 17:10:46.868 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (0933645d2e4e6d5dd924853440e7540e) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:10:49.059 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:10:49.421 [ip-10-150-96-68] INFO org.apache.flink.runtime.client.JobClient - Checking and uploading JAR files | |
| 2017-06-23 17:10:49.421 [ip-10-150-96-68] INFO org.apache.flink.runtime.blob.BlobClient - Blob client connecting to akka://flink/user/jobmanager | |
| 2017-06-23 17:10:49.745 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job recovers via failover strategy: full graph restart | |
| 2017-06-23 17:10:49.746 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:10:49.746 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Persisting periodic checkpoints externally at s3://mm-flink-checkpoints//cliff. | |
| 2017-06-23 17:10:49.747 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Starting job from savepoint s3://mm-flink-checkpoints/cliff/checkpoint_metadata-36fb0c30062e () | |
| 2017-06-23 17:10:49.747 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Starting job from savepoint s3://mm-flink-checkpoints/cliff/checkpoint_metadata-36fb0c30062e () | |
| 2017-06-23 17:10:49.747 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.savepoint.SavepointStore - Loading savepoint from s3://mm-flink-checkpoints/cliff/checkpoint_metadata-36fb0c30062e | |
| 2017-06-23 17:10:49.747 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.savepoint.SavepointStore - Loading savepoint from s3://mm-flink-checkpoints/cliff/checkpoint_metadata-36fb0c30062e | |
| 2017-06-23 17:10:49.922 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:10:49.922 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {} | |
| 2017-06-23 17:10:49.922 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes] | |
| 2017-06-23 17:10:49.922 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes] | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1} | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1} | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes] | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes] | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1} | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Registration for key WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst has result entry: SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1} | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1}, WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry AFTER registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1}, WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Reset the checkpoint ID to 3. | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Reset the checkpoint ID to 3. | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Restoring from latest valid checkpoint: Checkpoint 2 @ 0 for f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:49.923 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Restoring from latest valid checkpoint: Checkpoint 2 @ 0 for f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (f4775894b097d0d9aaa57e86ba7fb236) switched from state CREATED to RUNNING. | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:10:49.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) switched from CREATED to SCHEDULED. | |
| 2017-06-23 17:10:49.925 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:10:49.925 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: null source -> Sink: Redshift Transmitter (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:10:49.925 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:10:49.925 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: S3 File Watcher (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:10:49.926 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:10:49.926 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:10:49.927 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:10:49.927 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Aggregate Metrics -> Extract Value (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:10:49.928 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) switched from SCHEDULED to DEPLOYING. | |
| 2017-06-23 17:10:49.928 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying To Tsv -> Sink: pulse-streaming-report (1/1) (attempt #0) to ip-10-150-96-244 | |
| 2017-06-23 17:10:49.925 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:10:49.925 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) [DEPLOYING] | |
| 2017-06-23 17:10:49.925 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) [DEPLOYING]. | |
| 2017-06-23 17:10:49.927 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:10:49.927 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) [DEPLOYING] | |
| 2017-06-23 17:10:49.928 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) [DEPLOYING]. | |
| 2017-06-23 17:10:49.932 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) [DEPLOYING]. | |
| 2017-06-23 17:10:49.932 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) [DEPLOYING]. | |
| 2017-06-23 17:10:49.933 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:49.933 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:49.941 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (f03b21449527c44b05c937ff83f05b09) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:49.943 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: null source -> Sink: Redshift Transmitter (1/1) (30be36278574b2f284c9c529c2e71b2c) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:49.959 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:10:49.959 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) [DEPLOYING] | |
| 2017-06-23 17:10:49.949 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:10:49.959 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) [DEPLOYING]. | |
| 2017-06-23 17:10:49.959 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) [DEPLOYING]. | |
| 2017-06-23 17:10:49.970 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) [DEPLOYING] | |
| 2017-06-23 17:10:49.970 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) switched from CREATED to DEPLOYING. | |
| 2017-06-23 17:10:49.970 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) [DEPLOYING]. | |
| 2017-06-23 17:10:49.971 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak safety net for task To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) [DEPLOYING] | |
| 2017-06-23 17:10:49.971 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) [DEPLOYING]. | |
| 2017-06-23 17:10:49.971 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) [DEPLOYING]. | |
| 2017-06-23 17:10:49.971 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network: To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) [DEPLOYING]. | |
| 2017-06-23 17:10:49.992 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:50.000 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:50.003 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:49.987 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Aggregate Metrics -> Extract Value (1/1) (2fa1c1c7e2390594f88612abe909ec01) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:49.992 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1) (7da89a059de4c890bf7488384c79534c) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:49.998 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - To Tsv -> Sink: pulse-streaming-report (1/1) (2ab51732d798a5c5b89e5bc74822ac9f) switched from DEPLOYING to RUNNING. | |
| 2017-06-23 17:10:50.198 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.198 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.224 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.224 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.232 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.232 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.232 [ip-10-150-96-244] INFO o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Initializing RocksDB keyed state backend from snapshot. | |
| 2017-06-23 17:10:50.232 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Restoring snapshot from state handles: [org.apache.flink.runtime.state.IncrementalKeyedStateHandle@37f51cd7]. | |
| 2017-06-23 17:10:50.232 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - restoring instance org.apache.flink.runtime.state.IncrementalKeyedStateHandle@37f51cd7, hasExtrayKeys: false | |
| 2017-06-23 17:10:50.233 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000011.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes] | |
| 2017-06-23 17:10:50.266 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.266 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.276 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.276 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.302 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.302 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.434 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading 000012.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes] | |
| 2017-06-23 17:10:50.511 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading MANIFEST-000006 -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/f77abab5-8d9b-4958-a35a-962620f7528a'} | |
| 2017-06-23 17:10:50.511 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading OPTIONS-000010 -> File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a9f7a266-5df6-4320-a2b1-90c496f12566 [8290 bytes] | |
| 2017-06-23 17:10:50.542 [ip-10-150-96-244] TRACE o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - reading CURRENT -> ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a9e7be0b-038f-4fe1-9e6d-d421e3034bef'} | |
| 2017-06-23 17:10:50.549 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.549 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.576 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.576 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.604 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.604 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.610 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.610 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.883 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:10:50.883 [ip-10-150-96-244] DEBUG o.a.flink.runtime.state.filesystem.FsCheckpointStreamFactory - Initialed file stream factory to URI s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:05.006 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:11:05.007 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68). | |
| 2017-06-23 17:11:05.008 [ip-10-150-96-244] INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are closed for task Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) [CANCELED] | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: S3 File Watcher (1/1) (dbe1f5bbd22669eebb77208158f28f68) switched from CANCELING to CANCELED. | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Report Pipeline (931639851a3e82c83576b702a95573ad) switched from state CANCELLING to CANCELED. | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 931639851a3e82c83576b702a95573ad | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 931639851a3e82c83576b702a95573ad | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 1 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-5ffaa5ddadda' not discarded. | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 1 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-5ffaa5ddadda' not discarded. | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 2 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-36fb0c30062e' not discarded. | |
| 2017-06-23 17:11:05.014 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CompletedCheckpoint - Persistent checkpoint with ID 2 at 's3://mm-flink-checkpoints/cliff/checkpoint_metadata-36fb0c30062e' not discarded. | |
| 2017-06-23 17:11:19.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237879924 | |
| 2017-06-23 17:11:19.924 [ip-10-150-96-68] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 3 @ 1498237879924 | |
| 2017-06-23 17:11:19.928 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 30be36278574b2f284c9c529c2e71b2c of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:19.928 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 30be36278574b2f284c9c529c2e71b2c of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:22.304 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 2 ms. | |
| 2017-06-23 17:11:22.304 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, synchronous part) in thread Thread[Async calls on Source: S3 File Watcher (1/1),5,Flink Task Threads] took 2 ms. | |
| 2017-06-23 17:11:22.354 [ip-10-150-96-244] INFO com.mediamath.reporting.combiner.Combiner - start combiner snapshot for id 0 | |
| 2017-06-23 17:11:22.360 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2ab51732d798a5c5b89e5bc74822ac9f of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:22.360 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2ab51732d798a5c5b89e5bc74822ac9f of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:22.394 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 27 ms. | |
| 2017-06-23 17:11:22.394 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 27 ms. | |
| 2017-06-23 17:11:22.669 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, asynchronous part) in thread Thread[pool-43-thread-1,5,Flink Task Threads] took 365 ms. | |
| 2017-06-23 17:11:22.669 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, asynchronous part) in thread Thread[pool-43-thread-1,5,Flink Task Threads] took 365 ms. | |
| 2017-06-23 17:11:22.676 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task f03b21449527c44b05c937ff83f05b09 of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:22.676 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task f03b21449527c44b05c937ff83f05b09 of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:24.779 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236/chk-3/9c06193f-1209-43d8-933a-ef5b10c650ab [2475365 bytes] | |
| 2017-06-23 17:11:24.779 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - mapping 000017.sst -> File State: s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236/chk-3/9c06193f-1209-43d8-933a-ef5b10c650ab [2475365 bytes] | |
| 2017-06-23 17:11:24.779 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000012.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:11:24.779 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - 000011.sst exists already, mapping it -> PlaceholderStreamStateHandle | |
| 2017-06-23 17:11:24.780 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236/chk-3/5bc89945-31f9-4448-9a5c-e5cab3b5f632'} | |
| 2017-06-23 17:11:24.941 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 2544 ms. | |
| 2017-06-23 17:11:24.941 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, synchronous part) in thread Thread[S3 File Reader -> Parser -> Timestamps/Watermarks -> Combiner (1/1),5,Flink Task Threads] took 2544 ms. | |
| 2017-06-23 17:11:25.068 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle File State: s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236/chk-3/205abe99-0d07-4130-89c9-51ceb4d1db74 [8290 bytes] | |
| 2017-06-23 17:11:25.068 [ip-10-150-96-244] DEBUG o.a.flink.contrib.streaming.state.RocksDBKeyedStateBackend - Got StreamStateHandle from stream.closeAndGetHandle ByteStreamStateHandle{handleName='s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236/chk-3/a8831754-47be-4103-b6aa-5f2a2c99da0d'} | |
| 2017-06-23 17:11:25.074 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2fa1c1c7e2390594f88612abe909ec01 of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:25.074 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 2fa1c1c7e2390594f88612abe909ec01 of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:25.518 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, asynchronous part) in thread Thread[pool-46-thread-1,5,Flink Task Threads] took 576 ms. | |
| 2017-06-23 17:11:25.518 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, asynchronous part) in thread Thread[pool-46-thread-1,5,Flink Task Threads] took 576 ms. | |
| 2017-06-23 17:11:31.566 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, asynchronous part) in thread Thread[pool-46-thread-1,5,Flink Task Threads] took 6048 ms. | |
| 2017-06-23 17:11:31.566 [ip-10-150-96-244] INFO org.apache.flink.runtime.state.DefaultOperatorStateBackend - DefaultOperatorStateBackend snapshot (File Stream Factory @ s3://mm-prod-stats-pulse-streaming/test/state/f4775894b097d0d9aaa57e86ba7fb236, asynchronous part) in thread Thread[pool-46-thread-1,5,Flink Task Threads] took 6048 ms. | |
| 2017-06-23 17:11:31.576 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 7da89a059de4c890bf7488384c79534c of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:31.576 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received acknowledge message for checkpoint 3 from task 7da89a059de4c890bf7488384c79534c of job f4775894b097d0d9aaa57e86ba7fb236. | |
| 2017-06-23 17:11:31.576 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1}, WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:11:31.576 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - SharedStateRegistry BEFORE registerAll: {WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000012.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-2/a748f490-9091-42ff-bd66-d42e60295cde [4713957 bytes], referenceCount=1}, WindowOperator_80_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst=SharedStateEntry{stateHandle=File State: s3://mm-prod-stats-pulse-streaming/test/state/931639851a3e82c83576b702a95573ad/chk-1/be563ee2-f721-457a-9790-c7a22cae15ed [2332633 bytes], referenceCount=1}} | |
| 2017-06-23 17:11:31.577 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_98_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3b854ff | |
| 2017-06-23 17:11:31.577 [ip-10-150-96-68] DEBUG org.apache.flink.runtime.state.SharedStateRegistry - Attempt to register for key WindowOperator_98_0-KeyGroupRange{startKeyGroup=0, endKeyGroup=127}-000011.sst the state handle org.apache.flink.runtime.state.PlaceholderStreamStateHandle@3b854ff |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment