embulkでデータ欠損が発生した際の全ログとconfig

s3_to_file.yml.liquid

in:
  type: s3
  access_key_id: "{{ env.AWS_ACCESS_KEY_ID }}"
  secret_access_key: "{{ env.AWS_SECRET_ACCESS_KEY }}"
  bucket: "{{ env.S3_BUCKET }}"
  path_prefix: "{{ env.INPUT_PREFIX }}"
  decoders:
    - type: gzip
  parser:
    type: csv
    delimiter: "\t"
    charset: UTF-8
    newline: CRLF
    null_string: 'NULL'
    skip_header_lines: 0
    columns:
      - {name: s3_path, type: string}
      - {name: updated_at, type: string}
      - {name: vector, type: string}
out:
  type: file
  path_prefix: "{{ env.OUTPUT_PREFIX }}"
  file_ext: 'gz'
  overwrite: true
  formatter:
    type: csv
    header_line: false
    encoding: UTF-8
    delimiter: "\t"
  encoders:
    - {type: gzip, level: 9 }

欠損した.log

$ embulk -J-Dio.netty.leakDetectionLevel=advanced run -X max_threads=5 config/s3_to_file.yml.liquid -b embulk-bundle
2016-02-04 10:19:09.402 +0900: Embulk v0.8.2
2016-02-04 10:19:11.100 +0900 [INFO] (transaction): Loaded plugin embulk-input-s3 (0.2.7)
2016-02-04 10:19:14.286 +0900 [INFO] (transaction): Using local thread executor with max_threads=5 / tasks=64
2016-02-04 10:19:14.347 +0900 [INFO] (transaction): {done:  0 / 64, running: 0}
2016-02-04 10:19:14.513 +0900 [INFO] (task-0001): Writing local file '/tmp/embulk_civitaspo/test/part001.00.gz'
2016-02-04 10:19:14.513 +0900 [INFO] (task-0002): Writing local file '/tmp/embulk_civitaspo/test/part002.00.gz'
2016-02-04 10:19:14.513 +0900 [INFO] (task-0003): Writing local file '/tmp/embulk_civitaspo/test/part003.00.gz'
2016-02-04 10:19:14.513 +0900 [INFO] (task-0000): Writing local file '/tmp/embulk_civitaspo/test/part000.00.gz'
2016-02-04 10:19:14.514 +0900 [INFO] (task-0004): Writing local file '/tmp/embulk_civitaspo/test/part004.00.gz'
2016-02-04 10:19:57.380 +0900 [INFO] (task-0005): Writing local file '/tmp/embulk_civitaspo/test/part005.00.gz'
2016-02-04 10:20:15.021 +0900 [INFO] (task-0006): Writing local file '/tmp/embulk_civitaspo/test/part006.00.gz'
2016-02-04 10:20:19.433 +0900 [INFO] (task-0007): Writing local file '/tmp/embulk_civitaspo/test/part007.00.gz'
2016-02-04 10:20:25.247 +0900 [INFO] (transaction): {done:  4 / 64, running: 4}
2016-02-04 10:20:25.250 +0900 [INFO] (task-0008): Writing local file '/tmp/embulk_civitaspo/test/part008.00.gz'
2016-02-04 10:20:32.488 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:20:32.488 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:20:32.488 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:20:32.488 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:20:32.490 +0900 [INFO] (task-0009): Writing local file '/tmp/embulk_civitaspo/test/part009.00.gz'
2016-02-04 10:20:53.297 +0900 [INFO] (transaction): {done:  6 / 64, running: 4}
2016-02-04 10:20:53.300 +0900 [INFO] (task-0010): Writing local file '/tmp/embulk_civitaspo/test/part010.00.gz'
2016-02-04 10:20:53.379 +0900 [INFO] (task-0011): Writing local file '/tmp/embulk_civitaspo/test/part011.00.gz'
2016-02-04 10:21:20.836 +0900 [INFO] (transaction): {done:  8 / 64, running: 4}
2016-02-04 10:21:20.838 +0900 [INFO] (task-0012): Writing local file '/tmp/embulk_civitaspo/test/part012.00.gz'
2016-02-04 10:21:27.561 +0900 [INFO] (task-0013): Writing local file '/tmp/embulk_civitaspo/test/part013.00.gz'
2016-02-04 10:21:29.951 +0900 [INFO] (transaction): {done: 10 / 64, running: 4}
2016-02-04 10:21:29.951 +0900 [INFO] (transaction): {done: 10 / 64, running: 4}
2016-02-04 10:21:29.952 +0900 [INFO] (transaction): {done: 10 / 64, running: 4}
2016-02-04 10:21:29.956 +0900 [INFO] (task-0014): Writing local file '/tmp/embulk_civitaspo/test/part014.00.gz'
2016-02-04 10:21:40.294 +0900 [INFO] (transaction): {done: 11 / 64, running: 4}
2016-02-04 10:21:40.297 +0900 [INFO] (task-0015): Writing local file '/tmp/embulk_civitaspo/test/part015.00.gz'
2016-02-04 10:22:00.326 +0900 [INFO] (transaction): {done: 12 / 64, running: 4}
2016-02-04 10:22:00.328 +0900 [INFO] (task-0016): Writing local file '/tmp/embulk_civitaspo/test/part016.00.gz'
2016-02-04 10:22:11.680 +0900 [INFO] (task-0017): Writing local file '/tmp/embulk_civitaspo/test/part017.00.gz'
2016-02-04 10:22:14.363 +0900 [INFO] (task-0018): Writing local file '/tmp/embulk_civitaspo/test/part018.00.gz'
2016-02-04 10:22:17.266 +0900 [INFO] (task-0019): Writing local file '/tmp/embulk_civitaspo/test/part019.00.gz'
2016-02-04 10:22:18.238 +0900 [INFO] (task-0020): Writing local file '/tmp/embulk_civitaspo/test/part020.00.gz'
2016-02-04 10:22:24.314 +0900 [INFO] (transaction): {done: 17 / 64, running: 4}
2016-02-04 10:22:24.315 +0900 [INFO] (transaction): {done: 17 / 64, running: 4}
2016-02-04 10:22:24.315 +0900 [INFO] (transaction): {done: 17 / 64, running: 4}
2016-02-04 10:22:24.316 +0900 [INFO] (task-0021): Writing local file '/tmp/embulk_civitaspo/test/part021.00.gz'
2016-02-04 10:22:34.748 +0900 [INFO] (transaction): {done: 18 / 64, running: 4}
2016-02-04 10:22:34.748 +0900 [INFO] (transaction): {done: 18 / 64, running: 4}
2016-02-04 10:22:34.748 +0900 [INFO] (transaction): {done: 18 / 64, running: 4}
2016-02-04 10:22:34.750 +0900 [INFO] (task-0022): Writing local file '/tmp/embulk_civitaspo/test/part022.00.gz'
2016-02-04 10:22:50.623 +0900 [INFO] (transaction): {done: 19 / 64, running: 4}
2016-02-04 10:22:50.625 +0900 [INFO] (task-0023): Writing local file '/tmp/embulk_civitaspo/test/part023.00.gz'
2016-02-04 10:23:01.574 +0900 [INFO] (transaction): {done: 20 / 64, running: 4}
2016-02-04 10:23:01.576 +0900 [INFO] (task-0024): Writing local file '/tmp/embulk_civitaspo/test/part024.00.gz'
2016-02-04 10:23:21.967 +0900 [INFO] (transaction): {done: 21 / 64, running: 4}
2016-02-04 10:23:21.969 +0900 [INFO] (task-0025): Writing local file '/tmp/embulk_civitaspo/test/part025.00.gz'
2016-02-04 10:23:57.250 +0900 [INFO] (transaction): {done: 22 / 64, running: 4}
2016-02-04 10:23:57.253 +0900 [INFO] (task-0026): Writing local file '/tmp/embulk_civitaspo/test/part026.00.gz'
2016-02-04 10:24:09.651 +0900 [INFO] (transaction): {done: 23 / 64, running: 4}
2016-02-04 10:24:09.654 +0900 [INFO] (task-0027): Writing local file '/tmp/embulk_civitaspo/test/part027.00.gz'
2016-02-04 10:24:17.192 +0900 [INFO] (transaction): {done: 24 / 64, running: 4}
2016-02-04 10:24:17.194 +0900 [INFO] (task-0028): Writing local file '/tmp/embulk_civitaspo/test/part028.00.gz'
2016-02-04 10:24:20.585 +0900 [INFO] (transaction): {done: 25 / 64, running: 4}
2016-02-04 10:24:20.588 +0900 [INFO] (task-0029): Writing local file '/tmp/embulk_civitaspo/test/part029.00.gz'
2016-02-04 10:24:25.749 +0900 [INFO] (task-0030): Writing local file '/tmp/embulk_civitaspo/test/part030.00.gz'
2016-02-04 10:24:28.942 +0900 [INFO] (task-0031): Writing local file '/tmp/embulk_civitaspo/test/part031.00.gz'
2016-02-04 10:24:29.532 +0900 [INFO] (task-0032): Writing local file '/tmp/embulk_civitaspo/test/part032.00.gz'
2016-02-04 10:24:43.621 +0900 [INFO] (transaction): {done: 29 / 64, running: 4}
2016-02-04 10:24:43.624 +0900 [INFO] (task-0033): Writing local file '/tmp/embulk_civitaspo/test/part033.00.gz'
2016-02-04 10:24:46.311 +0900 [INFO] (task-0034): Writing local file '/tmp/embulk_civitaspo/test/part034.00.gz'
2016-02-04 10:24:57.968 +0900 [INFO] (task-0035): Writing local file '/tmp/embulk_civitaspo/test/part035.00.gz'
2016-02-04 10:25:02.438 +0900 [INFO] (task-0036): Writing local file '/tmp/embulk_civitaspo/test/part036.00.gz'
2016-02-04 10:25:21.196 +0900 [INFO] (transaction): {done: 33 / 64, running: 4}
2016-02-04 10:25:21.196 +0900 [INFO] (transaction): {done: 33 / 64, running: 4}
2016-02-04 10:25:21.196 +0900 [INFO] (transaction): {done: 33 / 64, running: 4}
2016-02-04 10:25:21.197 +0900 [INFO] (transaction): {done: 33 / 64, running: 4}
2016-02-04 10:25:21.197 +0900 [INFO] (transaction): {done: 33 / 64, running: 4}
2016-02-04 10:25:21.197 +0900 [INFO] (transaction): {done: 33 / 64, running: 4}
2016-02-04 10:25:21.198 +0900 [INFO] (task-0037): Writing local file '/tmp/embulk_civitaspo/test/part037.00.gz'
2016-02-04 10:25:22.404 +0900 [INFO] (task-0038): Writing local file '/tmp/embulk_civitaspo/test/part038.00.gz'
2016-02-04 10:25:24.170 +0900 [INFO] (task-0039): Writing local file '/tmp/embulk_civitaspo/test/part039.00.gz'
2016-02-04 10:25:26.869 +0900 [INFO] (task-0040): Writing local file '/tmp/embulk_civitaspo/test/part040.00.gz'
2016-02-04 10:25:27.309 +0900 [INFO] (task-0041): Writing local file '/tmp/embulk_civitaspo/test/part041.00.gz'
2016-02-04 10:25:30.063 +0900 [INFO] (task-0042): Writing local file '/tmp/embulk_civitaspo/test/part042.00.gz'
2016-02-04 10:25:34.604 +0900 [INFO] (transaction): {done: 39 / 64, running: 4}
2016-02-04 10:25:34.605 +0900 [INFO] (transaction): {done: 39 / 64, running: 4}
2016-02-04 10:25:34.605 +0900 [INFO] (transaction): {done: 39 / 64, running: 4}
2016-02-04 10:25:34.605 +0900 [INFO] (transaction): {done: 39 / 64, running: 4}
2016-02-04 10:25:34.606 +0900 [INFO] (transaction): {done: 39 / 64, running: 4}
2016-02-04 10:25:34.606 +0900 [INFO] (transaction): {done: 39 / 64, running: 4}
2016-02-04 10:25:34.606 +0900 [INFO] (transaction): {done: 39 / 64, running: 4}
2016-02-04 10:25:34.607 +0900 [INFO] (task-0043): Writing local file '/tmp/embulk_civitaspo/test/part043.00.gz'
2016-02-04 10:25:39.289 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:25:39.292 +0900 [INFO] (task-0044): Writing local file '/tmp/embulk_civitaspo/test/part044.00.gz'
2016-02-04 10:25:48.937 +0900 [INFO] (transaction): {done: 41 / 64, running: 4}
2016-02-04 10:25:48.940 +0900 [INFO] (task-0045): Writing local file '/tmp/embulk_civitaspo/test/part045.00.gz'
2016-02-04 10:25:54.891 +0900 [INFO] (transaction): {done: 42 / 64, running: 4}
2016-02-04 10:25:54.893 +0900 [INFO] (task-0046): Writing local file '/tmp/embulk_civitaspo/test/part046.00.gz'
2016-02-04 10:26:03.783 +0900 [INFO] (task-0047): Writing local file '/tmp/embulk_civitaspo/test/part047.00.gz'
2016-02-04 10:26:04.894 +0900 [INFO] (transaction): {done: 44 / 64, running: 4}
2016-02-04 10:26:04.897 +0900 [INFO] (task-0048): Writing local file '/tmp/embulk_civitaspo/test/part048.00.gz'
2016-02-04 10:26:08.378 +0900 [INFO] (transaction): {done: 45 / 64, running: 4}
2016-02-04 10:26:08.381 +0900 [INFO] (task-0049): Writing local file '/tmp/embulk_civitaspo/test/part049.00.gz'
2016-02-04 10:26:08.761 +0900 [INFO] (transaction): {done: 46 / 64, running: 4}
2016-02-04 10:26:08.761 +0900 [INFO] (transaction): {done: 46 / 64, running: 4}
2016-02-04 10:26:08.765 +0900 [INFO] (task-0050): Writing local file '/tmp/embulk_civitaspo/test/part050.00.gz'
2016-02-04 10:26:13.715 +0900 [INFO] (task-0051): Writing local file '/tmp/embulk_civitaspo/test/part051.00.gz'
2016-02-04 10:26:18.411 +0900 [INFO] (task-0052): Writing local file '/tmp/embulk_civitaspo/test/part052.00.gz'
2016-02-04 10:26:22.493 +0900 [INFO] (task-0053): Writing local file '/tmp/embulk_civitaspo/test/part053.00.gz'
2016-02-04 10:26:23.664 +0900 [INFO] (task-0054): Writing local file '/tmp/embulk_civitaspo/test/part054.00.gz'
2016-02-04 10:26:29.186 +0900 [INFO] (task-0055): Writing local file '/tmp/embulk_civitaspo/test/part055.00.gz'
2016-02-04 10:26:40.696 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:26:40.696 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:26:40.696 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:26:40.697 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:26:40.697 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:26:40.697 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:26:40.698 +0900 [INFO] (task-0056): Writing local file '/tmp/embulk_civitaspo/test/part056.00.gz'
2016-02-04 10:26:54.660 +0900 [INFO] (transaction): {done: 53 / 64, running: 4}
2016-02-04 10:26:54.664 +0900 [INFO] (task-0057): Writing local file '/tmp/embulk_civitaspo/test/part057.00.gz'
2016-02-04 10:27:41.315 +0900 [INFO] (task-0058): Writing local file '/tmp/embulk_civitaspo/test/part058.00.gz'
2016-02-04 10:27:42.349 +0900 [INFO] (transaction): {done: 55 / 64, running: 4}
2016-02-04 10:27:42.351 +0900 [INFO] (task-0059): Writing local file '/tmp/embulk_civitaspo/test/part059.00.gz'
2016-02-04 10:27:53.461 +0900 [INFO] (task-0060): Writing local file '/tmp/embulk_civitaspo/test/part060.00.gz'
2016-02-04 10:27:55.507 +0900 [INFO] (task-0061): Writing local file '/tmp/embulk_civitaspo/test/part061.00.gz'
2016-02-04 10:28:04.617 +0900 [INFO] (transaction): {done: 58 / 64, running: 4}
2016-02-04 10:28:04.617 +0900 [INFO] (transaction): {done: 58 / 64, running: 4}
2016-02-04 10:28:04.618 +0900 [INFO] (transaction): {done: 58 / 64, running: 4}
2016-02-04 10:28:04.619 +0900 [INFO] (task-0062): Writing local file '/tmp/embulk_civitaspo/test/part062.00.gz'
2016-02-04 10:28:07.881 +0900 [INFO] (transaction): {done: 59 / 64, running: 4}
2016-02-04 10:28:07.881 +0900 [INFO] (transaction): {done: 59 / 64, running: 4}
2016-02-04 10:28:07.884 +0900 [INFO] (task-0063): Writing local file '/tmp/embulk_civitaspo/test/part063.00.gz'
2016-02-04 10:29:15.044 +0900 [INFO] (transaction): {done: 62 / 64, running: 2}
2016-02-04 10:29:15.044 +0900 [INFO] (transaction): {done: 62 / 64, running: 2}
2016-02-04 10:29:15.045 +0900 [INFO] (transaction): {done: 62 / 64, running: 2}
2016-02-04 10:29:34.118 +0900 [INFO] (transaction): {done: 64 / 64, running: 0}
2016-02-04 10:29:34.118 +0900 [INFO] (transaction): {done: 64 / 64, running: 0}
2016-02-04 10:29:34.127 +0900 [INFO] (main): Committed.
2016-02-04 10:29:34.127 +0900 [INFO] (main): Next config diff: {"in":{"last_path":"staging/image-features/version=0.2-AN/type=ec_shops/d=2014-09-24/h=15/data.gz"},"out":{}}

欠損しなかったけどLEAK.log

2016-02-04 10:06:38.005 +0900: Embulk v0.8.2
2016-02-04 10:06:39.972 +0900 [INFO] (transaction): Loaded plugin embulk-input-s3 (0.2.7)
2016-02-04 10:06:43.384 +0900 [INFO] (transaction): Using local thread executor with max_threads=5 / tasks=64
2016-02-04 10:06:43.470 +0900 [INFO] (transaction): {done:  0 / 64, running: 0}
2016-02-04 10:06:43.653 +0900 [INFO] (task-0000): Writing local file '/tmp/embulk_civitaspo/test/part000.00.gz'
2016-02-04 10:06:43.654 +0900 [INFO] (task-0002): Writing local file '/tmp/embulk_civitaspo/test/part002.00.gz'
2016-02-04 10:06:43.655 +0900 [INFO] (task-0004): Writing local file '/tmp/embulk_civitaspo/test/part004.00.gz'
2016-02-04 10:06:43.656 +0900 [INFO] (task-0003): Writing local file '/tmp/embulk_civitaspo/test/part003.00.gz'
2016-02-04 10:06:43.661 +0900 [INFO] (task-0001): Writing local file '/tmp/embulk_civitaspo/test/part001.00.gz'
2016-02-04 10:07:39.182 +0900 [INFO] (task-0005): Writing local file '/tmp/embulk_civitaspo/test/part005.00.gz'
2016-02-04 10:07:45.081 +0900 [INFO] (task-0006): Writing local file '/tmp/embulk_civitaspo/test/part006.00.gz'
2016-02-04 10:07:47.054 +0900 [INFO] (task-0007): Writing local file '/tmp/embulk_civitaspo/test/part007.00.gz'
2016-02-04 10:07:55.943 +0900 [INFO] (transaction): {done:  4 / 64, running: 4}
2016-02-04 10:07:55.946 +0900 [INFO] (task-0008): Writing local file '/tmp/embulk_civitaspo/test/part008.00.gz'
2016-02-04 10:08:02.855 +0900 [ERROR] (task-0007): LEAK: ByteBuf.release() was not called before it's garbage-collected.
Recent access records: 0
Created at:
        io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:163)
        io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:42)
        io.netty.buffer.PooledByteBufAllocator.newHeapBuffer(PooledByteBufAllocator.java:228)
        io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:136)
        io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:127)
        io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:85)
        org.embulk.exec.PooledBufferAllocator.allocate(PooledBufferAllocator.java:37)
        org.embulk.spi.PageBuilder.newBuffer(PageBuilder.java:53)
        org.embulk.spi.PageBuilder.flush(PageBuilder.java:237)
        org.embulk.spi.PageBuilder.addRecord(PageBuilder.java:212)
        org.embulk.standards.CsvParserPlugin.run(CsvParserPlugin.java:373)
        org.embulk.spi.FileInputRunner.run(FileInputRunner.java:154)
        org.embulk.spi.util.Executors.process(Executors.java:67)
        org.embulk.spi.util.Executors.process(Executors.java:42)
        org.embulk.exec.LocalExecutorPlugin$DirectExecutor$1.call(LocalExecutorPlugin.java:184)
        org.embulk.exec.LocalExecutorPlugin$DirectExecutor$1.call(LocalExecutorPlugin.java:180)
        java.util.concurrent.FutureTask.run(FutureTask.java:262)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        java.lang.Thread.run(Thread.java:745)
2016-02-04 10:08:04.016 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:08:04.017 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:08:04.017 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:08:04.017 +0900 [INFO] (transaction): {done:  5 / 64, running: 4}
2016-02-04 10:08:04.020 +0900 [INFO] (task-0009): Writing local file '/tmp/embulk_civitaspo/test/part009.00.gz'
2016-02-04 10:08:26.868 +0900 [INFO] (task-0010): Writing local file '/tmp/embulk_civitaspo/test/part010.00.gz'
2016-02-04 10:08:46.767 +0900 [INFO] (transaction): {done:  7 / 64, running: 4}
2016-02-04 10:08:46.770 +0900 [INFO] (task-0011): Writing local file '/tmp/embulk_civitaspo/test/part011.00.gz'
2016-02-04 10:08:48.542 +0900 [INFO] (transaction): {done:  8 / 64, running: 4}
2016-02-04 10:08:48.545 +0900 [INFO] (task-0012): Writing local file '/tmp/embulk_civitaspo/test/part012.00.gz'
2016-02-04 10:08:50.888 +0900 [INFO] (transaction): {done:  9 / 64, running: 4}
2016-02-04 10:08:50.892 +0900 [INFO] (task-0013): Writing local file '/tmp/embulk_civitaspo/test/part013.00.gz'
2016-02-04 10:09:06.334 +0900 [INFO] (transaction): {done: 10 / 64, running: 4}
2016-02-04 10:09:06.334 +0900 [INFO] (transaction): {done: 10 / 64, running: 4}
2016-02-04 10:09:06.339 +0900 [INFO] (task-0014): Writing local file '/tmp/embulk_civitaspo/test/part014.00.gz'
2016-02-04 10:09:08.183 +0900 [INFO] (transaction): {done: 11 / 64, running: 4}
2016-02-04 10:09:08.186 +0900 [INFO] (task-0015): Writing local file '/tmp/embulk_civitaspo/test/part015.00.gz'
2016-02-04 10:09:48.267 +0900 [INFO] (task-0016): Writing local file '/tmp/embulk_civitaspo/test/part016.00.gz'
2016-02-04 10:09:50.255 +0900 [INFO] (task-0017): Writing local file '/tmp/embulk_civitaspo/test/part017.00.gz'
2016-02-04 10:09:51.270 +0900 [INFO] (task-0018): Writing local file '/tmp/embulk_civitaspo/test/part018.00.gz'
2016-02-04 10:09:56.113 +0900 [INFO] (task-0019): Writing local file '/tmp/embulk_civitaspo/test/part019.00.gz'
2016-02-04 10:09:59.325 +0900 [INFO] (transaction): {done: 16 / 64, running: 4}
2016-02-04 10:09:59.326 +0900 [INFO] (transaction): {done: 16 / 64, running: 4}
2016-02-04 10:09:59.326 +0900 [INFO] (transaction): {done: 16 / 64, running: 4}
2016-02-04 10:09:59.326 +0900 [INFO] (transaction): {done: 16 / 64, running: 4}
2016-02-04 10:09:59.328 +0900 [INFO] (task-0020): Writing local file '/tmp/embulk_civitaspo/test/part020.00.gz'
2016-02-04 10:10:04.292 +0900 [INFO] (transaction): {done: 17 / 64, running: 4}
2016-02-04 10:10:04.294 +0900 [INFO] (task-0021): Writing local file '/tmp/embulk_civitaspo/test/part021.00.gz'
2016-02-04 10:10:04.826 +0900 [INFO] (transaction): {done: 18 / 64, running: 4}
2016-02-04 10:10:04.826 +0900 [INFO] (transaction): {done: 18 / 64, running: 4}
2016-02-04 10:10:04.829 +0900 [INFO] (task-0022): Writing local file '/tmp/embulk_civitaspo/test/part022.00.gz'
2016-02-04 10:10:27.423 +0900 [INFO] (transaction): {done: 19 / 64, running: 4}
2016-02-04 10:10:27.425 +0900 [INFO] (task-0023): Writing local file '/tmp/embulk_civitaspo/test/part023.00.gz'
2016-02-04 10:10:36.353 +0900 [INFO] (transaction): {done: 20 / 64, running: 4}
2016-02-04 10:10:36.356 +0900 [INFO] (task-0024): Writing local file '/tmp/embulk_civitaspo/test/part024.00.gz'
2016-02-04 10:10:55.630 +0900 [INFO] (transaction): {done: 21 / 64, running: 4}
2016-02-04 10:10:55.632 +0900 [INFO] (task-0025): Writing local file '/tmp/embulk_civitaspo/test/part025.00.gz'
2016-02-04 10:11:27.444 +0900 [INFO] (task-0026): Writing local file '/tmp/embulk_civitaspo/test/part026.00.gz'
2016-02-04 10:11:36.923 +0900 [INFO] (transaction): {done: 23 / 64, running: 4}
2016-02-04 10:11:36.924 +0900 [INFO] (transaction): {done: 23 / 64, running: 4}
2016-02-04 10:11:36.925 +0900 [INFO] (task-0027): Writing local file '/tmp/embulk_civitaspo/test/part027.00.gz'
2016-02-04 10:11:54.955 +0900 [INFO] (task-0028): Writing local file '/tmp/embulk_civitaspo/test/part028.00.gz'
2016-02-04 10:12:05.573 +0900 [INFO] (transaction): {done: 25 / 64, running: 4}
2016-02-04 10:12:05.577 +0900 [INFO] (task-0029): Writing local file '/tmp/embulk_civitaspo/test/part029.00.gz'
2016-02-04 10:12:06.004 +0900 [INFO] (task-0030): Writing local file '/tmp/embulk_civitaspo/test/part030.00.gz'
2016-02-04 10:12:09.771 +0900 [INFO] (task-0031): Writing local file '/tmp/embulk_civitaspo/test/part031.00.gz'
2016-02-04 10:12:11.329 +0900 [INFO] (transaction): {done: 28 / 64, running: 4}
2016-02-04 10:12:11.331 +0900 [INFO] (task-0032): Writing local file '/tmp/embulk_civitaspo/test/part032.00.gz'
2016-02-04 10:12:17.201 +0900 [INFO] (transaction): {done: 29 / 64, running: 4}
2016-02-04 10:12:17.203 +0900 [INFO] (task-0033): Writing local file '/tmp/embulk_civitaspo/test/part033.00.gz'
2016-02-04 10:12:29.916 +0900 [INFO] (task-0034): Writing local file '/tmp/embulk_civitaspo/test/part034.00.gz'
2016-02-04 10:12:44.325 +0900 [INFO] (transaction): {done: 31 / 64, running: 4}
2016-02-04 10:12:44.325 +0900 [INFO] (transaction): {done: 31 / 64, running: 4}
2016-02-04 10:12:44.326 +0900 [INFO] (transaction): {done: 31 / 64, running: 4}
2016-02-04 10:12:44.327 +0900 [INFO] (task-0035): Writing local file '/tmp/embulk_civitaspo/test/part035.00.gz'
2016-02-04 10:12:45.847 +0900 [INFO] (transaction): {done: 32 / 64, running: 4}
2016-02-04 10:12:45.848 +0900 [INFO] (transaction): {done: 32 / 64, running: 4}
2016-02-04 10:12:45.848 +0900 [INFO] (transaction): {done: 32 / 64, running: 4}
2016-02-04 10:12:45.851 +0900 [INFO] (task-0036): Writing local file '/tmp/embulk_civitaspo/test/part036.00.gz'
2016-02-04 10:12:47.902 +0900 [INFO] (task-0037): Writing local file '/tmp/embulk_civitaspo/test/part037.00.gz'
2016-02-04 10:12:53.157 +0900 [INFO] (task-0038): Writing local file '/tmp/embulk_civitaspo/test/part038.00.gz'
2016-02-04 10:12:54.319 +0900 [INFO] (task-0039): Writing local file '/tmp/embulk_civitaspo/test/part039.00.gz'
2016-02-04 10:13:00.896 +0900 [INFO] (task-0040): Writing local file '/tmp/embulk_civitaspo/test/part040.00.gz'
2016-02-04 10:13:08.001 +0900 [INFO] (task-0041): Writing local file '/tmp/embulk_civitaspo/test/part041.00.gz'
2016-02-04 10:13:11.256 +0900 [INFO] (task-0042): Writing local file '/tmp/embulk_civitaspo/test/part042.00.gz'
2016-02-04 10:13:12.879 +0900 [INFO] (task-0043): Writing local file '/tmp/embulk_civitaspo/test/part043.00.gz'
2016-02-04 10:13:17.233 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.234 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.234 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.234 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.234 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.235 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.235 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.235 +0900 [INFO] (transaction): {done: 40 / 64, running: 4}
2016-02-04 10:13:17.236 +0900 [INFO] (task-0044): Writing local file '/tmp/embulk_civitaspo/test/part044.00.gz'
2016-02-04 10:13:27.780 +0900 [INFO] (transaction): {done: 41 / 64, running: 4}
2016-02-04 10:13:27.783 +0900 [INFO] (task-0045): Writing local file '/tmp/embulk_civitaspo/test/part045.00.gz'
2016-02-04 10:13:30.586 +0900 [INFO] (transaction): {done: 42 / 64, running: 4}
2016-02-04 10:13:30.588 +0900 [INFO] (task-0046): Writing local file '/tmp/embulk_civitaspo/test/part046.00.gz'
2016-02-04 10:13:43.104 +0900 [INFO] (task-0047): Writing local file '/tmp/embulk_civitaspo/test/part047.00.gz'
2016-02-04 10:13:44.766 +0900 [INFO] (task-0048): Writing local file '/tmp/embulk_civitaspo/test/part048.00.gz'
2016-02-04 10:13:52.795 +0900 [INFO] (transaction): {done: 45 / 64, running: 4}
2016-02-04 10:13:52.799 +0900 [INFO] (task-0049): Writing local file '/tmp/embulk_civitaspo/test/part049.00.gz'
2016-02-04 10:13:52.902 +0900 [INFO] (task-0050): Writing local file '/tmp/embulk_civitaspo/test/part050.00.gz'
2016-02-04 10:13:55.453 +0900 [INFO] (transaction): {done: 47 / 64, running: 4}
2016-02-04 10:13:55.453 +0900 [INFO] (transaction): {done: 47 / 64, running: 4}
2016-02-04 10:13:55.453 +0900 [INFO] (transaction): {done: 47 / 64, running: 4}
2016-02-04 10:13:55.456 +0900 [INFO] (task-0051): Writing local file '/tmp/embulk_civitaspo/test/part051.00.gz'
2016-02-04 10:14:02.062 +0900 [INFO] (task-0052): Writing local file '/tmp/embulk_civitaspo/test/part052.00.gz'
2016-02-04 10:14:03.676 +0900 [INFO] (task-0053): Writing local file '/tmp/embulk_civitaspo/test/part053.00.gz'
2016-02-04 10:14:07.599 +0900 [INFO] (task-0054): Writing local file '/tmp/embulk_civitaspo/test/part054.00.gz'
2016-02-04 10:14:14.132 +0900 [INFO] (task-0055): Writing local file '/tmp/embulk_civitaspo/test/part055.00.gz'
2016-02-04 10:14:18.065 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:14:18.065 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:14:18.065 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:14:18.066 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:14:18.066 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:14:18.066 +0900 [INFO] (transaction): {done: 52 / 64, running: 4}
2016-02-04 10:14:18.067 +0900 [INFO] (task-0056): Writing local file '/tmp/embulk_civitaspo/test/part056.00.gz'
2016-02-04 10:14:37.041 +0900 [INFO] (transaction): {done: 53 / 64, running: 4}
2016-02-04 10:14:37.044 +0900 [INFO] (task-0057): Writing local file '/tmp/embulk_civitaspo/test/part057.00.gz'
2016-02-04 10:15:08.264 +0900 [INFO] (task-0058): Writing local file '/tmp/embulk_civitaspo/test/part058.00.gz'
2016-02-04 10:15:19.759 +0900 [INFO] (task-0059): Writing local file '/tmp/embulk_civitaspo/test/part059.00.gz'
2016-02-04 10:15:38.770 +0900 [INFO] (task-0060): Writing local file '/tmp/embulk_civitaspo/test/part060.00.gz'
2016-02-04 10:15:38.776 +0900 [INFO] (task-0061): Writing local file '/tmp/embulk_civitaspo/test/part061.00.gz'
2016-02-04 10:15:40.755 +0900 [INFO] (transaction): {done: 58 / 64, running: 4}
2016-02-04 10:15:40.758 +0900 [INFO] (task-0062): Writing local file '/tmp/embulk_civitaspo/test/part062.00.gz'
2016-02-04 10:15:41.127 +0900 [INFO] (transaction): {done: 59 / 64, running: 4}
2016-02-04 10:15:41.127 +0900 [INFO] (transaction): {done: 59 / 64, running: 4}
2016-02-04 10:15:41.127 +0900 [INFO] (transaction): {done: 59 / 64, running: 4}
2016-02-04 10:15:41.128 +0900 [INFO] (transaction): {done: 59 / 64, running: 4}
2016-02-04 10:15:41.128 +0900 [INFO] (transaction): {done: 59 / 64, running: 4}
2016-02-04 10:15:41.130 +0900 [INFO] (task-0063): Writing local file '/tmp/embulk_civitaspo/test/part063.00.gz'
2016-02-04 10:16:51.617 +0900 [INFO] (transaction): {done: 62 / 64, running: 2}
2016-02-04 10:16:51.618 +0900 [INFO] (transaction): {done: 62 / 64, running: 2}
2016-02-04 10:16:51.618 +0900 [INFO] (transaction): {done: 62 / 64, running: 2}
2016-02-04 10:16:59.452 +0900 [INFO] (transaction): {done: 64 / 64, running: 0}
2016-02-04 10:16:59.453 +0900 [INFO] (transaction): {done: 64 / 64, running: 0}
2016-02-04 10:16:59.461 +0900 [INFO] (main): Committed.
2016-02-04 10:16:59.461 +0900 [INFO] (main): Next config diff: {"in":{"last_path":"staging/image-features/version=0.2-AN/type=ec_shops/d=2014-09-24/h=15/data.gz"},"out":{}}