Skip to content

Instantly share code, notes, and snippets.

@tony-lijinwen

tony-lijinwen/sql1 Secret

Created January 5, 2018 11:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tony-lijinwen/d72a0d4bd382c5321362bd5541dcc84d to your computer and use it in GitHub Desktop.
Save tony-lijinwen/d72a0d4bd382c5321362bd5541dcc84d to your computer and use it in GitHub Desktop.
Download ZIP
cache_clear_test
Raw
sql1
Query (id=664a8e28b9a6c9c6:6811bafc00000000):
Summary:
Session ID: 3d40b1fb92f698b7:f848cd946e34769b
Session Type: BEESWAX
Start Time: 2018-01-05 16:38:21.327814000
End Time: 2018-01-05 17:30:43.001170000
Query Type: QUERY
Query State: EXCEPTION
Query Status: Cancelled
Impala Version: impalad version 2.10.0-cdh5.13.0 RELEASE (build 2511805f1eaa991df1460276c7e9f19d819cd4e4)
User: apprun
Connected User: apprun
Delegated User:
Network Address: 10.205.58.36:4894
Default Db: kudu_6
Sql Statement: select * from app_fact_session_base_spark_time_final_4_6_6 e left join app_fact_page_base_spark_time_final_4_6_6 s on e.sessionlogid=s.sessionlogid
Coordinator: zp-stg-hadoop-33:22000
Query Options (set by configuration): MAX_IO_BUFFERS=10,QUERY_TIMEOUT_S=20,BUFFER_POOL_LIMIT=2147483648
Query Options (set by configuration and planner): MAX_IO_BUFFERS=10,QUERY_TIMEOUT_S=20,BUFFER_POOL_LIMIT=2147483648,MT_DOP=0
Plan:
----------------
Max Per-Host Resource Reservation: Memory=34.00MB
Per-Host Resource Estimates: Memory=57.76GB
F03:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=0B mem-reservation=0B
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B
|
05:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=1N,0 row-size=2.27KB cardinality=2621355937
|
F02:PLAN FRAGMENT [HASH(s.sessionlogid)] hosts=6 instances=6
Per-Host Resources: mem-estimate=57.76GB mem-reservation=34.00MB
02:HASH JOIN [RIGHT OUTER JOIN, PARTITIONED]
| hash predicates: s.sessionlogid = e.sessionlogid
| fk/pk conjuncts: none
| mem-estimate=57.76GB mem-reservation=34.00MB spill-buffer=2.00MB
| tuple-ids=1N,0 row-size=2.27KB cardinality=2621355937
|
|--04:EXCHANGE [HASH(e.sessionlogid)]
| | mem-estimate=0B mem-reservation=0B
| | tuple-ids=0 row-size=1.44KB cardinality=228937059
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=6 instances=6
| Per-Host Resources: mem-estimate=0B mem-reservation=0B
| 00:SCAN KUDU [kudu_6.app_fact_session_base_spark_time_final_4_6_6 e]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=0 row-size=1.44KB cardinality=228937059
|
03:EXCHANGE [HASH(s.sessionlogid)]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=1 row-size=845B cardinality=2667559045
|
F00:PLAN FRAGMENT [RANDOM] hosts=6 instances=6
Per-Host Resources: mem-estimate=0B mem-reservation=0B
01:SCAN KUDU [kudu_6.app_fact_page_base_spark_time_final_4_6_6 s]
mem-estimate=0B mem-reservation=0B
tuple-ids=1 row-size=845B cardinality=2667559045
----------------
Estimated Per-Host Mem: 62018256350
Per Host Min Reservation: zp-stg-hadoop-31:22000(34.00 MB) zp-stg-hadoop-32:22000(34.00 MB) zp-stg-hadoop-33:22000(34.00 MB) zp-stg-hadoop-34:22000(34.00 MB) zp-stg-hadoop-35:22000(34.00 MB) zp-stg-hadoop-36:22000(34.00 MB)
Request Pool: root.default
Admission result: Admitted immediately
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
-----------------------------------------------------------------------------------------------------------------------------------------------------
05:EXCHANGE 1 0.000ns 0.000ns 0 2.62B 0 0 UNPARTITIONED
02:HASH JOIN 6 7m34s 11m13s 0 2.62B 2.00 GB 57.76 GB RIGHT OUTER JOIN, PARTITIONED
|--04:EXCHANGE 6 10m14s 11m1s 268.34M 228.94M 0 0 HASH(e.sessionlogid)
| 00:SCAN KUDU 6 495.041ms 596.194ms 268.34M 228.94M 530.85 MB 0 kudu_6.app_fact_session_base_spark_time_final_4_6_6 e
03:EXCHANGE 6 33m32s 35m24s 1.03B 2.67B 0 0 HASH(s.sessionlogid)
01:SCAN KUDU 6 1s584ms 1s675ms 1.03B 2.67B 443.71 MB 0 kudu_6.app_fact_page_base_spark_time_final_4_6_6 s
Errors:
Planner Timeline: 819.024ms
- Analysis finished: 58.698ms (58.698ms)
- Equivalence classes computed: 99.429ms (40.730ms)
- Single node plan created: 744.398ms (644.968ms)
- Runtime filters computed: 752.704ms (8.306ms)
- Distributed plan created: 757.644ms (4.939ms)
- Planning finished: 819.024ms (61.380ms)
Query Timeline: 52m21s
- Query submitted: 54.148us (54.148us)
- Planning finished: 837.470ms (837.415ms)
- Submit for admission: 838.377ms (907.588us)
- Completed admission: 838.404ms (26.375us)
- Ready to start on 6 backends: 838.675ms (271.765us)
- All 6 execution backends (19 fragment instances) started: 842.994ms (4.318ms)
- Rows available: 52m21s (52m20s)
- Unregister query: 52m21s (608.425ms)
- ComputeScanRangeAssignmentTimer: 334.250us
ImpalaServer:
- ClientFetchWaitTimer: 0.000ns
- RowMaterializationTimer: 0.000ns
Execution Profile 664a8e28b9a6c9c6:6811bafc00000000:(Total: 52m20s, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 0
Filter routing table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
Backend startup latencies: Count: 6, min / max: 1ms / 4ms, 25th %-ile: 3ms, 50th %-ile: 3ms, 75th %-ile: 3ms, 90th %-ile: 3ms, 95th %-ile: 4ms, 99.9th %-ile: 4ms
Per Node Peak Memory Usage: zp-stg-hadoop-31:22000(2.96 GB) zp-stg-hadoop-32:22000(2.95 GB) zp-stg-hadoop-33:22000(2.96 GB) zp-stg-hadoop-34:22000(2.96 GB) zp-stg-hadoop-35:22000(2.96 GB) zp-stg-hadoop-36:22000(2.96 GB)
- FiltersReceived: 0 (0)
- FinalizationTimer: 0.000ns
Averaged Fragment F03:(Total: 19.062ms, non-child: 475.640us, % non-child: 2.50%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:52m20s max:52m20s mean: 52m20s stddev:0.000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 16.00 KB (16384)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178101835)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 0.000ns
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 0 (0)
Fragment Instance Lifecycle Timings:
- OpenTime: 0.000ns
- ExecTreeOpenTime: 0.000ns
- PrepareTime: 19.041ms
- ExecTreePrepareTime: 38.947us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 18.586ms, non-child: 18.586ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 18.101ms
EXCHANGE_NODE (id=5):
- ConvertRowBatchTime: 0.000ns
- PeakMemoryUsage: 0
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
DataStreamReceiver:
- BytesReceived: 0
- DeserializeRowBatchTimer: 0.000ns
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 0
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Coordinator Fragment F03:
Instance 664a8e28b9a6c9c6:6811bafc00000000 (host=zp-stg-hadoop-33:22000):(Total: 19.062ms, non-child: 475.640us, % non-child: 2.50%)
MemoryUsage(1m4s): 15.88 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 16.00 KB (16384)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178101835)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 0.000ns
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 0 (0)
Fragment Instance Lifecycle Timings:
- OpenTime: 0.000ns
- ExecTreeOpenTime: 0.000ns
- PrepareTime: 19.041ms
- ExecTreePrepareTime: 38.947us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 18.586ms, non-child: 18.586ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 18.101ms
EXCHANGE_NODE (id=5):
- ConvertRowBatchTime: 0.000ns
- PeakMemoryUsage: 0
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
DataStreamReceiver:
BytesReceived(1m4s): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesReceived: 0
- DeserializeRowBatchTimer: 0.000ns
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 0
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F02:(Total: 11m48s, non-child: 0.000ns, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:52m20s max:52m20s mean: 52m20s stddev:877.893us
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 6
- AverageThreadTokens: 1.23
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.02 GB (2166021336)
- PeakReservation: 2.00 GB (2147483648)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178567650)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 43m27s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 6.71K (6714)
- TotalThreadsTotalWallClockTime: 23m36s
- TotalThreadsSysTime: 24s333ms
- TotalThreadsUserTime: 1m32s
- TotalThreadsVoluntaryContextSwitches: 2.03M (2030811)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 39m32s
- OpenTime: 11m48s
- ExecTreeOpenTime: 11m48s
- PrepareTime: 46.617ms
- ExecTreePrepareTime: 262.325us
DataStreamSender (dst_id=5):(Total: 1.006ms, non-child: 1.006ms, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 192.00 B (192)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 487.667ms, non-child: 487.667ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 123.276ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 63 (63)
- NumInstructions: 2.67K (2670)
- OptimizationTime: 326.303ms
- PeakMemoryUsage: 1.30 MB (1367040)
- PrepareTime: 37.067ms
HASH_JOIN_NODE (id=2):(Total: 51m21s, non-child: 7m34s, % non-child: 14.74%)
- BuildRows: 44.72M (44722873)
- BuildTime: 1m26s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 2.00 GB (2148946128)
- ProbeRows: 171.43M (171428181)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 5m59s
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
Buffer pool:
- AllocTime: 33s670ms
- CumulativeAllocationBytes: 192.15 GB (206318862336)
- CumulativeAllocations: 98.38K (98380)
- PeakReservation: 2.00 GB (2147483648)
- PeakUnpinnedBytes: 192.12 GB (206285307904)
- PeakUsedReservation: 2.00 GB (2147483648)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 192.12 GB (206285307904)
- WriteIoOps: 98.36K (98364)
- WriteIoWaitTime: 530.233ms
Hash Join Builder (join_node_id=2):
- BuildRowsPartitionTime: 1m26s
- BuildRowsPartitioned: 44.72M (44722873)
- HashBuckets: 0 (0)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 0.000ns
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 16 (16)
EXCHANGE_NODE (id=4):(Total: 10m14s, non-child: 10m14s, % non-child: 100.00%)
- ConvertRowBatchTime: 3s077ms
- PeakMemoryUsage: 0
- RowsReturned: 44.72M (44722873)
- RowsReturnedRate: 72.95 K/sec
DataStreamReceiver:
- BytesReceived: 28.28 GB (30362869987)
- DeserializeRowBatchTimer: 3m8s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.02 MB (10503125)
- SendersBlockedTimer: 447.965ms
- SendersBlockedTotalTimer(*): 2s116ms
EXCHANGE_NODE (id=3):(Total: 33m32s, non-child: 33m32s, % non-child: 100.00%)
- ConvertRowBatchTime: 8s518ms
- PeakMemoryUsage: 0
- RowsReturned: 171.43M (171428528)
- RowsReturnedRate: 85.40 K/sec
DataStreamReceiver:
- BytesReceived: 69.55 GB (74681930742)
- DeserializeRowBatchTimer: 12m26s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.07 MB (10553956)
- SendersBlockedTimer: 9m50s
- SendersBlockedTotalTimer(*): 43m19s
Fragment F02:
Instance 664a8e28b9a6c9c6:6811bafc00000012 (host=zp-stg-hadoop-31:22000):(Total: 11m48s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(1m4s): 1.73 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.01 GB, 2.00 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB
ThreadUsage(1m4s): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.23
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.02 GB (2170023844)
- PeakReservation: 2.00 GB (2147483648)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3182379986)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 42m7s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 7.25K (7255)
- TotalThreadsTotalWallClockTime: 23m36s
- TotalThreadsSysTime: 23s930ms
- TotalThreadsUserTime: 1m41s
- TotalThreadsVoluntaryContextSwitches: 1.98M (1983207)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 39m26s
- OpenTime: 11m48s
- ExecTreeOpenTime: 11m48s
- PrepareTime: 51.674ms
- ExecTreePrepareTime: 323.371us
DataStreamSender (dst_id=5):(Total: 1.357ms, non-child: 1.357ms, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 192.00 B (192)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 512.584ms, non-child: 512.584ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 121.516ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 63 (63)
- NumInstructions: 2.67K (2670)
- OptimizationTime: 348.752ms
- PeakMemoryUsage: 1.30 MB (1367040)
- PrepareTime: 41.157ms
HASH_JOIN_NODE (id=2):(Total: 51m14s, non-child: 8m45s, % non-child: 17.10%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 44.73M (44729713)
- BuildTime: 1m29s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 2.00 GB (2148943463)
- ProbeRows: 171.39M (171392000)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 7m10s
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
Buffer pool:
- AllocTime: 33s125ms
- CumulativeAllocationBytes: 192.13 GB (206301036544)
- CumulativeAllocations: 98.37K (98372)
- PeakReservation: 2.00 GB (2147483648)
- PeakUnpinnedBytes: 192.10 GB (206267482112)
- PeakUsedReservation: 2.00 GB (2147483648)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 192.10 GB (206267482112)
- WriteIoOps: 98.36K (98356)
- WriteIoWaitTime: 865.503ms
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled, Spilled
- BuildRowsPartitionTime: 1m29s
- BuildRowsPartitioned: 44.73M (44729713)
- HashBuckets: 0 (0)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 0.000ns
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 16 (16)
EXCHANGE_NODE (id=4):(Total: 10m14s, non-child: 10m14s, % non-child: 100.00%)
- ConvertRowBatchTime: 3s387ms
- PeakMemoryUsage: 0
- RowsReturned: 44.73M (44729713)
- RowsReturnedRate: 72.85 K/sec
DataStreamReceiver:
BytesReceived(16s000ms): 442.92 MB, 1.37 GB, 2.29 GB, 3.16 GB, 4.01 GB, 4.87 GB, 5.71 GB, 6.55 GB, 7.38 GB, 8.09 GB, 8.79 GB, 9.50 GB, 10.18 GB, 10.87 GB, 11.58 GB, 12.29 GB, 12.98 GB, 13.67 GB, 14.37 GB, 15.09 GB, 15.80 GB, 16.53 GB, 17.26 GB, 17.98 GB, 18.70 GB, 19.42 GB, 20.19 GB, 20.97 GB, 21.75 GB, 22.52 GB, 23.30 GB, 24.08 GB, 24.82 GB, 25.39 GB, 25.88 GB, 26.19 GB, 26.28 GB, 26.35 GB, 26.43 GB, 26.55 GB, 26.90 GB, 27.27 GB, 27.66 GB, 28.01 GB
- BytesReceived: 28.28 GB (30367914618)
- DeserializeRowBatchTimer: 3m18s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.04 MB (10528957)
- SendersBlockedTimer: 710.602ms
- SendersBlockedTotalTimer(*): 3s133ms
EXCHANGE_NODE (id=3):(Total: 32m15s, non-child: 32m15s, % non-child: 100.00%)
- ConvertRowBatchTime: 9s175ms
- PeakMemoryUsage: 0
- RowsReturned: 171.39M (171392000)
- RowsReturnedRate: 88.57 K/sec
DataStreamReceiver:
BytesReceived(1m4s): 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 5.13 MB, 1.43 GB, 4.74 GB, 7.89 GB, 10.71 GB, 13.71 GB, 16.82 GB, 20.08 GB, 22.96 GB, 25.60 GB, 27.78 GB, 30.54 GB, 33.27 GB, 36.04 GB, 38.14 GB, 40.03 GB, 41.98 GB, 43.78 GB, 45.77 GB, 46.56 GB, 46.83 GB, 47.00 GB, 47.43 GB, 48.00 GB, 48.37 GB, 48.86 GB, 49.21 GB, 49.70 GB, 49.96 GB, 50.33 GB, 50.75 GB, 51.58 GB, 53.48 GB, 56.25 GB, 58.65 GB, 61.15 GB, 63.60 GB, 66.00 GB
- BytesReceived: 69.54 GB (74664748384)
- DeserializeRowBatchTimer: 14m17s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.07 MB (10556918)
- SendersBlockedTimer: 11m48s
- SendersBlockedTotalTimer(*): 11m48s
Instance 664a8e28b9a6c9c6:6811bafc0000000f (host=zp-stg-hadoop-32:22000):(Total: 11m48s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(1m4s): 1.72 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB
ThreadUsage(1m4s): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.23
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.01 GB (2163094925)
- PeakReservation: 2.00 GB (2147483648)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.95 GB (3172116030)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 43m35s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 7.40K (7401)
- TotalThreadsTotalWallClockTime: 23m36s
- TotalThreadsSysTime: 24s363ms
- TotalThreadsUserTime: 1m39s
- TotalThreadsVoluntaryContextSwitches: 1.99M (1987459)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 39m29s
- OpenTime: 11m48s
- ExecTreeOpenTime: 11m48s
- PrepareTime: 51.108ms
- ExecTreePrepareTime: 257.072us
DataStreamSender (dst_id=5):(Total: 1.306ms, non-child: 1.306ms, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 192.00 B (192)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 498.180ms, non-child: 498.180ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 116.812ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 63 (63)
- NumInstructions: 2.67K (2670)
- OptimizationTime: 340.068ms
- PeakMemoryUsage: 1.30 MB (1367040)
- PrepareTime: 40.179ms
HASH_JOIN_NODE (id=2):(Total: 51m17s, non-child: 7m20s, % non-child: 14.32%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 44.72M (44721483)
- BuildTime: 1m28s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 2.00 GB (2148950031)
- ProbeRows: 171.50M (171495424)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 5m47s
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
Buffer pool:
- AllocTime: 20s520ms
- CumulativeAllocationBytes: 192.20 GB (206372339712)
- CumulativeAllocations: 98.41K (98406)
- PeakReservation: 2.00 GB (2147483648)
- PeakUnpinnedBytes: 192.17 GB (206338785280)
- PeakUsedReservation: 2.00 GB (2147483648)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 192.17 GB (206338785280)
- WriteIoOps: 98.39K (98390)
- WriteIoWaitTime: 297.232ms
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled, Spilled
- BuildRowsPartitionTime: 1m28s
- BuildRowsPartitioned: 44.72M (44721483)
- HashBuckets: 0 (0)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 0.000ns
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 16 (16)
EXCHANGE_NODE (id=4):(Total: 10m15s, non-child: 10m15s, % non-child: 100.00%)
- ConvertRowBatchTime: 3s489ms
- PeakMemoryUsage: 0
- RowsReturned: 44.72M (44721483)
- RowsReturnedRate: 72.62 K/sec
DataStreamReceiver:
BytesReceived(16s000ms): 434.72 MB, 1.36 GB, 2.29 GB, 3.16 GB, 4.00 GB, 4.86 GB, 5.71 GB, 6.55 GB, 7.37 GB, 8.09 GB, 8.79 GB, 9.50 GB, 10.18 GB, 10.87 GB, 11.58 GB, 12.28 GB, 12.98 GB, 13.67 GB, 14.37 GB, 15.09 GB, 15.79 GB, 16.53 GB, 17.26 GB, 17.97 GB, 18.70 GB, 19.41 GB, 20.18 GB, 20.96 GB, 21.75 GB, 22.52 GB, 23.29 GB, 24.07 GB, 24.81 GB, 25.38 GB, 25.87 GB, 26.18 GB, 26.28 GB, 26.34 GB, 26.43 GB, 26.55 GB, 26.89 GB, 27.26 GB, 27.65 GB, 28.00 GB
- BytesReceived: 28.28 GB (30362298928)
- DeserializeRowBatchTimer: 3m19s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 9.91 MB (10393983)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=3):(Total: 33m41s, non-child: 33m41s, % non-child: 100.00%)
- ConvertRowBatchTime: 9s368ms
- PeakMemoryUsage: 0
- RowsReturned: 171.50M (171495424)
- RowsReturnedRate: 84.84 K/sec
DataStreamReceiver:
BytesReceived(1m4s): 5.12 MB, 5.16 MB, 5.16 MB, 5.16 MB, 5.16 MB, 5.16 MB, 5.16 MB, 5.16 MB, 5.16 MB, 5.16 MB, 5.16 MB, 1.42 GB, 4.72 GB, 7.88 GB, 10.69 GB, 13.69 GB, 16.80 GB, 20.06 GB, 22.94 GB, 25.58 GB, 27.75 GB, 30.52 GB, 33.24 GB, 36.02 GB, 38.12 GB, 40.00 GB, 41.96 GB, 43.75 GB, 45.74 GB, 46.54 GB, 46.81 GB, 46.98 GB, 47.40 GB, 47.97 GB, 48.34 GB, 48.83 GB, 49.18 GB, 49.67 GB, 49.94 GB, 50.31 GB, 50.73 GB, 51.55 GB, 53.45 GB, 56.22 GB, 58.62 GB, 61.12 GB, 63.57 GB, 65.97 GB, 68.23 GB
- BytesReceived: 69.58 GB (74710381010)
- DeserializeRowBatchTimer: 12m50s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.06 MB (10550158)
- SendersBlockedTimer: 11m48s
- SendersBlockedTotalTimer(*): 47m14s
Instance 664a8e28b9a6c9c6:6811bafc00000011 (host=zp-stg-hadoop-34:22000):(Total: 11m48s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(1m4s): 1.72 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.01 GB, 2.00 GB, 2.00 GB
ThreadUsage(1m4s): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.23
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.02 GB (2163711626)
- PeakReservation: 2.00 GB (2147483648)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3180574639)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 44m27s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 8.45K (8450)
- TotalThreadsTotalWallClockTime: 23m36s
- TotalThreadsSysTime: 24s918ms
- TotalThreadsUserTime: 1m40s
- TotalThreadsVoluntaryContextSwitches: 1.99M (1985578)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 39m27s
- OpenTime: 11m48s
- ExecTreeOpenTime: 11m48s
- PrepareTime: 50.045ms
- ExecTreePrepareTime: 311.095us
DataStreamSender (dst_id=5):(Total: 2.345ms, non-child: 2.345ms, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 192.00 B (192)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 548.815ms, non-child: 548.815ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 115.525ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 63 (63)
- NumInstructions: 2.67K (2670)
- OptimizationTime: 393.175ms
- PeakMemoryUsage: 1.30 MB (1367040)
- PrepareTime: 38.829ms
HASH_JOIN_NODE (id=2):(Total: 51m15s, non-child: 6m27s, % non-child: 12.59%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 44.72M (44716809)
- BuildTime: 1m28s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 2.00 GB (2148944555)
- ProbeRows: 171.27M (171268096)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 4m53s
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
Buffer pool:
- AllocTime: 14s383ms
- CumulativeAllocationBytes: 192.02 GB (206175207424)
- CumulativeAllocations: 98.31K (98312)
- PeakReservation: 2.00 GB (2147483648)
- PeakUnpinnedBytes: 191.98 GB (206141652992)
- PeakUsedReservation: 2.00 GB (2147483648)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 191.98 GB (206141652992)
- WriteIoOps: 98.30K (98296)
- WriteIoWaitTime: 244.005ms
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled, Spilled
- BuildRowsPartitionTime: 1m28s
- BuildRowsPartitioned: 44.72M (44716809)
- HashBuckets: 0 (0)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 0.000ns
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 16 (16)
EXCHANGE_NODE (id=4):(Total: 10m15s, non-child: 10m15s, % non-child: 100.00%)
- ConvertRowBatchTime: 3s319ms
- PeakMemoryUsage: 0
- RowsReturned: 44.72M (44716809)
- RowsReturnedRate: 72.60 K/sec
DataStreamReceiver:
BytesReceived(16s000ms): 433.68 MB, 1.36 GB, 2.28 GB, 3.15 GB, 3.99 GB, 4.85 GB, 5.70 GB, 6.54 GB, 7.37 GB, 8.08 GB, 8.78 GB, 9.49 GB, 10.17 GB, 10.87 GB, 11.57 GB, 12.28 GB, 12.97 GB, 13.66 GB, 14.37 GB, 15.08 GB, 15.79 GB, 16.52 GB, 17.25 GB, 17.97 GB, 18.69 GB, 19.41 GB, 20.18 GB, 20.96 GB, 21.74 GB, 22.51 GB, 23.29 GB, 24.07 GB, 24.81 GB, 25.38 GB, 25.87 GB, 26.18 GB, 26.27 GB, 26.34 GB, 26.43 GB, 26.55 GB, 26.89 GB, 27.26 GB, 27.64 GB, 28.00 GB
- BytesReceived: 28.27 GB (30359263790)
- DeserializeRowBatchTimer: 3m17s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.03 MB (10521798)
- SendersBlockedTimer: 318.010ms
- SendersBlockedTotalTimer(*): 635.638ms
EXCHANGE_NODE (id=3):(Total: 34m32s, non-child: 34m32s, % non-child: 100.00%)
- ConvertRowBatchTime: 9s107ms
- PeakMemoryUsage: 0
- RowsReturned: 171.27M (171268512)
- RowsReturnedRate: 82.64 K/sec
DataStreamReceiver:
BytesReceived(1m4s): 5.03 MB, 5.07 MB, 5.07 MB, 5.07 MB, 5.07 MB, 5.07 MB, 5.07 MB, 5.07 MB, 5.07 MB, 5.07 MB, 5.07 MB, 1.42 GB, 4.72 GB, 7.87 GB, 10.68 GB, 13.68 GB, 16.78 GB, 20.03 GB, 22.91 GB, 25.55 GB, 27.72 GB, 30.48 GB, 33.20 GB, 35.98 GB, 38.07 GB, 39.95 GB, 41.91 GB, 43.70 GB, 45.69 GB, 46.48 GB, 46.75 GB, 46.92 GB, 47.35 GB, 47.92 GB, 48.29 GB, 48.78 GB, 49.12 GB, 49.62 GB, 49.88 GB, 50.25 GB, 50.67 GB, 51.50 GB, 53.40 GB, 56.16 GB, 58.57 GB, 61.07 GB, 63.51 GB, 65.92 GB, 68.17 GB
- BytesReceived: 69.49 GB (74610378094)
- DeserializeRowBatchTimer: 12m7s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.07 MB (10554482)
- SendersBlockedTimer: 2.121ms
- SendersBlockedTotalTimer(*): 10.652ms
Instance 664a8e28b9a6c9c6:6811bafc00000010 (host=zp-stg-hadoop-36:22000):(Total: 11m48s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(1m4s): 1.74 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB
ThreadUsage(1m4s): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.23
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.01 GB (2162471588)
- PeakReservation: 2.00 GB (2147483648)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3181645337)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 39m39s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 5.81K (5812)
- TotalThreadsTotalWallClockTime: 23m36s
- TotalThreadsSysTime: 33s412ms
- TotalThreadsUserTime: 1m40s
- TotalThreadsVoluntaryContextSwitches: 2.01M (2005300)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 39m26s
- OpenTime: 11m48s
- ExecTreeOpenTime: 11m48s
- PrepareTime: 50.891ms
- ExecTreePrepareTime: 238.072us
DataStreamSender (dst_id=5):(Total: 170.912us, non-child: 170.912us, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 192.00 B (192)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 559.838ms, non-child: 559.838ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 152.272ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 63 (63)
- NumInstructions: 2.67K (2670)
- OptimizationTime: 365.490ms
- PeakMemoryUsage: 1.30 MB (1367040)
- PrepareTime: 41.075ms
HASH_JOIN_NODE (id=2):(Total: 51m14s, non-child: 11m13s, % non-child: 21.91%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 44.72M (44722935)
- BuildTime: 1m59s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 2.00 GB (2148943503)
- ProbeRows: 171.77M (171769856)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 8m51s
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
Buffer pool:
- AllocTime: 1m42s
- CumulativeAllocationBytes: 192.42 GB (206609317888)
- CumulativeAllocations: 98.52K (98519)
- PeakReservation: 2.00 GB (2147483648)
- PeakUnpinnedBytes: 192.39 GB (206575763456)
- PeakUsedReservation: 2.00 GB (2147483648)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 192.39 GB (206575763456)
- WriteIoOps: 98.50K (98503)
- WriteIoWaitTime: 243.754ms
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled, Spilled
- BuildRowsPartitionTime: 1m58s
- BuildRowsPartitioned: 44.72M (44722935)
- HashBuckets: 0 (0)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 0.000ns
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 16 (16)
EXCHANGE_NODE (id=4):(Total: 9m26s, non-child: 9m26s, % non-child: 100.00%)
- ConvertRowBatchTime: 3s182ms
- PeakMemoryUsage: 0
- RowsReturned: 44.72M (44722935)
- RowsReturnedRate: 78.94 K/sec
DataStreamReceiver:
BytesReceived(16s000ms): 455.10 MB, 1.38 GB, 2.31 GB, 3.17 GB, 4.02 GB, 4.88 GB, 5.72 GB, 6.56 GB, 7.39 GB, 8.10 GB, 8.80 GB, 9.51 GB, 10.19 GB, 10.88 GB, 11.59 GB, 12.30 GB, 12.99 GB, 13.68 GB, 14.38 GB, 15.10 GB, 15.81 GB, 16.54 GB, 17.27 GB, 17.99 GB, 18.71 GB, 19.43 GB, 20.20 GB, 20.98 GB, 21.76 GB, 22.53 GB, 23.31 GB, 24.09 GB, 24.82 GB, 25.40 GB, 25.88 GB, 26.19 GB, 26.28 GB, 26.34 GB, 26.43 GB, 26.55 GB, 26.90 GB, 27.27 GB, 27.66 GB, 28.01 GB
- BytesReceived: 28.28 GB (30362534066)
- DeserializeRowBatchTimer: 4m5s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.04 MB (10526808)
- SendersBlockedTimer: 335.421ms
- SendersBlockedTotalTimer(*): 2s003ms
EXCHANGE_NODE (id=3):(Total: 30m34s, non-child: 30m34s, % non-child: 100.00%)
- ConvertRowBatchTime: 9s351ms
- PeakMemoryUsage: 0
- RowsReturned: 171.77M (171769856)
- RowsReturnedRate: 93.64 K/sec
DataStreamReceiver:
BytesReceived(1m4s): 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 5.18 MB, 1.44 GB, 4.75 GB, 7.90 GB, 10.71 GB, 13.71 GB, 16.82 GB, 20.07 GB, 22.95 GB, 25.59 GB, 27.76 GB, 30.53 GB, 33.25 GB, 36.02 GB, 38.12 GB, 40.00 GB, 41.95 GB, 43.75 GB, 45.74 GB, 46.53 GB, 46.80 GB, 46.97 GB, 47.40 GB, 47.97 GB, 48.34 GB, 48.83 GB, 49.17 GB, 49.66 GB, 49.93 GB, 50.30 GB, 50.72 GB, 51.56 GB, 53.48 GB, 56.25 GB, 58.65 GB, 61.15 GB, 63.60 GB, 66.00 GB, 68.26 GB
- BytesReceived: 69.69 GB (74832867129)
- DeserializeRowBatchTimer: 16m4s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.06 MB (10552922)
- SendersBlockedTimer: 11m48s
- SendersBlockedTotalTimer(*): 59m2s
Instance 664a8e28b9a6c9c6:6811bafc0000000d (host=zp-stg-hadoop-33:22000):(Total: 11m48s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(1m4s): 1.72 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB
ThreadUsage(1m4s): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.23
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.02 GB (2166811928)
- PeakReservation: 2.00 GB (2147483648)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178101835)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 46m14s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 5.12K (5116)
- TotalThreadsTotalWallClockTime: 23m36s
- TotalThreadsSysTime: 13s587ms
- TotalThreadsUserTime: 54s305ms
- TotalThreadsVoluntaryContextSwitches: 2.23M (2233446)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 39m57s
- OpenTime: 11m48s
- ExecTreeOpenTime: 11m48s
- PrepareTime: 24.344ms
- ExecTreePrepareTime: 153.548us
DataStreamSender (dst_id=5):(Total: 687.968us, non-child: 687.968us, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 192.00 B (192)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 337.722ms, non-child: 337.722ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 108.705ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 63 (63)
- NumInstructions: 2.67K (2670)
- OptimizationTime: 208.922ms
- PeakMemoryUsage: 1.30 MB (1367040)
- PrepareTime: 19.516ms
HASH_JOIN_NODE (id=2):(Total: 51m46s, non-child: 5m20s, % non-child: 10.30%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 44.72M (44724172)
- BuildTime: 45s039ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 2.00 GB (2148943517)
- ProbeRows: 171.51M (171512832)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 4m32s
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
Buffer pool:
- AllocTime: 15s256ms
- CumulativeAllocationBytes: 192.22 GB (206391214080)
- CumulativeAllocations: 98.42K (98415)
- PeakReservation: 2.00 GB (2147483648)
- PeakUnpinnedBytes: 192.19 GB (206357659648)
- PeakUsedReservation: 2.00 GB (2147483648)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 192.19 GB (206357659648)
- WriteIoOps: 98.40K (98399)
- WriteIoWaitTime: 145.786ms
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled, Spilled
- BuildRowsPartitionTime: 45s020ms
- BuildRowsPartitioned: 44.72M (44724172)
- HashBuckets: 0 (0)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 0.000ns
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 16 (16)
EXCHANGE_NODE (id=4):(Total: 11m1s, non-child: 11m1s, % non-child: 100.00%)
- ConvertRowBatchTime: 1s814ms
- PeakMemoryUsage: 0
- RowsReturned: 44.72M (44724172)
- RowsReturnedRate: 67.61 K/sec
DataStreamReceiver:
BytesReceived(16s000ms): 431.16 MB, 1.36 GB, 2.28 GB, 3.15 GB, 3.99 GB, 4.85 GB, 5.70 GB, 6.54 GB, 7.37 GB, 8.08 GB, 8.78 GB, 9.49 GB, 10.17 GB, 10.87 GB, 11.57 GB, 12.28 GB, 12.97 GB, 13.66 GB, 14.37 GB, 15.08 GB, 15.79 GB, 16.52 GB, 17.25 GB, 17.97 GB, 18.69 GB, 19.41 GB, 20.18 GB, 20.96 GB, 21.74 GB, 22.51 GB, 23.29 GB, 24.07 GB, 24.80 GB, 25.38 GB, 25.87 GB, 26.18 GB, 26.27 GB, 26.34 GB, 26.43 GB, 26.55 GB, 26.89 GB, 27.26 GB, 27.65 GB, 28.00 GB
- BytesReceived: 28.28 GB (30363278995)
- DeserializeRowBatchTimer: 1m27s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.04 MB (10523437)
- SendersBlockedTimer: 98.929ms
- SendersBlockedTotalTimer(*): 296.083ms
EXCHANGE_NODE (id=3):(Total: 35m24s, non-child: 35m24s, % non-child: 100.00%)
- ConvertRowBatchTime: 4s997ms
- PeakMemoryUsage: 0
- RowsReturned: 171.51M (171513504)
- RowsReturnedRate: 80.73 K/sec
DataStreamReceiver:
BytesReceived(1m4s): 5.15 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 1.42 GB, 4.72 GB, 7.88 GB, 10.69 GB, 13.69 GB, 16.79 GB, 20.04 GB, 22.93 GB, 25.56 GB, 27.74 GB, 30.50 GB, 33.22 GB, 36.00 GB, 38.09 GB, 39.98 GB, 41.93 GB, 43.72 GB, 45.71 GB, 46.51 GB, 46.78 GB, 46.95 GB, 47.37 GB, 47.94 GB, 48.31 GB, 48.80 GB, 49.15 GB, 49.64 GB, 49.90 GB, 50.27 GB, 50.69 GB, 51.52 GB, 53.41 GB, 56.18 GB, 58.58 GB, 61.08 GB, 63.53 GB, 65.93 GB, 68.19 GB
- BytesReceived: 69.59 GB (74720722700)
- DeserializeRowBatchTimer: 7m38s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.07 MB (10557145)
- SendersBlockedTimer: 11m50s
- SendersBlockedTotalTimer(*): 1h11m
Instance 664a8e28b9a6c9c6:6811bafc0000000e (host=zp-stg-hadoop-35:22000):(Total: 11m48s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(1m4s): 1.74 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.01 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB, 2.00 GB
ThreadUsage(1m4s): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.23
- BloomFilterBytes: 0
- PeakMemoryUsage: 2.02 GB (2170014108)
- PeakReservation: 2.00 GB (2147483648)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3176588076)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 44m37s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 6.25K (6251)
- TotalThreadsTotalWallClockTime: 23m36s
- TotalThreadsSysTime: 25s789ms
- TotalThreadsUserTime: 1m40s
- TotalThreadsVoluntaryContextSwitches: 1.99M (1989879)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 39m29s
- OpenTime: 11m48s
- ExecTreeOpenTime: 11m48s
- PrepareTime: 51.640ms
- ExecTreePrepareTime: 290.794us
DataStreamSender (dst_id=5):(Total: 168.812us, non-child: 168.812us, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 192.00 B (192)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 468.861ms, non-child: 468.861ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 124.829ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 63 (63)
- NumInstructions: 2.67K (2670)
- OptimizationTime: 301.412ms
- PeakMemoryUsage: 1.30 MB (1367040)
- PrepareTime: 41.646ms
HASH_JOIN_NODE (id=2):(Total: 51m17s, non-child: 6m17s, % non-child: 12.28%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 44.72M (44722130)
- BuildTime: 1m31s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 2.00 GB (2148951704)
- ProbeRows: 171.13M (171130880)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 4m40s
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
Buffer pool:
- AllocTime: 15s911ms
- CumulativeAllocationBytes: 191.91 GB (206064058368)
- CumulativeAllocations: 98.26K (98259)
- PeakReservation: 2.00 GB (2147483648)
- PeakUnpinnedBytes: 191.88 GB (206030503936)
- PeakUsedReservation: 2.00 GB (2147483648)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 191.88 GB (206030503936)
- WriteIoOps: 98.24K (98243)
- WriteIoWaitTime: 1s385ms
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled, Spilled
- BuildRowsPartitionTime: 1m31s
- BuildRowsPartitioned: 44.72M (44722130)
- HashBuckets: 0 (0)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 0.000ns
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 16 (16)
EXCHANGE_NODE (id=4):(Total: 10m11s, non-child: 10m11s, % non-child: 100.00%)
- ConvertRowBatchTime: 3s271ms
- PeakMemoryUsage: 0
- RowsReturned: 44.72M (44722130)
- RowsReturnedRate: 73.10 K/sec
DataStreamReceiver:
BytesReceived(16s000ms): 457.33 MB, 1.38 GB, 2.30 GB, 3.17 GB, 4.02 GB, 4.88 GB, 5.72 GB, 6.56 GB, 7.39 GB, 8.10 GB, 8.80 GB, 9.51 GB, 10.19 GB, 10.88 GB, 11.59 GB, 12.30 GB, 12.99 GB, 13.68 GB, 14.39 GB, 15.10 GB, 15.81 GB, 16.54 GB, 17.27 GB, 17.99 GB, 18.71 GB, 19.43 GB, 20.20 GB, 20.98 GB, 21.76 GB, 22.53 GB, 23.31 GB, 24.09 GB, 24.82 GB, 25.40 GB, 25.88 GB, 26.19 GB, 26.28 GB, 26.34 GB, 26.43 GB, 26.55 GB, 26.90 GB, 27.27 GB, 27.66 GB, 28.01 GB
- BytesReceived: 28.28 GB (30361929529)
- DeserializeRowBatchTimer: 3m20s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.04 MB (10523772)
- SendersBlockedTimer: 1s224ms
- SendersBlockedTotalTimer(*): 6s629ms
EXCHANGE_NODE (id=3):(Total: 34m47s, non-child: 34m47s, % non-child: 100.00%)
- ConvertRowBatchTime: 9s110ms
- PeakMemoryUsage: 0
- RowsReturned: 171.13M (171131872)
- RowsReturnedRate: 81.98 K/sec
DataStreamReceiver:
BytesReceived(1m4s): 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 5.19 MB, 1.44 GB, 4.75 GB, 7.90 GB, 10.71 GB, 13.71 GB, 16.81 GB, 20.06 GB, 22.94 GB, 25.58 GB, 27.75 GB, 30.51 GB, 33.23 GB, 36.01 GB, 38.10 GB, 39.98 GB, 41.93 GB, 43.73 GB, 45.71 GB, 46.50 GB, 46.77 GB, 46.94 GB, 47.37 GB, 47.94 GB, 48.31 GB, 48.80 GB, 49.15 GB, 49.64 GB, 49.90 GB, 50.27 GB, 50.69 GB, 51.52 GB, 53.42 GB, 56.19 GB, 58.59 GB, 61.09 GB, 63.53 GB, 65.94 GB
- BytesReceived: 69.43 GB (74552487138)
- DeserializeRowBatchTimer: 11m42s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.06 MB (10552116)
- SendersBlockedTimer: 11m48s
- SendersBlockedTotalTimer(*): 1h10m
Averaged Fragment F00:(Total: 56.583ms, non-child: 0.000ns, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:52m20s max:52m20s mean: 52m20s stddev:946.336us
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 6
- AverageThreadTokens: 12.08
- BloomFilterBytes: 0
- PeakMemoryUsage: 441.06 MB (462485674)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178567650)
- RowsProduced: 171.36M (171359992)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 31m53s
- TotalStorageWaitTime: 27m58s
- TotalThreadsInvoluntaryContextSwitches: 2.61K (2613)
- TotalThreadsTotalWallClockTime: 3h24m
- TotalThreadsSysTime: 53s015ms
- TotalThreadsUserTime: 1m52s
- TotalThreadsVoluntaryContextSwitches: 101.43K (101426)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s865ms
- OpenTime: 20.219ms
- ExecTreeOpenTime: 20.203ms
- PrepareTime: 36.303ms
- ExecTreePrepareTime: 71.946us
DataStreamSender (dst_id=3):(Total: 17m18s, non-child: 17m18s, % non-child: 100.00%)
- BytesSent: 69.52 GB (74651860734)
- NetworkThroughput(*): 422.54 MB/sec
- OverallThroughput: 72.13 MB/sec
- PeakMemoryUsage: 361.50 KB (370176)
- RowsReturned: 171.36M (171359138)
- SerializeBatchTime: 14m18s
- TransmitDataRPCTime: 2m58s
- UncompressedRowBatchSize: 136.12 GB (146160568107)
CodeGen:(Total: 33.540ms, non-child: 33.540ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 32.419ms
KUDU_SCAN_NODE (id=1):(Total: 1s584ms, non-child: 1s584ms, % non-child: 100.00%)
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 440.71 MB (462120960)
- RowsRead: 171.66M (171656959)
- RowsReturned: 171.36M (171359992)
- RowsReturnedRate: 110.82 M/sec
- ScanRangesComplete: 10 (10)
- ScannerThreadsInvoluntaryContextSwitches: 2.61K (2613)
- ScannerThreadsTotalWallClockTime: 3h24m
- MaterializeTupleTime(*): 1s489ms
- ScannerThreadsSysTime: 53s013ms
- ScannerThreadsUserTime: 1m52s
- ScannerThreadsVoluntaryContextSwitches: 101.39K (101388)
- TotalKuduScanRoundTrips: 132.15K (132149)
- TotalReadThroughput: 0.00 /sec
Fragment F00:
Instance 664a8e28b9a6c9c6:6811bafc00000005 (host=zp-stg-hadoop-34:22000):(Total: 63.725ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(1m4s): 438.23 MB, 442.40 MB, 442.40 MB, 442.40 MB, 442.40 MB, 442.40 MB, 442.40 MB, 442.40 MB, 442.40 MB, 442.40 MB, 442.40 MB, 441.85 MB, 437.87 MB, 437.87 MB, 437.88 MB, 437.88 MB, 437.87 MB, 437.86 MB, 437.86 MB, 437.86 MB, 437.45 MB, 436.37 MB, 436.37 MB, 436.37 MB, 436.38 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.38 MB, 436.40 MB, 436.05 MB, 434.88 MB, 434.89 MB, 434.89 MB, 434.90 MB, 434.89 MB, 434.89 MB, 434.89 MB, 432.39 MB, 430.40 MB, 430.39 MB, 430.37 MB, 430.39 MB, 430.40 MB, 430.39 MB
ThreadUsage(1m4s): 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 14, 12, 12, 12, 12, 12, 12, 12, 12, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 10, 10, 10, 10, 10, 10, 10, 10, 8, 7, 7, 7, 7, 7, 7
- AverageThreadTokens: 11.46
- BloomFilterBytes: 0
- PeakMemoryUsage: 442.56 MB (464062976)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3180574639)
- RowsProduced: 161.46M (161463272)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 31m19s
- TotalStorageWaitTime: 27m11s
- TotalThreadsInvoluntaryContextSwitches: 3.21K (3206)
- TotalThreadsTotalWallClockTime: 3h53m
- TotalThreadsSysTime: 1m1s
- TotalThreadsUserTime: 2m10s
- TotalThreadsVoluntaryContextSwitches: 112.02K (112024)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s924ms
- OpenTime: 23.850ms
- ExecTreeOpenTime: 23.837ms
- PrepareTime: 39.814ms
- ExecTreePrepareTime: 78.493us
DataStreamSender (dst_id=3):(Total: 18m26s, non-child: 18m26s, % non-child: 100.00%)
- BytesSent: 65.57 GB (70405633672)
- NetworkThroughput(*): 350.79 MB/sec
- OverallThroughput: 60.67 MB/sec
- PeakMemoryUsage: 361.50 KB (370176)
- RowsReturned: 161.46M (161462248)
- SerializeBatchTime: 15m13s
- TransmitDataRPCTime: 3m11s
- UncompressedRowBatchSize: 128.29 GB (137748411716)
CodeGen:(Total: 36.337ms, non-child: 36.337ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 35.270ms
KUDU_SCAN_NODE (id=1):(Total: 1s634ms, non-child: 1s634ms, % non-child: 100.00%)
BytesRead(1m4s): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 442.21 MB (463692800)
- RowsRead: 161.76M (161758554)
- RowsReturned: 161.46M (161463272)
- RowsReturnedRate: 98.81 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 3.21K (3206)
- ScannerThreadsTotalWallClockTime: 3h53m
- MaterializeTupleTime(*): 1s543ms
- ScannerThreadsSysTime: 1m1s
- ScannerThreadsUserTime: 2m10s
- ScannerThreadsVoluntaryContextSwitches: 111.99K (111992)
- TotalKuduScanRoundTrips: 124.53K (124528)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc00000003 (host=zp-stg-hadoop-32:22000):(Total: 62.812ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(1m4s): 429.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 434.39 MB, 438.55 MB, 437.87 MB, 437.87 MB, 437.88 MB, 437.88 MB, 437.87 MB, 437.88 MB, 437.89 MB, 437.88 MB, 437.51 MB, 436.39 MB, 436.37 MB, 436.38 MB, 436.38 MB, 436.38 MB, 436.39 MB, 436.39 MB, 436.38 MB, 436.39 MB, 436.39 MB, 436.38 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.39 MB, 436.38 MB, 436.38 MB, 436.38 MB, 435.52 MB, 432.04 MB, 431.87 MB, 431.88 MB, 431.88 MB, 431.89 MB, 431.87 MB
ThreadUsage(1m4s): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 12, 12, 12, 12, 12, 12, 12, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 10, 8, 8, 8, 8, 8, 8
- AverageThreadTokens: 11.29
- BloomFilterBytes: 0
- PeakMemoryUsage: 439.57 MB (460919296)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.95 GB (3172116030)
- RowsProduced: 159.60M (159597263)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 31m4s
- TotalStorageWaitTime: 27m18s
- TotalThreadsInvoluntaryContextSwitches: 2.21K (2210)
- TotalThreadsTotalWallClockTime: 2h51m
- TotalThreadsSysTime: 48s674ms
- TotalThreadsUserTime: 1m41s
- TotalThreadsVoluntaryContextSwitches: 87.07K (87068)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s946ms
- OpenTime: 21.159ms
- ExecTreeOpenTime: 21.143ms
- PrepareTime: 41.599ms
- ExecTreePrepareTime: 67.484us
DataStreamSender (dst_id=3):(Total: 18m22s, non-child: 18m22s, % non-child: 100.00%)
- BytesSent: 64.73 GB (69504908151)
- NetworkThroughput(*): 345.20 MB/sec
- OverallThroughput: 60.10 MB/sec
- PeakMemoryUsage: 361.50 KB (370176)
- RowsReturned: 159.60M (159596239)
- SerializeBatchTime: 15m9s
- TransmitDataRPCTime: 3m12s
- UncompressedRowBatchSize: 126.79 GB (136138678186)
CodeGen:(Total: 38.085ms, non-child: 38.085ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 36.407ms
KUDU_SCAN_NODE (id=1):(Total: 1s653ms, non-child: 1s653ms, % non-child: 100.00%)
BytesRead(1m4s): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 439.21 MB (460549120)
- RowsRead: 159.89M (159893295)
- RowsReturned: 159.60M (159597263)
- RowsReturnedRate: 96.53 M/sec
- ScanRangesComplete: 11 (11)
- ScannerThreadsInvoluntaryContextSwitches: 2.21K (2210)
- ScannerThreadsTotalWallClockTime: 2h51m
- MaterializeTupleTime(*): 1s555ms
- ScannerThreadsSysTime: 48s671ms
- ScannerThreadsUserTime: 1m41s
- ScannerThreadsVoluntaryContextSwitches: 87.03K (87034)
- TotalKuduScanRoundTrips: 123.09K (123094)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc00000006 (host=zp-stg-hadoop-31:22000):(Total: 62.808ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(1m4s): 439.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 442.86 MB, 443.80 MB, 443.87 MB, 443.88 MB, 443.88 MB, 443.88 MB, 443.87 MB, 443.87 MB, 443.85 MB, 443.88 MB, 443.88 MB, 443.89 MB, 443.87 MB, 443.89 MB, 443.19 MB, 442.37 MB, 442.39 MB, 442.38 MB, 442.39 MB, 442.39 MB, 442.39 MB, 442.38 MB, 442.38 MB, 442.36 MB, 442.36 MB, 442.35 MB, 442.38 MB, 442.38 MB, 442.39 MB, 442.39 MB, 442.38 MB, 442.38 MB, 442.39 MB, 442.37 MB, 442.37 MB, 442.38 MB, 442.35 MB, 435.52 MB
ThreadUsage(1m4s): 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 10
- AverageThreadTokens: 15.29
- BloomFilterBytes: 0
- PeakMemoryUsage: 444.06 MB (465634816)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3182379986)
- RowsProduced: 161.96M (161961429)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 30m28s
- TotalStorageWaitTime: 29m50s
- TotalThreadsInvoluntaryContextSwitches: 3.69K (3693)
- TotalThreadsTotalWallClockTime: 5h28m
- TotalThreadsSysTime: 1m19s
- TotalThreadsUserTime: 2m47s
- TotalThreadsVoluntaryContextSwitches: 143.30K (143299)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s962ms
- OpenTime: 21.398ms
- ExecTreeOpenTime: 21.377ms
- PrepareTime: 41.340ms
- ExecTreePrepareTime: 80.076us
DataStreamSender (dst_id=3):(Total: 18m27s, non-child: 18m27s, % non-child: 100.00%)
- BytesSent: 65.67 GB (70511855726)
- NetworkThroughput(*): 349.10 MB/sec
- OverallThroughput: 60.74 MB/sec
- PeakMemoryUsage: 361.50 KB (370176)
- RowsReturned: 161.96M (161960405)
- SerializeBatchTime: 15m13s
- TransmitDataRPCTime: 3m12s
- UncompressedRowBatchSize: 128.62 GB (138105413644)
CodeGen:(Total: 38.502ms, non-child: 38.502ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 37.333ms
KUDU_SCAN_NODE (id=1):(Total: 1s661ms, non-child: 1s661ms, % non-child: 100.00%)
BytesRead(1m4s): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 443.71 MB (465264640)
- RowsRead: 162.26M (162259701)
- RowsReturned: 161.96M (161961429)
- RowsReturnedRate: 97.50 M/sec
- ScanRangesComplete: 10 (10)
- ScannerThreadsInvoluntaryContextSwitches: 3.69K (3693)
- ScannerThreadsTotalWallClockTime: 5h28m
- MaterializeTupleTime(*): 1s564ms
- ScannerThreadsSysTime: 1m19s
- ScannerThreadsUserTime: 2m47s
- ScannerThreadsVoluntaryContextSwitches: 143.27K (143267)
- TotalKuduScanRoundTrips: 124.93K (124926)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc00000002 (host=zp-stg-hadoop-35:22000):(Total: 60.493ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(1m4s): 436.54 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.96 MB, 437.89 MB, 437.88 MB, 437.87 MB, 437.89 MB, 437.86 MB, 437.88 MB, 437.89 MB, 437.87 MB, 437.88 MB, 437.88 MB, 437.88 MB, 437.87 MB, 437.88 MB, 437.88 MB, 437.88 MB, 437.85 MB, 437.88 MB, 437.89 MB, 437.88 MB, 437.89 MB, 437.89 MB, 437.89 MB, 437.89 MB, 437.89 MB, 437.88 MB, 437.87 MB, 437.88 MB, 437.83 MB, 437.89 MB, 437.80 MB, 434.56 MB, 433.39 MB, 432.28 MB, 431.88 MB, 431.89 MB, 431.87 MB, 431.87 MB, 431.89 MB
ThreadUsage(1m4s): 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 11, 9, 9, 8, 8, 8, 8, 8, 8
- AverageThreadTokens: 11.41
- BloomFilterBytes: 0
- PeakMemoryUsage: 438.07 MB (459347456)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3176588076)
- RowsProduced: 157.87M (157868864)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 31m13s
- TotalStorageWaitTime: 25m51s
- TotalThreadsInvoluntaryContextSwitches: 2.49K (2494)
- TotalThreadsTotalWallClockTime: 2h58m
- TotalThreadsSysTime: 48s993ms
- TotalThreadsUserTime: 1m45s
- TotalThreadsVoluntaryContextSwitches: 88.88K (88879)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s914ms
- OpenTime: 22.281ms
- ExecTreeOpenTime: 22.262ms
- PrepareTime: 38.151ms
- ExecTreePrepareTime: 83.419us
DataStreamSender (dst_id=3):(Total: 18m23s, non-child: 18m23s, % non-child: 100.00%)
- BytesSent: 64.04 GB (68757242715)
- NetworkThroughput(*): 345.54 MB/sec
- OverallThroughput: 59.45 MB/sec
- PeakMemoryUsage: 361.50 KB (370176)
- RowsReturned: 157.87M (157867840)
- SerializeBatchTime: 15m11s
- TransmitDataRPCTime: 3m9s
- UncompressedRowBatchSize: 125.37 GB (134617280924)
CodeGen:(Total: 34.710ms, non-child: 34.710ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 33.324ms
KUDU_SCAN_NODE (id=1):(Total: 1s610ms, non-child: 1s610ms, % non-child: 100.00%)
BytesRead(1m4s): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 437.71 MB (458977280)
- RowsRead: 158.17M (158165765)
- RowsReturned: 157.87M (157868864)
- RowsReturnedRate: 98.00 M/sec
- ScanRangesComplete: 11 (11)
- ScannerThreadsInvoluntaryContextSwitches: 2.49K (2494)
- ScannerThreadsTotalWallClockTime: 2h58m
- MaterializeTupleTime(*): 1s498ms
- ScannerThreadsSysTime: 48s984ms
- ScannerThreadsUserTime: 1m45s
- ScannerThreadsVoluntaryContextSwitches: 88.82K (88821)
- TotalKuduScanRoundTrips: 121.79K (121791)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc00000004 (host=zp-stg-hadoop-36:22000):(Total: 56.633ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(1m4s): 439.09 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.87 MB, 440.88 MB, 440.87 MB, 440.89 MB, 440.88 MB, 440.84 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.88 MB, 440.85 MB, 438.45 MB, 436.39 MB, 436.38 MB, 436.39 MB, 436.39 MB, 436.37 MB, 436.38 MB, 436.39 MB, 436.36 MB, 436.35 MB, 436.39 MB, 436.38 MB, 436.37 MB, 436.39 MB, 436.38 MB, 436.40 MB, 436.39 MB, 436.39 MB, 436.38 MB, 436.35 MB, 435.66 MB, 434.88 MB, 433.77 MB, 431.89 MB, 431.87 MB, 431.88 MB, 431.88 MB
ThreadUsage(1m4s): 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 12, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 10, 10, 9, 8, 8, 8, 8
- AverageThreadTokens: 12.06
- BloomFilterBytes: 0
- PeakMemoryUsage: 441.07 MB (462491136)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3181645337)
- RowsProduced: 159.62M (159621563)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 29m49s
- TotalStorageWaitTime: 31m38s
- TotalThreadsInvoluntaryContextSwitches: 2.83K (2828)
- TotalThreadsTotalWallClockTime: 3h32m
- TotalThreadsSysTime: 59s680ms
- TotalThreadsUserTime: 2m8s
- TotalThreadsVoluntaryContextSwitches: 108.33K (108329)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s975ms
- OpenTime: 19.362ms
- ExecTreeOpenTime: 19.347ms
- PrepareTime: 37.215ms
- ExecTreePrepareTime: 84.587us
DataStreamSender (dst_id=3):(Total: 18m6s, non-child: 18m6s, % non-child: 100.00%)
- BytesSent: 64.73 GB (69498815990)
- NetworkThroughput(*): 360.17 MB/sec
- OverallThroughput: 61.00 MB/sec
- PeakMemoryUsage: 361.50 KB (370176)
- RowsReturned: 159.62M (159620539)
- SerializeBatchTime: 15m1s
- TransmitDataRPCTime: 3m4s
- UncompressedRowBatchSize: 126.80 GB (136147019390)
CodeGen:(Total: 35.846ms, non-child: 35.846ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 34.905ms
KUDU_SCAN_NODE (id=1):(Total: 1s675ms, non-child: 1s675ms, % non-child: 100.00%)
BytesRead(1m4s): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 440.71 MB (462120960)
- RowsRead: 159.92M (159918824)
- RowsReturned: 159.62M (159621563)
- RowsReturnedRate: 95.29 M/sec
- ScanRangesComplete: 11 (11)
- ScannerThreadsInvoluntaryContextSwitches: 2.83K (2828)
- ScannerThreadsTotalWallClockTime: 3h32m
- MaterializeTupleTime(*): 1s572ms
- ScannerThreadsSysTime: 59s680ms
- ScannerThreadsUserTime: 2m8s
- ScannerThreadsVoluntaryContextSwitches: 108.30K (108295)
- TotalKuduScanRoundTrips: 123.11K (123113)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc00000001 (host=zp-stg-hadoop-33:22000):(Total: 33.024ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(1m4s): 437.25 MB, 440.91 MB, 440.91 MB, 440.91 MB, 440.91 MB, 440.91 MB, 440.91 MB, 440.91 MB, 440.91 MB, 440.91 MB, 440.91 MB, 439.66 MB, 437.86 MB, 437.88 MB, 437.87 MB, 437.86 MB, 437.89 MB, 437.88 MB, 437.86 MB, 437.89 MB, 437.89 MB, 437.88 MB, 437.88 MB, 436.42 MB, 436.38 MB, 433.95 MB, 433.39 MB, 433.39 MB, 433.38 MB, 433.36 MB, 433.39 MB, 433.40 MB, 433.39 MB, 433.39 MB, 433.38 MB, 433.39 MB, 433.39 MB, 433.38 MB, 433.39 MB, 433.39 MB, 433.37 MB, 433.39 MB, 433.36 MB, 433.39 MB, 433.38 MB, 433.38 MB, 433.40 MB, 433.38 MB
ThreadUsage(1m4s): 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 13, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 11, 11, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9
- AverageThreadTokens: 10.97
- BloomFilterBytes: 0
- PeakMemoryUsage: 441.03 MB (462458368)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178101835)
- RowsProduced: 227.65M (227647561)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 37m24s
- TotalStorageWaitTime: 25m59s
- TotalThreadsInvoluntaryContextSwitches: 1.25K (1247)
- TotalThreadsTotalWallClockTime: 1h43m
- TotalThreadsSysTime: 19s680ms
- TotalThreadsUserTime: 41s852ms
- TotalThreadsVoluntaryContextSwitches: 68.96K (68958)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s470ms
- OpenTime: 13.260ms
- ExecTreeOpenTime: 13.252ms
- PrepareTime: 19.697ms
- ExecTreePrepareTime: 37.621us
DataStreamSender (dst_id=3):(Total: 12m3s, non-child: 12m3s, % non-child: 100.00%)
- BytesSent: 92.42 GB (99232708151)
- NetworkThroughput(*): 784.45 MB/sec
- OverallThroughput: 130.84 MB/sec
- PeakMemoryUsage: 361.50 KB (370176)
- RowsReturned: 227.65M (227647561)
- SerializeBatchTime: 10m1s
- TransmitDataRPCTime: 2m
- UncompressedRowBatchSize: 180.87 GB (194206604787)
CodeGen:(Total: 17.763ms, non-child: 17.763ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 17.275ms
KUDU_SCAN_NODE (id=1):(Total: 1s273ms, non-child: 1s273ms, % non-child: 100.00%)
BytesRead(1m4s): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 440.71 MB (462120960)
- RowsRead: 227.95M (227945617)
- RowsReturned: 227.65M (227647561)
- RowsReturnedRate: 178.78 M/sec
- ScanRangesComplete: 10 (10)
- ScannerThreadsInvoluntaryContextSwitches: 1.25K (1247)
- ScannerThreadsTotalWallClockTime: 1h43m
- MaterializeTupleTime(*): 1s200ms
- ScannerThreadsSysTime: 19s679ms
- ScannerThreadsUserTime: 41s852ms
- ScannerThreadsVoluntaryContextSwitches: 68.92K (68923)
- TotalKuduScanRoundTrips: 175.45K (175447)
- TotalReadThroughput: 0.00 /sec
Averaged Fragment F01:(Total: 9m52s, non-child: 2m56s, % non-child: 29.78%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:8m36s max:11m48s mean: 9m52s stddev:1m22s
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 6
- AverageThreadTokens: 6.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 528.71 MB (554393146)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178567650)
- RowsProduced: 44.72M (44722873)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 2m11s
- TotalStorageWaitTime: 8m20s
- TotalThreadsInvoluntaryContextSwitches: 11.66K (11664)
- TotalThreadsTotalWallClockTime: 59m50s
- TotalThreadsSysTime: 1m29s
- TotalThreadsUserTime: 9m34s
- TotalThreadsVoluntaryContextSwitches: 422.12K (422124)
Fragment Instance Lifecycle Timings:
- ExecTime: 9m51s
- ExecTreeExecTime: 546.207ms
- OpenTime: 20.211ms
- ExecTreeOpenTime: 20.182ms
- PrepareTime: 35.159ms
- ExecTreePrepareTime: 61.857us
DataStreamSender (dst_id=4):(Total: 6m55s, non-child: 6m55s, % non-child: 100.00%)
- BytesSent: 28.28 GB (30362869987)
- NetworkThroughput(*): 426.12 MB/sec
- OverallThroughput: 74.49 MB/sec
- PeakMemoryUsage: 328.89 KB (336784)
- RowsReturned: 44.72M (44722873)
- SerializeBatchTime: 5m41s
- TransmitDataRPCTime: 1m13s
- UncompressedRowBatchSize: 56.73 GB (60913562731)
CodeGen:(Total: 32.769ms, non-child: 32.769ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 31.693ms
KUDU_SCAN_NODE (id=0):(Total: 495.041ms, non-child: 495.041ms, % non-child: 100.00%)
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 12 (12)
- PeakMemoryUsage: 527.81 MB (553446058)
- RowsRead: 44.72M (44722873)
- RowsReturned: 44.72M (44722873)
- RowsReturnedRate: 93.65 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 3.76K (3757)
- ScannerThreadsTotalWallClockTime: 49m58s
- MaterializeTupleTime(*): 442.640ms
- ScannerThreadsSysTime: 1m7s
- ScannerThreadsUserTime: 2m12s
- ScannerThreadsVoluntaryContextSwitches: 114.31K (114312)
- TotalKuduScanRoundTrips: 55.92K (55923)
- TotalReadThroughput: 0.00 /sec
Fragment F01:
Instance 664a8e28b9a6c9c6:6811bafc00000007 (host=zp-stg-hadoop-32:22000):(Total: 11m48s, non-child: 3m12s, % non-child: 27.20%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 502.09 MB, 529.17 MB, 529.18 MB, 529.15 MB, 529.17 MB, 526.29 MB, 525.53 MB, 525.52 MB, 525.52 MB, 524.55 MB, 523.71 MB, 523.71 MB, 523.71 MB, 523.71 MB, 523.70 MB, 523.70 MB, 523.70 MB, 523.71 MB, 523.64 MB, 523.71 MB, 523.70 MB, 523.70 MB, 523.69 MB, 523.66 MB, 523.70 MB, 523.71 MB, 523.70 MB, 523.37 MB, 521.88 MB, 521.87 MB, 521.85 MB, 521.87 MB, 521.87 MB, 521.83 MB, 521.88 MB, 521.88 MB, 521.88 MB, 521.88 MB, 521.89 MB, 521.89 MB, 519.59 MB, 508.02 MB, 516.75 MB, 515.28 MB
ThreadUsage(16s000ms): 10, 10, 10, 10, 10, 8, 8, 8, 8, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 4, 4, 4
- AverageThreadTokens: 6.91
- BloomFilterBytes: 0
- PeakMemoryUsage: 529.34 MB (555058064)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.95 GB (3172116030)
- RowsProduced: 51.29M (51289874)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 2m23s
- TotalStorageWaitTime: 10m58s
- TotalThreadsInvoluntaryContextSwitches: 11.60K (11604)
- TotalThreadsTotalWallClockTime: 1h21m
- TotalThreadsSysTime: 1m49s
- TotalThreadsUserTime: 11m49s
- TotalThreadsVoluntaryContextSwitches: 378.48K (378477)
Fragment Instance Lifecycle Timings:
- ExecTime: 11m48s
- ExecTreeExecTime: 643.393ms
- OpenTime: 23.366ms
- ExecTreeOpenTime: 23.338ms
- PrepareTime: 38.336ms
- ExecTreePrepareTime: 58.844us
DataStreamSender (dst_id=4):(Total: 8m35s, non-child: 8m35s, % non-child: 100.00%)
- BytesSent: 32.42 GB (34810686011)
- NetworkThroughput(*): 354.47 MB/sec
- OverallThroughput: 64.43 MB/sec
- PeakMemoryUsage: 328.89 KB (336784)
- RowsReturned: 51.29M (51289874)
- SerializeBatchTime: 7m1s
- TransmitDataRPCTime: 1m33s
- UncompressedRowBatchSize: 65.06 GB (69858111787)
CodeGen:(Total: 36.048ms, non-child: 36.048ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 34.955ms
KUDU_SCAN_NODE (id=0):(Total: 575.753ms, non-child: 575.753ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 12 (12)
- PeakMemoryUsage: 529.02 MB (554721280)
- RowsRead: 51.29M (51289874)
- RowsReturned: 51.29M (51289874)
- RowsReturnedRate: 89.08 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 3.86K (3858)
- ScannerThreadsTotalWallClockTime: 1h9m
- MaterializeTupleTime(*): 517.582ms
- ScannerThreadsSysTime: 1m23s
- ScannerThreadsUserTime: 2m48s
- ScannerThreadsVoluntaryContextSwitches: 130.16K (130162)
- TotalKuduScanRoundTrips: 64.15K (64148)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc0000000b (host=zp-stg-hadoop-34:22000):(Total: 11m47s, non-child: 3m12s, % non-child: 27.19%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 494.96 MB, 527.35 MB, 527.33 MB, 527.34 MB, 527.33 MB, 527.35 MB, 527.36 MB, 527.35 MB, 527.35 MB, 523.88 MB, 521.88 MB, 521.85 MB, 521.88 MB, 521.84 MB, 521.89 MB, 521.88 MB, 521.83 MB, 521.88 MB, 521.82 MB, 521.87 MB, 521.85 MB, 521.88 MB, 521.88 MB, 521.87 MB, 521.87 MB, 521.88 MB, 521.88 MB, 521.82 MB, 521.88 MB, 521.88 MB, 521.89 MB, 521.70 MB, 521.88 MB, 521.88 MB, 521.89 MB, 521.19 MB, 520.05 MB, 520.06 MB, 520.05 MB, 519.77 MB, 519.86 MB, 520.05 MB, 519.93 MB, 506.88 MB
ThreadUsage(16s000ms): 8, 9, 9, 9, 9, 9, 9, 9, 9, 7, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 5, 5, 5, 5, 5, 5, 5, 3
- AverageThreadTokens: 6.39
- BloomFilterBytes: 0
- PeakMemoryUsage: 527.52 MB (553145232)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3180574639)
- RowsProduced: 51.16M (51155485)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 2m21s
- TotalStorageWaitTime: 11m12s
- TotalThreadsInvoluntaryContextSwitches: 12.87K (12866)
- TotalThreadsTotalWallClockTime: 1h15m
- TotalThreadsSysTime: 1m53s
- TotalThreadsUserTime: 11m48s
- TotalThreadsVoluntaryContextSwitches: 372.06K (372057)
Fragment Instance Lifecycle Timings:
- ExecTime: 11m47s
- ExecTreeExecTime: 659.320ms
- OpenTime: 21.778ms
- ExecTreeOpenTime: 21.744ms
- PrepareTime: 39.124ms
- ExecTreePrepareTime: 61.149us
DataStreamSender (dst_id=4):(Total: 8m34s, non-child: 8m34s, % non-child: 100.00%)
- BytesSent: 32.34 GB (34720563613)
- NetworkThroughput(*): 360.55 MB/sec
- OverallThroughput: 64.37 MB/sec
- PeakMemoryUsage: 328.89 KB (336784)
- RowsReturned: 51.16M (51155485)
- SerializeBatchTime: 7m1s
- TransmitDataRPCTime: 1m31s
- UncompressedRowBatchSize: 64.90 GB (69687968837)
CodeGen:(Total: 35.748ms, non-child: 35.748ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 34.016ms
KUDU_SCAN_NODE (id=0):(Total: 596.194ms, non-child: 596.194ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 12 (12)
- PeakMemoryUsage: 527.20 MB (552808448)
- RowsRead: 51.16M (51155485)
- RowsReturned: 51.16M (51155485)
- RowsReturnedRate: 85.80 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 4.75K (4753)
- ScannerThreadsTotalWallClockTime: 1h3m
- MaterializeTupleTime(*): 540.473ms
- ScannerThreadsSysTime: 1m26s
- ScannerThreadsUserTime: 2m46s
- ScannerThreadsVoluntaryContextSwitches: 129.00K (129002)
- TotalKuduScanRoundTrips: 63.98K (63984)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc0000000c (host=zp-stg-hadoop-36:22000):(Total: 9m9s, non-child: 2m20s, % non-child: 25.62%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 515.24 MB, 531.00 MB, 530.99 MB, 531.00 MB, 530.96 MB, 530.09 MB, 529.17 MB, 529.09 MB, 529.16 MB, 527.12 MB, 521.08 MB, 520.07 MB, 520.06 MB, 520.06 MB, 520.04 MB, 520.07 MB, 520.06 MB, 520.06 MB, 520.07 MB, 520.06 MB, 520.06 MB, 520.05 MB, 520.05 MB, 518.91 MB, 518.24 MB, 518.24 MB, 518.05 MB, 518.23 MB, 517.49 MB, 518.13 MB, 517.15 MB, 510.71 MB, 508.88 MB, 511.28 MB
ThreadUsage(16s000ms): 11, 11, 11, 11, 11, 10, 10, 10, 10, 8, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3
- AverageThreadTokens: 6.16
- BloomFilterBytes: 0
- PeakMemoryUsage: 531.17 MB (556970896)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3181645337)
- RowsProduced: 39.30M (39296380)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 1m32s
- TotalStorageWaitTime: 7m46s
- TotalThreadsInvoluntaryContextSwitches: 13.44K (13443)
- TotalThreadsTotalWallClockTime: 56m23s
- TotalThreadsSysTime: 1m26s
- TotalThreadsUserTime: 9m28s
- TotalThreadsVoluntaryContextSwitches: 345.11K (345113)
Fragment Instance Lifecycle Timings:
- ExecTime: 9m9s
- ExecTreeExecTime: 569.609ms
- OpenTime: 18.819ms
- ExecTreeOpenTime: 18.785ms
- PrepareTime: 36.768ms
- ExecTreePrepareTime: 72.419us
DataStreamSender (dst_id=4):(Total: 6m47s, non-child: 6m47s, % non-child: 100.00%)
- BytesSent: 24.85 GB (26681445625)
- NetworkThroughput(*): 376.51 MB/sec
- OverallThroughput: 62.38 MB/sec
- PeakMemoryUsage: 328.89 KB (336784)
- RowsReturned: 39.30M (39296380)
- SerializeBatchTime: 5m39s
- TransmitDataRPCTime: 1m7s
- UncompressedRowBatchSize: 49.83 GB (53505096512)
CodeGen:(Total: 35.682ms, non-child: 35.682ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 34.687ms
KUDU_SCAN_NODE (id=0):(Total: 513.545ms, non-child: 513.545ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 12 (12)
- PeakMemoryUsage: 530.85 MB (556634112)
- RowsRead: 39.30M (39296380)
- RowsReturned: 39.30M (39296380)
- RowsReturnedRate: 76.52 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 3.84K (3844)
- ScannerThreadsTotalWallClockTime: 47m14s
- MaterializeTupleTime(*): 456.917ms
- ScannerThreadsSysTime: 1m5s
- ScannerThreadsUserTime: 2m11s
- ScannerThreadsVoluntaryContextSwitches: 108.25K (108253)
- TotalKuduScanRoundTrips: 49.13K (49130)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc00000008 (host=zp-stg-hadoop-35:22000):(Total: 9m7s, non-child: 2m8s, % non-child: 23.40%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 504.48 MB, 521.88 MB, 521.87 MB, 521.87 MB, 521.25 MB, 520.06 MB, 520.05 MB, 520.06 MB, 520.03 MB, 520.06 MB, 520.06 MB, 520.06 MB, 520.05 MB, 520.05 MB, 520.06 MB, 520.06 MB, 520.05 MB, 520.06 MB, 520.07 MB, 520.05 MB, 519.94 MB, 520.06 MB, 520.05 MB, 520.06 MB, 520.03 MB, 520.06 MB, 520.05 MB, 520.06 MB, 519.77 MB, 519.60 MB, 520.06 MB, 520.06 MB, 520.06 MB, 463.33 MB
ThreadUsage(16s000ms): 6, 6, 6, 6, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 2
- AverageThreadTokens: 5.05
- BloomFilterBytes: 0
- PeakMemoryUsage: 523.79 MB (549237648)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3176588076)
- RowsProduced: 40.13M (40128675)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 1m24s
- TotalStorageWaitTime: 7m49s
- TotalThreadsInvoluntaryContextSwitches: 12.05K (12054)
- TotalThreadsTotalWallClockTime: 46m9s
- TotalThreadsSysTime: 1m28s
- TotalThreadsUserTime: 9m35s
- TotalThreadsVoluntaryContextSwitches: 307.19K (307187)
Fragment Instance Lifecycle Timings:
- ExecTime: 9m7s
- ExecTreeExecTime: 522.758ms
- OpenTime: 22.104ms
- ExecTreeOpenTime: 22.062ms
- PrepareTime: 37.960ms
- ExecTreePrepareTime: 79.097us
DataStreamSender (dst_id=4):(Total: 6m59s, non-child: 6m59s, % non-child: 100.00%)
- BytesSent: 25.38 GB (27250298589)
- NetworkThroughput(*): 347.39 MB/sec
- OverallThroughput: 61.99 MB/sec
- PeakMemoryUsage: 328.89 KB (336784)
- RowsReturned: 40.13M (40128675)
- SerializeBatchTime: 5m43s
- TransmitDataRPCTime: 1m14s
- UncompressedRowBatchSize: 50.91 GB (54665118206)
CodeGen:(Total: 34.570ms, non-child: 34.570ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 33.606ms
KUDU_SCAN_NODE (id=0):(Total: 473.845ms, non-child: 473.845ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 12 (12)
- PeakMemoryUsage: 521.73 MB (547069952)
- RowsRead: 40.13M (40128675)
- RowsReturned: 40.13M (40128675)
- RowsReturnedRate: 84.69 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 3.41K (3406)
- ScannerThreadsTotalWallClockTime: 37m1s
- MaterializeTupleTime(*): 415.143ms
- ScannerThreadsSysTime: 1m6s
- ScannerThreadsUserTime: 2m11s
- ScannerThreadsVoluntaryContextSwitches: 103.50K (103503)
- TotalKuduScanRoundTrips: 50.18K (50177)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc0000000a (host=zp-stg-hadoop-33:22000):(Total: 8m42s, non-child: 4m41s, % non-child: 53.82%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 505.97 MB, 529.18 MB, 529.14 MB, 529.19 MB, 529.19 MB, 529.18 MB, 526.84 MB, 522.05 MB, 521.88 MB, 521.82 MB, 521.87 MB, 521.89 MB, 521.88 MB, 521.89 MB, 521.89 MB, 521.88 MB, 521.88 MB, 521.88 MB, 521.59 MB, 520.06 MB, 520.07 MB, 520.06 MB, 520.05 MB, 520.06 MB, 520.06 MB, 520.06 MB, 520.06 MB, 520.07 MB, 519.77 MB, 520.06 MB, 520.07 MB, 519.78 MB
ThreadUsage(16s000ms): 10, 10, 10, 10, 10, 10, 8, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 4
- AverageThreadTokens: 6.33
- BloomFilterBytes: 0
- PeakMemoryUsage: 532.96 MB (558850960)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3178101835)
- RowsProduced: 49.40M (49402106)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 4m8s
- TotalStorageWaitTime: 5m18s
- TotalThreadsInvoluntaryContextSwitches: 9.01K (9005)
- TotalThreadsTotalWallClockTime: 55m13s
- TotalThreadsSysTime: 55s698ms
- TotalThreadsUserTime: 5m44s
- TotalThreadsVoluntaryContextSwitches: 839.84K (839837)
Fragment Instance Lifecycle Timings:
- ExecTime: 8m42s
- ExecTreeExecTime: 358.184ms
- OpenTime: 13.174ms
- ExecTreeOpenTime: 13.168ms
- PrepareTime: 19.290ms
- ExecTreePrepareTime: 22.319us
DataStreamSender (dst_id=4):(Total: 4m1s, non-child: 4m1s, % non-child: 100.00%)
- BytesSent: 31.25 GB (33552637894)
- NetworkThroughput(*): 767.14 MB/sec
- OverallThroughput: 132.74 MB/sec
- PeakMemoryUsage: 328.89 KB (336784)
- RowsReturned: 49.40M (49402106)
- SerializeBatchTime: 3m19s
- TransmitDataRPCTime: 41s711ms
- UncompressedRowBatchSize: 62.66 GB (67285813834)
CodeGen:(Total: 17.435ms, non-child: 17.435ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 17.006ms
KUDU_SCAN_NODE (id=0):(Total: 333.425ms, non-child: 333.425ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 12 (12)
- PeakMemoryUsage: 530.85 MB (556634112)
- RowsRead: 49.40M (49402106)
- RowsReturned: 49.40M (49402106)
- RowsReturnedRate: 148.17 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 3.23K (3234)
- ScannerThreadsTotalWallClockTime: 46m31s
- MaterializeTupleTime(*): 302.141ms
- ScannerThreadsSysTime: 42s007ms
- ScannerThreadsUserTime: 1m18s
- ScannerThreadsVoluntaryContextSwitches: 118.59K (118590)
- TotalKuduScanRoundTrips: 61.76K (61764)
- TotalReadThroughput: 0.00 /sec
Instance 664a8e28b9a6c9c6:6811bafc00000009 (host=zp-stg-hadoop-31:22000):(Total: 8m36s, non-child: 2m2s, % non-child: 23.71%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 500.01 MB, 525.52 MB, 525.52 MB, 525.53 MB, 523.81 MB, 523.71 MB, 523.30 MB, 520.74 MB, 520.06 MB, 520.04 MB, 520.07 MB, 520.05 MB, 520.05 MB, 520.05 MB, 520.00 MB, 520.05 MB, 520.05 MB, 520.04 MB, 520.06 MB, 519.31 MB, 518.20 MB, 518.23 MB, 518.23 MB, 518.24 MB, 518.23 MB, 518.06 MB, 517.43 MB, 518.23 MB, 517.84 MB, 517.88 MB, 518.00 MB, 517.73 MB
ThreadUsage(16s000ms): 8, 8, 8, 8, 7, 7, 6, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 3
- AverageThreadTokens: 5.16
- BloomFilterBytes: 0
- PeakMemoryUsage: 527.47 MB (553096080)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 2.96 GB (3182379986)
- RowsProduced: 37.06M (37064722)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 1m21s
- TotalStorageWaitTime: 6m57s
- TotalThreadsInvoluntaryContextSwitches: 11.02K (11016)
- TotalThreadsTotalWallClockTime: 44m21s
- TotalThreadsSysTime: 1m21s
- TotalThreadsUserTime: 8m57s
- TotalThreadsVoluntaryContextSwitches: 290.07K (290075)
Fragment Instance Lifecycle Timings:
- ExecTime: 8m36s
- ExecTreeExecTime: 523.980ms
- OpenTime: 22.023ms
- ExecTreeOpenTime: 21.995ms
- PrepareTime: 39.480ms
- ExecTreePrepareTime: 77.316us
DataStreamSender (dst_id=4):(Total: 6m33s, non-child: 6m33s, % non-child: 100.00%)
- BytesSent: 23.43 GB (25161588194)
- NetworkThroughput(*): 350.69 MB/sec
- OverallThroughput: 61.00 MB/sec
- PeakMemoryUsage: 328.89 KB (336784)
- RowsReturned: 37.06M (37064722)
- SerializeBatchTime: 5m24s
- TransmitDataRPCTime: 1m8s
- UncompressedRowBatchSize: 47.01 GB (50479267214)
CodeGen:(Total: 37.130ms, non-child: 37.130ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 35.888ms
KUDU_SCAN_NODE (id=0):(Total: 477.482ms, non-child: 477.482ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 12 (12)
- PeakMemoryUsage: 527.20 MB (552808448)
- RowsRead: 37.06M (37064722)
- RowsReturned: 37.06M (37064722)
- RowsReturnedRate: 77.63 M/sec
- ScanRangesComplete: 12 (12)
- ScannerThreadsInvoluntaryContextSwitches: 3.45K (3452)
- ScannerThreadsTotalWallClockTime: 35m44s
- MaterializeTupleTime(*): 423.586ms
- ScannerThreadsSysTime: 1m1s
- ScannerThreadsUserTime: 2m
- ScannerThreadsVoluntaryContextSwitches: 96.37K (96367)
- TotalKuduScanRoundTrips: 46.34K (46335)
- TotalReadThroughput: 0.00 /sec
Raw
sql2
Query (id=f947e489a0df32c4:58c89d8300000000):
Summary:
Session ID: ea4138a3623baeac:87c1a1b36b771287
Session Type: BEESWAX
Start Time: 2018-01-05 17:09:34.271187000
End Time: 2018-01-05 17:30:40.319488000
Query Type: QUERY
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.10.0-cdh5.13.0 RELEASE (build 2511805f1eaa991df1460276c7e9f19d819cd4e4)
User: apprun
Connected User: apprun
Delegated User:
Network Address: 10.205.58.34:43384
Default Db: kudu_6
Sql Statement: select * from app_fact_event_base_spark_time_final_4_6_6 e left join /* +SHUFFLE */ app_fact_event_base_spark_time_final_4_6_6_plain s on e.eventname=s.eventname
Coordinator: zp-stg-hadoop-34:22000
Query Options (set by configuration): MAX_IO_BUFFERS=10,REQUEST_POOL=dev,QUERY_TIMEOUT_S=30
Query Options (set by configuration and planner): MAX_IO_BUFFERS=10,REQUEST_POOL=dev,QUERY_TIMEOUT_S=30,MT_DOP=0
Plan:
----------------
Max Per-Host Resource Reservation: Memory=34.00MB
Per-Host Resource Estimates: Memory=14.79GB
F03:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=0B mem-reservation=0B
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B
|
05:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=0,1N row-size=1.20KB cardinality=15895644927818
|
F02:PLAN FRAGMENT [HASH(e.eventname)] hosts=6 instances=6
Per-Host Resources: mem-estimate=14.79GB mem-reservation=34.00MB
02:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| hash predicates: e.eventname = s.eventname
| fk/pk conjuncts: none
| mem-estimate=14.79GB mem-reservation=34.00MB spill-buffer=2.00MB
| tuple-ids=0,1N row-size=1.20KB cardinality=15895644927818
|
|--04:EXCHANGE [HASH(s.eventname)]
| | mem-estimate=0B mem-reservation=0B
| | tuple-ids=1 row-size=182B cardinality=477165387
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=6 instances=6
| Per-Host Resources: mem-estimate=0B mem-reservation=0B
| 01:SCAN KUDU [kudu_6.app_fact_event_base_spark_time_final_4_6_6_plain s]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=1 row-size=182B cardinality=477165387
|
03:EXCHANGE [HASH(e.eventname)]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=0 row-size=1.02KB cardinality=2907894787
|
F00:PLAN FRAGMENT [RANDOM] hosts=6 instances=6
Per-Host Resources: mem-estimate=0B mem-reservation=0B
00:SCAN KUDU [kudu_6.app_fact_event_base_spark_time_final_4_6_6 e]
mem-estimate=0B mem-reservation=0B
tuple-ids=0 row-size=1.02KB cardinality=2907894787
----------------
Estimated Per-Host Mem: 15883871028
Per Host Min Reservation: zp-stg-hadoop-31:22000(34.00 MB) zp-stg-hadoop-32:22000(34.00 MB) zp-stg-hadoop-33:22000(34.00 MB) zp-stg-hadoop-34:22000(34.00 MB) zp-stg-hadoop-35:22000(34.00 MB) zp-stg-hadoop-36:22000(34.00 MB)
Request Pool: root.dev
Admission result: Admitted immediately
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
--------------------------------------------------------------------------------------------------------------------------------------------------------
05:EXCHANGE 1 20m11s 20m11s 34.82K 15895.64B 0 0 UNPARTITIONED
02:HASH JOIN 6 1m42s 5m2s 102.40K 15895.64B 45.21 GB 14.79 GB LEFT OUTER JOIN, PARTITIONED
|--04:EXCHANGE 6 18m35s 19m36s 477.17M 477.17M 0 0 HASH(s.eventname)
| 01:SCAN KUDU 5 2s983ms 5s767ms 477.17M 477.17M 82.63 MB 0 kudu_6.app_fact_event_base_spark_time_final_4_6_6_plain s
03:EXCHANGE 6 479.418us 1.990ms 6.14K 2.91B 0 0 HASH(e.eventname)
00:SCAN KUDU 6 39.863ms 54.494ms 35.84K 2.91B 487.32 MB 0 kudu_6.app_fact_event_base_spark_time_final_4_6_6 e
Errors:
Planner Timeline: 90.279ms
- Analysis finished: 9.902ms (9.902ms)
- Equivalence classes computed: 17.171ms (7.269ms)
- Single node plan created: 81.338ms (64.166ms)
- Runtime filters computed: 81.625ms (287.193us)
- Distributed plan created: 82.665ms (1.040ms)
- Planning finished: 90.279ms (7.614ms)
Query Timeline: 21m6s
- Query submitted: 66.486us (66.486us)
- Planning finished: 95.792ms (95.726ms)
- Submit for admission: 97.556ms (1.763ms)
- Completed admission: 97.607ms (51.203us)
- Ready to start on 6 backends: 98.786ms (1.179ms)
- All 6 execution backends (18 fragment instances) started: 105.399ms (6.612ms)
- Rows available: 20m11s (20m11s)
- First row fetched: 20m11s (181.439ms)
- Unregister query: 21m6s (54s096ms)
- ComputeScanRangeAssignmentTimer: 577.437us
ImpalaServer:
- ClientFetchWaitTimer: 53s339ms
- RowMaterializationTimer: 937.306ms
Execution Profile f947e489a0df32c4:58c89d8300000000:(Total: 20m12s, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 0
Filter routing table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
Backend startup latencies: Count: 6, min / max: 1ms / 6ms, 25th %-ile: 2ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 2ms, 95th %-ile: 6ms, 99.9th %-ile: 6ms
Per Node Peak Memory Usage: zp-stg-hadoop-31:22000(11.61 GB) zp-stg-hadoop-32:22000(13.94 GB) zp-stg-hadoop-33:22000(13.12 GB) zp-stg-hadoop-34:22000(8.07 GB) zp-stg-hadoop-35:22000(16.50 GB) zp-stg-hadoop-36:22000(45.70 GB)
- FiltersReceived: 0 (0)
- FinalizationTimer: 0.000ns
Averaged Fragment F03:(Total: 20m11s, non-child: 0.000ns, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:21m5s max:21m5s mean: 21m5s stddev:0.000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 10.81 MB (11330830)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 8.07 GB (8660998263)
- RowsProduced: 34.82K (34816)
- TotalNetworkReceiveTime: 20m11s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 20m11s
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 1 (1)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1.982ms
- OpenTime: 20m11s
- ExecTreeOpenTime: 20m11s
- PrepareTime: 45.183ms
- ExecTreePrepareTime: 41.500us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 44.616ms, non-child: 44.616ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 43.310ms
EXCHANGE_NODE (id=5):(Total: 20m11s, non-child: 20m11s, % non-child: 100.00%)
- ConvertRowBatchTime: 1.167ms
- PeakMemoryUsage: 0
- RowsReturned: 34.82K (34816)
- RowsReturnedRate: 28.00 /sec
DataStreamReceiver:
- BytesReceived: 10.09 MB (10584000)
- DeserializeRowBatchTimer: 48.008ms
- FirstBatchArrivalWaitTime: 20m11s
- PeakMemoryUsage: 10.62 MB (11132796)
- SendersBlockedTimer: 51s345ms
- SendersBlockedTotalTimer(*): 4m11s
Coordinator Fragment F03:
Instance f947e489a0df32c4:58c89d8300000000 (host=zp-stg-hadoop-34:22000):(Total: 20m11s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 1.53 MB, 10.77 MB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 10.81 MB (11330830)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 8.07 GB (8660998263)
- RowsProduced: 34.82K (34816)
- TotalNetworkReceiveTime: 20m11s
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 20m11s
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 1 (1)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1.982ms
- OpenTime: 20m11s
- ExecTreeOpenTime: 20m11s
- PrepareTime: 45.183ms
- ExecTreePrepareTime: 41.500us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 44.616ms, non-child: 44.616ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 43.310ms
EXCHANGE_NODE (id=5):(Total: 20m11s, non-child: 20m11s, % non-child: 100.00%)
- ConvertRowBatchTime: 1.167ms
- PeakMemoryUsage: 0
- RowsReturned: 34.82K (34816)
- RowsReturnedRate: 28.00 /sec
DataStreamReceiver:
BytesReceived(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 909.83 KB, 7.94 MB
- BytesReceived: 10.09 MB (10584000)
- DeserializeRowBatchTimer: 48.008ms
- FirstBatchArrivalWaitTime: 20m11s
- PeakMemoryUsage: 10.62 MB (11132796)
- SendersBlockedTimer: 51s345ms
- SendersBlockedTotalTimer(*): 4m11s
Averaged Fragment F02:(Total: 20m18s, non-child: 0.000ns, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:21m5s max:21m5s mean: 21m5s stddev:1.373ms
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 6
- AverageThreadTokens: 1.96
- BloomFilterBytes: 0
- PeakMemoryUsage: 17.68 GB (18981133532)
- PeakReservation: 17.67 GB (18975730346)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 18.16 GB (19494499008)
- RowsProduced: 17.07K (17066)
- TotalNetworkReceiveTime: 18m31s
- TotalNetworkSendTime: 40s648ms
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 3.95K (3953)
- TotalThreadsTotalWallClockTime: 40m36s
- TotalThreadsSysTime: 56s596ms
- TotalThreadsUserTime: 38s993ms
- TotalThreadsVoluntaryContextSwitches: 422.61K (422607)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 2.639ms
- OpenTime: 20m18s
- ExecTreeOpenTime: 20m17s
- PrepareTime: 53.756ms
- ExecTreePrepareTime: 377.629us
DataStreamSender (dst_id=5):(Total: 28.639ms, non-child: 28.639ms, % non-child: 100.00%)
- BytesSent: 1.79 MB (1879418)
- NetworkThroughput(*): 301.27 MB/sec
- OverallThroughput: 60.88 MB/sec
- PeakMemoryUsage: 336.00 B (336)
- RowsReturned: 16.04K (16042)
- SerializeBatchTime: 22.921ms
- TransmitDataRPCTime: 6.173ms
- UncompressedRowBatchSize: 3.03 MB (3181186)
CodeGen:(Total: 629.104ms, non-child: 629.104ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 309.680ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 57 (57)
- NumInstructions: 2.60K (2602)
- OptimizationTime: 273.173ms
- PeakMemoryUsage: 1.27 MB (1332224)
- PrepareTime: 45.256ms
HASH_JOIN_NODE (id=2):(Total: 20m17s, non-child: 1m42s, % non-child: 8.41%)
- BuildRows: 79.53M (79527564)
- BuildTime: 1m40s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 17.67 GB (18976796099)
- ProbeRows: 1.02K (1024)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 2.572ms
- RowsReturned: 17.07K (17066)
- RowsReturnedRate: 13.00 /sec
Buffer pool:
- AllocTime: 1s581ms
- CumulativeAllocationBytes: 17.67 GB (18975730346)
- CumulativeAllocations: 7.86K (7857)
- PeakReservation: 17.67 GB (18975730346)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 17.67 GB (18975730346)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
- BuildRowsPartitionTime: 1m28s
- BuildRowsPartitioned: 79.53M (79527564)
- HashBuckets: 158.16M (158160213)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 12s531ms
- LargestPartitionPercent: 25 (25)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.12 KB (29824)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 18m35s, non-child: 18m35s, % non-child: 100.00%)
- ConvertRowBatchTime: 2s738ms
- PeakMemoryUsage: 0
- RowsReturned: 79.53M (79527564)
- RowsReturnedRate: 76.41 K/sec
DataStreamReceiver:
- BytesReceived: 8.33 GB (8942553389)
- DeserializeRowBatchTimer: 3m8s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 4.17 MB (4373620)
- SendersBlockedTimer: 225.782ms
- SendersBlockedTotalTimer(*): 1s127ms
EXCHANGE_NODE (id=3):(Total: 479.418us, non-child: 479.418us, % non-child: 100.00%)
- ConvertRowBatchTime: 66.349us
- PeakMemoryUsage: 0
- RowsReturned: 1.02K (1024)
- RowsReturnedRate: 5.00 M/sec
DataStreamReceiver:
- BytesReceived: 2.35 MB (2461613)
- DeserializeRowBatchTimer: 42.144ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 4.65 MB (4871560)
- SendersBlockedTimer: 3m24s
- SendersBlockedTotalTimer(*): 20m28s
Fragment F02:
Instance f947e489a0df32c4:58c89d830000000d (host=zp-stg-hadoop-36:22000):(Total: 20m29s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 395.95 MB, 759.46 MB, 1023.76 MB, 1.18 GB, 1.42 GB, 1.74 GB, 2.52 GB, 3.31 GB, 3.97 GB, 4.45 GB, 4.88 GB, 5.52 GB, 6.23 GB, 6.69 GB, 7.06 GB, 7.75 GB, 8.42 GB, 8.77 GB, 9.06 GB, 9.50 GB, 10.05 GB, 10.58 GB, 11.11 GB, 11.91 GB, 13.23 GB, 15.16 GB, 18.23 GB, 21.37 GB, 24.11 GB, 26.59 GB, 28.75 GB, 30.49 GB, 31.34 GB, 32.09 GB, 32.79 GB, 33.50 GB, 34.26 GB, 36.03 GB, 44.36 GB
ThreadUsage(32s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1
- AverageThreadTokens: 1.97
- BloomFilterBytes: 0
- PeakMemoryUsage: 45.22 GB (48556497962)
- PeakReservation: 45.21 GB (48544874496)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 45.70 GB (49069035690)
- RowsProduced: 5.12K (5120)
- TotalNetworkReceiveTime: 15m15s
- TotalNetworkSendTime: 30s802ms
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 7.45K (7450)
- TotalThreadsTotalWallClockTime: 40m56s
- TotalThreadsSysTime: 2m45s
- TotalThreadsUserTime: 1m31s
- TotalThreadsVoluntaryContextSwitches: 969.72K (969716)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 356.974us
- OpenTime: 20m28s
- ExecTreeOpenTime: 20m27s
- PrepareTime: 48.943ms
- ExecTreePrepareTime: 448.899us
DataStreamSender (dst_id=5):(Total: 11.415ms, non-child: 11.415ms, % non-child: 100.00%)
- BytesSent: 555.25 KB (568578)
- NetworkThroughput(*): 89.60 MB/sec
- OverallThroughput: 47.50 MB/sec
- PeakMemoryUsage: 336.00 B (336)
- RowsReturned: 4.10K (4096)
- SerializeBatchTime: 6.606ms
- TransmitDataRPCTime: 6.051ms
- UncompressedRowBatchSize: 932.88 KB (955271)
CodeGen:(Total: 1s509ms, non-child: 1s509ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 1s214ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 57 (57)
- NumInstructions: 2.60K (2602)
- OptimizationTime: 252.520ms
- PeakMemoryUsage: 1.27 MB (1332224)
- PrepareTime: 42.267ms
HASH_JOIN_NODE (id=2):(Total: 20m27s, non-child: 5m2s, % non-child: 24.68%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 207.35M (207346050)
- BuildTime: 4m57s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 45.21 GB (48545929509)
- ProbeRows: 1.02K (1024)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 324.311us
- RowsReturned: 5.12K (5120)
- RowsReturnedRate: 4.00 /sec
Buffer pool:
- AllocTime: 6s295ms
- CumulativeAllocationBytes: 45.21 GB (48544874496)
- CumulativeAllocations: 20.37K (20368)
- PeakReservation: 45.21 GB (48544874496)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 45.21 GB (48544874496)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
- BuildRowsPartitionTime: 4m34s
- BuildRowsPartitioned: 207.35M (207346050)
- HashBuckets: 366.48M (366477312)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 23s125ms
- LargestPartitionPercent: 41 (41)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.12 KB (29824)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 15m24s, non-child: 15m24s, % non-child: 100.00%)
- ConvertRowBatchTime: 7s058ms
- PeakMemoryUsage: 0
- RowsReturned: 207.35M (207346050)
- RowsReturnedRate: 224.27 K/sec
DataStreamReceiver:
BytesReceived(32s000ms): 226.39 MB, 450.16 MB, 611.75 MB, 727.93 MB, 877.49 MB, 1.05 GB, 1.53 GB, 2.02 GB, 2.42 GB, 2.72 GB, 2.98 GB, 3.37 GB, 3.80 GB, 4.09 GB, 4.32 GB, 4.74 GB, 5.15 GB, 5.37 GB, 5.54 GB, 5.81 GB, 6.15 GB, 6.48 GB, 6.81 GB, 7.29 GB, 8.10 GB, 9.29 GB, 11.17 GB, 13.10 GB, 14.78 GB, 16.30 GB, 17.63 GB, 18.70 GB, 19.22 GB, 19.68 GB, 20.11 GB, 20.54 GB, 21.01 GB, 21.45 GB
- BytesReceived: 21.54 GB (23128477348)
- DeserializeRowBatchTimer: 6m50s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.22 MB (10715306)
- SendersBlockedTimer: 1s354ms
- SendersBlockedTotalTimer(*): 6s765ms
EXCHANGE_NODE (id=3):(Total: 1.990ms, non-child: 1.990ms, % non-child: 100.00%)
- ConvertRowBatchTime: 82.257us
- PeakMemoryUsage: 0
- RowsReturned: 1.02K (1024)
- RowsReturnedRate: 514.38 K/sec
DataStreamReceiver:
BytesReceived(32s000ms): 4.60 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.66 MB, 4.93 MB
- BytesReceived: 5.10 MB (5348360)
- DeserializeRowBatchTimer: 74.278ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 10.05 MB (10541672)
- SendersBlockedTimer: 20m28s
- SendersBlockedTotalTimer(*): 2h2m
Instance f947e489a0df32c4:58c89d8300000010 (host=zp-stg-hadoop-35:22000):(Total: 20m23s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 152.34 MB, 274.74 MB, 366.54 MB, 431.86 MB, 515.53 MB, 626.79 MB, 900.05 MB, 1.16 GB, 1.39 GB, 1.55 GB, 1.70 GB, 1.92 GB, 2.17 GB, 2.33 GB, 2.46 GB, 2.70 GB, 2.93 GB, 3.05 GB, 3.15 GB, 3.30 GB, 3.50 GB, 3.68 GB, 3.87 GB, 4.14 GB, 4.59 GB, 5.26 GB, 6.32 GB, 7.41 GB, 8.36 GB, 9.23 GB, 9.98 GB, 10.59 GB, 10.89 GB, 11.15 GB, 11.39 GB, 11.63 GB, 11.90 GB, 12.64 GB, 15.95 GB
ThreadUsage(32s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1
- AverageThreadTokens: 1.97
- BloomFilterBytes: 0
- PeakMemoryUsage: 16.02 GB (17204333963)
- PeakReservation: 16.02 GB (17198743552)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 16.50 GB (17715128843)
- RowsProduced: 7.17K (7168)
- TotalNetworkReceiveTime: 19m12s
- TotalNetworkSendTime: 37s825ms
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 3.99K (3988)
- TotalThreadsTotalWallClockTime: 40m46s
- TotalThreadsSysTime: 32s123ms
- TotalThreadsUserTime: 39s479ms
- TotalThreadsVoluntaryContextSwitches: 407.29K (407293)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 405.588us
- OpenTime: 20m23s
- ExecTreeOpenTime: 20m23s
- PrepareTime: 69.062ms
- ExecTreePrepareTime: 454.078us
DataStreamSender (dst_id=5):(Total: 15.394ms, non-child: 15.394ms, % non-child: 100.00%)
- BytesSent: 793.37 KB (812406)
- NetworkThroughput(*): 144.01 MB/sec
- OverallThroughput: 50.33 MB/sec
- PeakMemoryUsage: 336.00 B (336)
- RowsReturned: 6.14K (6144)
- SerializeBatchTime: 10.611ms
- TransmitDataRPCTime: 5.379ms
- UncompressedRowBatchSize: 1.28 MB (1344143)
CodeGen:(Total: 514.392ms, non-child: 514.392ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 127.253ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 57 (57)
- NumInstructions: 2.60K (2602)
- OptimizationTime: 329.049ms
- PeakMemoryUsage: 1.27 MB (1332224)
- PrepareTime: 56.698ms
HASH_JOIN_NODE (id=2):(Total: 20m23s, non-child: 1m7s, % non-child: 5.51%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 71.44M (71438389)
- BuildTime: 1m6s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 16.02 GB (17199801043)
- ProbeRows: 1.02K (1024)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 354.502us
- RowsReturned: 7.17K (7168)
- RowsReturnedRate: 5.00 /sec
Buffer pool:
- AllocTime: 930.106ms
- CumulativeAllocationBytes: 16.02 GB (17198743552)
- CumulativeAllocations: 7.08K (7077)
- PeakReservation: 16.02 GB (17198743552)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 16.02 GB (17198743552)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
- BuildRowsPartitionTime: 48s347ms
- BuildRowsPartitioned: 71.44M (71438389)
- HashBuckets: 149.42M (149422080)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 17s767ms
- LargestPartitionPercent: 24 (24)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.12 KB (29824)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 19m15s, non-child: 19m15s, % non-child: 100.00%)
- ConvertRowBatchTime: 2s652ms
- PeakMemoryUsage: 0
- RowsReturned: 71.44M (71438389)
- RowsReturnedRate: 61.81 K/sec
DataStreamReceiver:
BytesReceived(32s000ms): 78.81 MB, 156.56 MB, 212.64 MB, 252.54 MB, 304.11 MB, 372.11 MB, 540.56 MB, 713.92 MB, 859.20 MB, 965.14 MB, 1.03 GB, 1.17 GB, 1.32 GB, 1.42 GB, 1.50 GB, 1.64 GB, 1.79 GB, 1.86 GB, 1.93 GB, 2.02 GB, 2.14 GB, 2.25 GB, 2.36 GB, 2.53 GB, 2.81 GB, 3.22 GB, 3.87 GB, 4.54 GB, 5.12 GB, 5.66 GB, 6.12 GB, 6.49 GB, 6.68 GB, 6.83 GB, 6.98 GB, 7.14 GB, 7.30 GB, 7.45 GB
- BytesReceived: 7.48 GB (8034736012)
- DeserializeRowBatchTimer: 2m6s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 4.10 MB (4302188)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=3):(Total: 211.278us, non-child: 211.278us, % non-child: 100.00%)
- ConvertRowBatchTime: 61.071us
- PeakMemoryUsage: 0
- RowsReturned: 1.02K (1024)
- RowsReturnedRate: 4.85 M/sec
DataStreamReceiver:
BytesReceived(32s000ms): 2.16 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.18 MB, 2.30 MB
- BytesReceived: 2.39 MB (2504280)
- DeserializeRowBatchTimer: 38.528ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 4.80 MB (5033660)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Instance f947e489a0df32c4:58c89d830000000c (host=zp-stg-hadoop-33:22000):(Total: 20m17s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 120.39 MB, 218.91 MB, 289.69 MB, 339.17 MB, 405.56 MB, 491.53 MB, 704.44 MB, 925.53 MB, 1.08 GB, 1.21 GB, 1.33 GB, 1.50 GB, 1.69 GB, 1.82 GB, 1.92 GB, 2.10 GB, 2.29 GB, 2.38 GB, 2.46 GB, 2.58 GB, 2.73 GB, 2.87 GB, 3.02 GB, 3.23 GB, 3.58 GB, 4.10 GB, 4.93 GB, 5.78 GB, 6.52 GB, 7.19 GB, 7.77 GB, 8.25 GB, 8.48 GB, 8.68 GB, 8.87 GB, 9.06 GB, 9.26 GB, 9.95 GB, 12.64 GB
ThreadUsage(32s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1
- AverageThreadTokens: 1.96
- BloomFilterBytes: 0
- PeakMemoryUsage: 12.65 GB (13581274828)
- PeakReservation: 12.64 GB (13576962048)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 13.12 GB (14092036940)
- RowsProduced: 8.19K (8192)
- TotalNetworkReceiveTime: 19m23s
- TotalNetworkSendTime: 39s263ms
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 3.38K (3382)
- TotalThreadsTotalWallClockTime: 40m34s
- TotalThreadsSysTime: 32s315ms
- TotalThreadsUserTime: 19s905ms
- TotalThreadsVoluntaryContextSwitches: 333.01K (333014)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 475.241us
- OpenTime: 20m17s
- ExecTreeOpenTime: 20m17s
- PrepareTime: 25.570ms
- ExecTreePrepareTime: 393.729us
DataStreamSender (dst_id=5):(Total: 9.952ms, non-child: 9.952ms, % non-child: 100.00%)
- BytesSent: 886.19 KB (907455)
- NetworkThroughput(*): 149.30 MB/sec
- OverallThroughput: 86.95 MB/sec
- PeakMemoryUsage: 336.00 B (336)
- RowsReturned: 7.17K (7168)
- SerializeBatchTime: 4.924ms
- TransmitDataRPCTime: 5.796ms
- UncompressedRowBatchSize: 1.47 MB (1546394)
CodeGen:(Total: 310.494ms, non-child: 310.494ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 108.331ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 57 (57)
- NumInstructions: 2.60K (2602)
- OptimizationTime: 180.357ms
- PeakMemoryUsage: 1.27 MB (1332224)
- PrepareTime: 21.384ms
HASH_JOIN_NODE (id=2):(Total: 20m17s, non-child: 52s542ms, % non-child: 4.32%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 56.43M (56426036)
- BuildTime: 52s083ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 12.65 GB (13578040420)
- ProbeRows: 1.02K (1024)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 438.822us
- RowsReturned: 8.19K (8192)
- RowsReturnedRate: 6.00 /sec
Buffer pool:
- AllocTime: 456.789ms
- CumulativeAllocationBytes: 12.64 GB (13576962048)
- CumulativeAllocations: 5.52K (5522)
- PeakReservation: 12.64 GB (13576962048)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 12.64 GB (13576962048)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
- BuildRowsPartitionTime: 40s442ms
- BuildRowsPartitioned: 56.43M (56426036)
- HashBuckets: 126.88M (126877696)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 11s618ms
- LargestPartitionPercent: 23 (23)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.12 KB (29824)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 19m24s, non-child: 19m24s, % non-child: 100.00%)
- ConvertRowBatchTime: 1s120ms
- PeakMemoryUsage: 0
- RowsReturned: 56.43M (56426036)
- RowsReturnedRate: 48.45 K/sec
DataStreamReceiver:
BytesReceived(32s000ms): 61.94 MB, 122.68 MB, 166.62 MB, 197.92 MB, 238.23 MB, 291.70 MB, 423.65 MB, 559.33 MB, 673.12 MB, 755.95 MB, 829.08 MB, 937.81 MB, 1.03 GB, 1.11 GB, 1.17 GB, 1.29 GB, 1.40 GB, 1.46 GB, 1.51 GB, 1.58 GB, 1.67 GB, 1.76 GB, 1.85 GB, 1.98 GB, 2.20 GB, 2.52 GB, 3.03 GB, 3.56 GB, 4.01 GB, 4.43 GB, 4.79 GB, 5.08 GB, 5.22 GB, 5.35 GB, 5.46 GB, 5.58 GB, 5.71 GB, 5.83 GB
- BytesReceived: 5.85 GB (6285612161)
- DeserializeRowBatchTimer: 1m43s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 2.81 MB (2943193)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=3):(Total: 141.644us, non-child: 141.644us, % non-child: 100.00%)
- ConvertRowBatchTime: 53.274us
- PeakMemoryUsage: 0
- RowsReturned: 1.02K (1024)
- RowsReturnedRate: 7.23 M/sec
DataStreamReceiver:
BytesReceived(32s000ms): 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.71 MB, 1.81 MB
- BytesReceived: 1.87 MB (1963735)
- DeserializeRowBatchTimer: 14.028ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 3.69 MB (3868444)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Instance f947e489a0df32c4:58c89d8300000011 (host=zp-stg-hadoop-32:22000):(Total: 20m16s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 129.21 MB, 232.17 MB, 308.77 MB, 361.03 MB, 430.13 MB, 525.20 MB, 751.91 MB, 986.71 MB, 1.15 GB, 1.29 GB, 1.42 GB, 1.60 GB, 1.80 GB, 1.94 GB, 2.04 GB, 2.24 GB, 2.43 GB, 2.54 GB, 2.62 GB, 2.74 GB, 2.90 GB, 3.06 GB, 3.21 GB, 3.44 GB, 3.81 GB, 4.37 GB, 5.25 GB, 6.15 GB, 6.94 GB, 7.66 GB, 8.28 GB, 8.78 GB, 9.03 GB, 9.24 GB, 9.44 GB, 9.64 GB, 9.86 GB, 10.68 GB, 13.46 GB
ThreadUsage(32s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1
- AverageThreadTokens: 1.96
- BloomFilterBytes: 0
- PeakMemoryUsage: 13.46 GB (14456575703)
- PeakReservation: 13.46 GB (14451474432)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 13.94 GB (14965984087)
- RowsProduced: 9.22K (9216)
- TotalNetworkReceiveTime: 19m11s
- TotalNetworkSendTime: 41s614ms
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 3.85K (3855)
- TotalThreadsTotalWallClockTime: 40m32s
- TotalThreadsSysTime: 29s059ms
- TotalThreadsUserTime: 34s515ms
- TotalThreadsVoluntaryContextSwitches: 334.38K (334375)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 565.419us
- OpenTime: 20m16s
- ExecTreeOpenTime: 20m16s
- PrepareTime: 63.655ms
- ExecTreePrepareTime: 362.892us
DataStreamSender (dst_id=5):(Total: 19.510ms, non-child: 19.510ms, % non-child: 100.00%)
- BytesSent: 1.02 MB (1068724)
- NetworkThroughput(*): 143.75 MB/sec
- OverallThroughput: 52.24 MB/sec
- PeakMemoryUsage: 336.00 B (336)
- RowsReturned: 8.19K (8192)
- SerializeBatchTime: 13.245ms
- TransmitDataRPCTime: 7.090ms
- UncompressedRowBatchSize: 1.65 MB (1730225)
CodeGen:(Total: 403.150ms, non-child: 403.150ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 108.415ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 57 (57)
- NumInstructions: 2.60K (2602)
- OptimizationTime: 238.488ms
- PeakMemoryUsage: 1.27 MB (1332224)
- PrepareTime: 55.183ms
HASH_JOIN_NODE (id=2):(Total: 20m16s, non-child: 1m2s, % non-child: 5.12%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 58.14M (58135533)
- BuildTime: 1m1s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 13.46 GB (14452550254)
- ProbeRows: 1.02K (1024)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 503.043us
- RowsReturned: 9.22K (9216)
- RowsReturnedRate: 7.00 /sec
Buffer pool:
- AllocTime: 539.710ms
- CumulativeAllocationBytes: 13.46 GB (14451474432)
- CumulativeAllocations: 5.86K (5855)
- PeakReservation: 13.46 GB (14451474432)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 13.46 GB (14451474432)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
- BuildRowsPartitionTime: 50s411ms
- BuildRowsPartitioned: 58.14M (58135533)
- HashBuckets: 137.89M (137887744)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 10s563ms
- LargestPartitionPercent: 19 (19)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.12 KB (29824)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 19m14s, non-child: 19m14s, % non-child: 100.00%)
- ConvertRowBatchTime: 2s282ms
- PeakMemoryUsage: 0
- RowsReturned: 58.14M (58135533)
- RowsReturnedRate: 50.38 K/sec
DataStreamReceiver:
BytesReceived(32s000ms): 66.38 MB, 130.85 MB, 177.60 MB, 210.88 MB, 253.88 MB, 311.15 MB, 451.86 MB, 596.27 MB, 717.32 MB, 805.43 MB, 883.42 MB, 999.05 MB, 1.10 GB, 1.18 GB, 1.25 GB, 1.37 GB, 1.49 GB, 1.55 GB, 1.61 GB, 1.68 GB, 1.78 GB, 1.88 GB, 1.97 GB, 2.11 GB, 2.34 GB, 2.68 GB, 3.23 GB, 3.79 GB, 4.27 GB, 4.72 GB, 5.10 GB, 5.41 GB, 5.56 GB, 5.69 GB, 5.82 GB, 5.94 GB, 6.08 GB, 6.20 GB
- BytesReceived: 6.23 GB (6692464812)
- DeserializeRowBatchTimer: 2m15s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 3.02 MB (3169471)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=3):(Total: 160.681us, non-child: 160.681us, % non-child: 100.00%)
- ConvertRowBatchTime: 56.569us
- PeakMemoryUsage: 0
- RowsReturned: 1.02K (1024)
- RowsReturnedRate: 6.37 M/sec
DataStreamReceiver:
BytesReceived(32s000ms): 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.08 MB, 2.18 MB
- BytesReceived: 2.25 MB (2362654)
- DeserializeRowBatchTimer: 30.095ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 4.42 MB (4632771)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Instance f947e489a0df32c4:58c89d830000000e (host=zp-stg-hadoop-34:22000):(Total: 20m12s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 82.38 MB, 139.10 MB, 181.80 MB, 212.29 MB, 254.65 MB, 306.66 MB, 436.36 MB, 570.21 MB, 683.76 MB, 765.08 MB, 837.03 MB, 943.94 MB, 1.04 GB, 1.12 GB, 1.18 GB, 1.29 GB, 1.40 GB, 1.46 GB, 1.51 GB, 1.58 GB, 1.67 GB, 1.76 GB, 1.85 GB, 1.98 GB, 2.19 GB, 2.51 GB, 3.01 GB, 3.53 GB, 3.99 GB, 4.40 GB, 4.75 GB, 5.05 GB, 5.19 GB, 5.31 GB, 5.42 GB, 5.54 GB, 5.66 GB, 6.17 GB, 7.58 GB
ThreadUsage(32s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1
- AverageThreadTokens: 1.96
- BloomFilterBytes: 0
- PeakMemoryUsage: 7.58 GB (8137178013)
- PeakReservation: 7.58 GB (8134852608)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 8.07 GB (8660998263)
- RowsProduced: 7.17K (7168)
- TotalNetworkReceiveTime: 19m34s
- TotalNetworkSendTime: 50s826ms
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 1.75K (1747)
- TotalThreadsTotalWallClockTime: 40m24s
- TotalThreadsSysTime: 19s000ms
- TotalThreadsUserTime: 20s785ms
- TotalThreadsVoluntaryContextSwitches: 200.42K (200416)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1.264ms
- OpenTime: 20m12s
- ExecTreeOpenTime: 20m11s
- PrepareTime: 61.941ms
- ExecTreePrepareTime: 337.483us
DataStreamSender (dst_id=5):(Total: 12.666ms, non-child: 12.666ms, % non-child: 100.00%)
- BytesSent: 811.20 KB (830668)
- NetworkThroughput(*): 697.44 MB/sec
- OverallThroughput: 62.54 MB/sec
- PeakMemoryUsage: 336.00 B (336)
- RowsReturned: 6.14K (6144)
- SerializeBatchTime: 11.223ms
- TransmitDataRPCTime: 1.135ms
- UncompressedRowBatchSize: 1.30 MB (1366306)
CodeGen:(Total: 458.284ms, non-child: 458.284ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 116.450ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 57 (57)
- NumInstructions: 2.60K (2602)
- OptimizationTime: 288.027ms
- PeakMemoryUsage: 1.27 MB (1332224)
- PrepareTime: 52.363ms
HASH_JOIN_NODE (id=2):(Total: 20m11s, non-child: 35s224ms, % non-child: 2.91%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 33.70M (33697761)
- BuildTime: 34s539ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 7.58 GB (8135919271)
- ProbeRows: 1.02K (1024)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 1.209ms
- RowsReturned: 7.17K (7168)
- RowsReturnedRate: 5.00 /sec
Buffer pool:
- AllocTime: 749.677ms
- CumulativeAllocationBytes: 7.58 GB (8134852608)
- CumulativeAllocations: 3.37K (3371)
- PeakReservation: 7.58 GB (8134852608)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 7.58 GB (8134852608)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
- BuildRowsPartitionTime: 28s139ms
- BuildRowsPartitioned: 33.70M (33697761)
- HashBuckets: 68.68M (68681728)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 6s368ms
- LargestPartitionPercent: 27 (27)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.12 KB (29824)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 19m36s, non-child: 19m36s, % non-child: 100.00%)
- ConvertRowBatchTime: 1s366ms
- PeakMemoryUsage: 0
- RowsReturned: 33.70M (33697761)
- RowsReturnedRate: 28.64 K/sec
DataStreamReceiver:
BytesReceived(32s000ms): 38.03 MB, 75.11 MB, 101.97 MB, 121.06 MB, 145.83 MB, 178.69 MB, 259.34 MB, 342.42 MB, 411.92 MB, 462.57 MB, 507.42 MB, 574.04 MB, 647.82 MB, 696.73 MB, 735.46 MB, 807.15 MB, 877.60 MB, 914.79 MB, 945.16 MB, 991.03 MB, 1.02 GB, 1.08 GB, 1.13 GB, 1.21 GB, 1.35 GB, 1.54 GB, 1.86 GB, 2.18 GB, 2.46 GB, 2.71 GB, 2.93 GB, 3.11 GB, 3.20 GB, 3.28 GB, 3.35 GB, 3.42 GB, 3.50 GB
- BytesReceived: 3.59 GB (3852139607)
- DeserializeRowBatchTimer: 1m30s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 2.41 MB (2525199)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=3):(Total: 178.722us, non-child: 178.722us, % non-child: 100.00%)
- ConvertRowBatchTime: 67.310us
- PeakMemoryUsage: 0
- RowsReturned: 1.02K (1024)
- RowsReturnedRate: 5.73 M/sec
DataStreamReceiver:
BytesReceived(32s000ms): 912.29 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 912.80 KB, 972.43 KB
- BytesReceived: 1010.65 KB (1034907)
- DeserializeRowBatchTimer: 15.565ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 1.94 MB (2029767)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Instance f947e489a0df32c4:58c89d830000000f (host=zp-stg-hadoop-31:22000):(Total: 20m11s, non-child: 0.000ns, % non-child: 0.00%)
MemoryUsage(32s000ms): 111.23 MB, 197.12 MB, 263.02 MB, 307.08 MB, 363.77 MB, 443.67 MB, 635.66 MB, 832.35 MB, 996.43 MB, 1.09 GB, 1.19 GB, 1.35 GB, 1.52 GB, 1.63 GB, 1.72 GB, 1.89 GB, 2.05 GB, 2.14 GB, 2.21 GB, 2.31 GB, 2.45 GB, 2.58 GB, 2.71 GB, 2.90 GB, 3.21 GB, 3.68 GB, 4.42 GB, 5.18 GB, 5.85 GB, 6.45 GB, 6.98 GB, 7.40 GB, 7.61 GB, 7.79 GB, 7.96 GB, 8.13 GB, 8.31 GB, 9.13 GB, 11.13 GB
ThreadUsage(32s000ms): 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1
- AverageThreadTokens: 1.96
- BloomFilterBytes: 0
- PeakMemoryUsage: 11.13 GB (11950940725)
- PeakReservation: 11.13 GB (11947474944)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 11.61 GB (12463810229)
- RowsProduced: 65.54K (65536)
- TotalNetworkReceiveTime: 18m34s
- TotalNetworkSendTime: 43s556ms
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 3.30K (3297)
- TotalThreadsTotalWallClockTime: 40m22s
- TotalThreadsSysTime: 1m1s
- TotalThreadsUserTime: 28s108ms
- TotalThreadsVoluntaryContextSwitches: 290.83K (290829)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 12.771ms
- OpenTime: 20m11s
- ExecTreeOpenTime: 20m11s
- PrepareTime: 53.363ms
- ExecTreePrepareTime: 268.693us
DataStreamSender (dst_id=5):(Total: 102.898ms, non-child: 102.898ms, % non-child: 100.00%)
- BytesSent: 6.76 MB (7088679)
- NetworkThroughput(*): 583.52 MB/sec
- OverallThroughput: 65.70 MB/sec
- PeakMemoryUsage: 336.00 B (336)
- RowsReturned: 64.51K (64512)
- SerializeBatchTime: 90.918ms
- TransmitDataRPCTime: 11.585ms
- UncompressedRowBatchSize: 11.58 MB (12144780)
CodeGen:(Total: 578.489ms, non-child: 578.489ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 183.173ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 57 (57)
- NumInstructions: 2.60K (2602)
- OptimizationTime: 350.600ms
- PeakMemoryUsage: 1.27 MB (1332224)
- PrepareTime: 43.642ms
HASH_JOIN_NODE (id=2):(Total: 20m11s, non-child: 1m34s, % non-child: 7.80%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 50.12M (50121618)
- BuildTime: 1m33s
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 11.13 GB (11948536101)
- ProbeRows: 1.02K (1024)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 12.606ms
- RowsReturned: 65.54K (65536)
- RowsReturnedRate: 54.00 /sec
Buffer pool:
- AllocTime: 519.568ms
- CumulativeAllocationBytes: 11.13 GB (11947474944)
- CumulativeAllocations: 4.95K (4953)
- PeakReservation: 11.13 GB (11947474944)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 11.13 GB (11947474944)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=2):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
- BuildRowsPartitionTime: 1m27s
- BuildRowsPartitioned: 50.12M (50121618)
- HashBuckets: 99.61M (99614720)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 5s744ms
- LargestPartitionPercent: 21 (21)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.12 KB (29824)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=4):(Total: 18m36s, non-child: 18m36s, % non-child: 100.00%)
- ConvertRowBatchTime: 1s952ms
- PeakMemoryUsage: 0
- RowsReturned: 50.12M (50121618)
- RowsReturnedRate: 44.88 K/sec
DataStreamReceiver:
BytesReceived(32s000ms): 56.22 MB, 110.67 MB, 150.11 MB, 178.13 MB, 214.47 MB, 262.99 MB, 381.85 MB, 503.88 MB, 605.95 MB, 680.29 MB, 746.31 MB, 844.34 MB, 952.62 MB, 1.00 GB, 1.06 GB, 1.16 GB, 1.26 GB, 1.31 GB, 1.36 GB, 1.42 GB, 1.51 GB, 1.59 GB, 1.67 GB, 1.78 GB, 1.98 GB, 2.27 GB, 2.73 GB, 3.20 GB, 3.61 GB, 3.99 GB, 4.31 GB, 4.58 GB, 4.71 GB, 4.82 GB, 4.92 GB, 5.03 GB, 5.14 GB
- BytesReceived: 5.27 GB (5661890397)
- DeserializeRowBatchTimer: 4m23s
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 2.47 MB (2586366)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=3):(Total: 193.423us, non-child: 193.423us, % non-child: 100.00%)
- ConvertRowBatchTime: 77.617us
- PeakMemoryUsage: 0
- RowsReturned: 1.02K (1024)
- RowsReturnedRate: 5.29 M/sec
DataStreamReceiver:
BytesReceived(32s000ms): 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.37 MB, 1.44 MB
- BytesReceived: 1.48 MB (1555746)
- DeserializeRowBatchTimer: 80.368ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 2.98 MB (3123048)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F00:(Total: 49.602ms, non-child: 0.000ns, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:21m5s max:21m5s mean: 21m5s stddev:1.386ms
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 6
- AverageThreadTokens: 14.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 487.82 MB (511513728)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 18.16 GB (19494499008)
- RowsProduced: 5.97K (5973)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20m28s
- TotalStorageWaitTime: 18s625ms
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 266.749ms
- TotalThreadsSysTime: 1.999ms
- TotalThreadsUserTime: 1.665ms
- TotalThreadsVoluntaryContextSwitches: 63 (63)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 29.095ms
- OpenTime: 7.159ms
- ExecTreeOpenTime: 7.147ms
- PrepareTime: 42.367ms
- ExecTreePrepareTime: 71.305us
DataStreamSender (dst_id=3):(Total: 43.157ms, non-child: 43.157ms, % non-child: 100.00%)
- BytesSent: 2.35 MB (2461613)
- NetworkThroughput(*): 350.94 MB/sec
- OverallThroughput: 57.54 MB/sec
- PeakMemoryUsage: 169.12 KB (173184)
- RowsReturned: 4.95K (4949)
- SerializeBatchTime: 33.861ms
- TransmitDataRPCTime: 6.992ms
- UncompressedRowBatchSize: 5.08 MB (5328312)
CodeGen:(Total: 39.505ms, non-child: 39.505ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 38.560ms
KUDU_SCAN_NODE (id=0):(Total: 39.863ms, non-child: 39.863ms, % non-child: 100.00%)
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 486.00 MB (509605888)
- RowsRead: 313.42K (313422)
- RowsReturned: 5.97K (5973)
- RowsReturnedRate: 169.52 K/sec
- ScanRangesComplete: 5 (5)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 259.608ms
- MaterializeTupleTime(*): 29.051ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 1.332ms
- ScannerThreadsVoluntaryContextSwitches: 30 (30)
- TotalKuduScanRoundTrips: 287 (287)
- TotalReadThroughput: 0.00 /sec
Fragment F00:
Instance f947e489a0df32c4:58c89d8300000003 (host=zp-stg-hadoop-34:22000):(Total: 58.809ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(32s000ms): 483.18 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.76 MB, 488.74 MB
ThreadUsage(32s000ms): 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14
- AverageThreadTokens: 14.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 488.81 MB (512554112)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 8.07 GB (8660998263)
- RowsProduced: 5.12K (5120)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20m28s
- TotalStorageWaitTime: 5s965ms
- TotalThreadsInvoluntaryContextSwitches: 1 (1)
- TotalThreadsTotalWallClockTime: 33.943ms
- TotalThreadsSysTime: 3.000ms
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 60 (60)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 28.504ms
- OpenTime: 4.918ms
- ExecTreeOpenTime: 4.905ms
- PrepareTime: 53.813ms
- ExecTreePrepareTime: 75.180us
DataStreamSender (dst_id=3):(Total: 42.829ms, non-child: 42.829ms, % non-child: 100.00%)
- BytesSent: 2.05 MB (2151891)
- NetworkThroughput(*): 340.47 MB/sec
- OverallThroughput: 47.92 MB/sec
- PeakMemoryUsage: 169.12 KB (173184)
- RowsReturned: 4.10K (4096)
- SerializeBatchTime: 34.538ms
- TransmitDataRPCTime: 6.027ms
- UncompressedRowBatchSize: 4.44 MB (4658808)
CodeGen:(Total: 50.171ms, non-child: 50.171ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 48.795ms
KUDU_SCAN_NODE (id=0):(Total: 34.281ms, non-child: 34.281ms, % non-child: 100.00%)
BytesRead(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 17 (17)
- PeakMemoryUsage: 487.00 MB (510654464)
- RowsRead: 314.77K (314772)
- RowsReturned: 5.12K (5120)
- RowsReturnedRate: 149.35 K/sec
- ScanRangesComplete: 4 (4)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 29.040ms
- MaterializeTupleTime(*): 28.471ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 0.000ns
- ScannerThreadsVoluntaryContextSwitches: 24 (24)
- TotalKuduScanRoundTrips: 287 (287)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000006 (host=zp-stg-hadoop-32:22000):(Total: 56.091ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(32s000ms): 462.91 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.79 MB, 485.80 MB
ThreadUsage(32s000ms): 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14
- AverageThreadTokens: 14.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 485.81 MB (509408384)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 13.94 GB (14965984087)
- RowsProduced: 5.12K (5120)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20m28s
- TotalStorageWaitTime: 44s144ms
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 558.848ms
- TotalThreadsSysTime: 1.999ms
- TotalThreadsUserTime: 2.998ms
- TotalThreadsVoluntaryContextSwitches: 68 (68)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 32.046ms
- OpenTime: 4.155ms
- ExecTreeOpenTime: 4.141ms
- PrepareTime: 51.876ms
- ExecTreePrepareTime: 71.632us
DataStreamSender (dst_id=3):(Total: 39.177ms, non-child: 39.177ms, % non-child: 100.00%)
- BytesSent: 2.03 MB (2127182)
- NetworkThroughput(*): 319.74 MB/sec
- OverallThroughput: 51.78 MB/sec
- PeakMemoryUsage: 169.12 KB (173184)
- RowsReturned: 4.10K (4096)
- SerializeBatchTime: 31.507ms
- TransmitDataRPCTime: 6.344ms
- UncompressedRowBatchSize: 4.40 MB (4610440)
CodeGen:(Total: 50.160ms, non-child: 50.160ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 49.219ms
KUDU_SCAN_NODE (id=0):(Total: 36.833ms, non-child: 36.833ms, % non-child: 100.00%)
BytesRead(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 484.00 MB (507508736)
- RowsRead: 309.39K (309391)
- RowsReturned: 5.12K (5120)
- RowsReturnedRate: 139.00 K/sec
- ScanRangesComplete: 5 (5)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 554.714ms
- MaterializeTupleTime(*): 32.018ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 1.998ms
- ScannerThreadsVoluntaryContextSwitches: 31 (31)
- TotalKuduScanRoundTrips: 285 (285)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000004 (host=zp-stg-hadoop-31:22000):(Total: 54.616ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(32s000ms): 484.36 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.13 MB, 489.12 MB
ThreadUsage(32s000ms): 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16
- AverageThreadTokens: 16.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 489.15 MB (512910464)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 11.61 GB (12463810229)
- RowsProduced: 5.12K (5120)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20m28s
- TotalStorageWaitTime: 9s053ms
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 882.880ms
- TotalThreadsSysTime: 999.000us
- TotalThreadsUserTime: 2.998ms
- TotalThreadsVoluntaryContextSwitches: 59 (59)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 32.958ms
- OpenTime: 11.069ms
- ExecTreeOpenTime: 11.053ms
- PrepareTime: 43.465ms
- ExecTreePrepareTime: 105.862us
DataStreamSender (dst_id=3):(Total: 36.095ms, non-child: 36.095ms, % non-child: 100.00%)
- BytesSent: 1.84 MB (1924258)
- NetworkThroughput(*): 274.96 MB/sec
- OverallThroughput: 50.84 MB/sec
- PeakMemoryUsage: 169.12 KB (173184)
- RowsReturned: 4.10K (4096)
- SerializeBatchTime: 25.957ms
- TransmitDataRPCTime: 6.674ms
- UncompressedRowBatchSize: 3.98 MB (4171236)
CodeGen:(Total: 39.544ms, non-child: 39.544ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 38.666ms
KUDU_SCAN_NODE (id=0):(Total: 50.989ms, non-child: 50.989ms, % non-child: 100.00%)
BytesRead(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 19 (19)
- PeakMemoryUsage: 487.32 MB (510994432)
- RowsRead: 313.90K (313904)
- RowsReturned: 5.12K (5120)
- RowsReturnedRate: 100.41 K/sec
- ScanRangesComplete: 4 (4)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 871.832ms
- MaterializeTupleTime(*): 32.925ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 1.998ms
- ScannerThreadsVoluntaryContextSwitches: 29 (29)
- TotalKuduScanRoundTrips: 286 (286)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000005 (host=zp-stg-hadoop-35:22000):(Total: 50.913ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(32s000ms): 480.20 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.11 MB, 487.12 MB
ThreadUsage(32s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13
- AverageThreadTokens: 13.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 487.16 MB (510827648)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 16.50 GB (17715128843)
- RowsProduced: 5.12K (5120)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20m28s
- TotalStorageWaitTime: 5s717ms
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 50.906ms
- TotalThreadsSysTime: 1.999ms
- TotalThreadsUserTime: 1.998ms
- TotalThreadsVoluntaryContextSwitches: 70 (70)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 34.937ms
- OpenTime: 11.300ms
- ExecTreeOpenTime: 11.284ms
- PrepareTime: 39.545ms
- ExecTreePrepareTime: 75.403us
DataStreamSender (dst_id=3):(Total: 39.995ms, non-child: 39.995ms, % non-child: 100.00%)
- BytesSent: 1.96 MB (2059618)
- NetworkThroughput(*): 281.94 MB/sec
- OverallThroughput: 49.11 MB/sec
- PeakMemoryUsage: 169.12 KB (173184)
- RowsReturned: 4.10K (4096)
- SerializeBatchTime: 31.618ms
- TransmitDataRPCTime: 6.966ms
- UncompressedRowBatchSize: 4.25 MB (4459945)
CodeGen:(Total: 37.577ms, non-child: 37.577ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 36.602ms
KUDU_SCAN_NODE (id=0):(Total: 54.494ms, non-child: 54.494ms, % non-child: 100.00%)
BytesRead(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 485.34 MB (508911616)
- RowsRead: 314.51K (314511)
- RowsReturned: 5.12K (5120)
- RowsReturnedRate: 93.95 K/sec
- ScanRangesComplete: 6 (6)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 39.630ms
- MaterializeTupleTime(*): 34.889ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 1.998ms
- ScannerThreadsVoluntaryContextSwitches: 38 (38)
- TotalKuduScanRoundTrips: 286 (286)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000002 (host=zp-stg-hadoop-36:22000):(Total: 49.287ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(32s000ms): 464.81 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.78 MB, 488.79 MB
ThreadUsage(32s000ms): 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14
- AverageThreadTokens: 14.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 488.83 MB (512570496)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 45.70 GB (49069035690)
- RowsProduced: 8.19K (8192)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20m28s
- TotalStorageWaitTime: 29s732ms
- TotalThreadsInvoluntaryContextSwitches: 1 (1)
- TotalThreadsTotalWallClockTime: 50.579ms
- TotalThreadsSysTime: 1.000ms
- TotalThreadsUserTime: 1.998ms
- TotalThreadsVoluntaryContextSwitches: 69 (69)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 31.808ms
- OpenTime: 6.776ms
- ExecTreeOpenTime: 6.765ms
- PrepareTime: 42.373ms
- ExecTreePrepareTime: 63.660us
DataStreamSender (dst_id=3):(Total: 71.406ms, non-child: 71.406ms, % non-child: 100.00%)
- BytesSent: 3.27 MB (3424775)
- NetworkThroughput(*): 297.93 MB/sec
- OverallThroughput: 45.74 MB/sec
- PeakMemoryUsage: 169.12 KB (173184)
- RowsReturned: 7.17K (7168)
- SerializeBatchTime: 56.800ms
- TransmitDataRPCTime: 10.962ms
- UncompressedRowBatchSize: 7.07 MB (7415225)
CodeGen:(Total: 38.395ms, non-child: 38.395ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 37.362ms
KUDU_SCAN_NODE (id=0):(Total: 41.231ms, non-child: 41.231ms, % non-child: 100.00%)
BytesRead(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 487.00 MB (510654464)
- RowsRead: 314.15K (314153)
- RowsReturned: 8.19K (8192)
- RowsReturnedRate: 198.68 K/sec
- ScanRangesComplete: 5 (5)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 43.820ms
- MaterializeTupleTime(*): 31.711ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 1.998ms
- ScannerThreadsVoluntaryContextSwitches: 30 (30)
- TotalKuduScanRoundTrips: 291 (291)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000001 (host=zp-stg-hadoop-33:22000):(Total: 27.894ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(32s000ms): 475.93 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.09 MB, 487.10 MB
ThreadUsage(32s000ms): 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13
- AverageThreadTokens: 13.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 487.15 MB (510811264)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 13.12 GB (14092036940)
- RowsProduced: 7.17K (7168)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20m29s
- TotalStorageWaitTime: 17s137ms
- TotalThreadsInvoluntaryContextSwitches: 1 (1)
- TotalThreadsTotalWallClockTime: 23.340ms
- TotalThreadsSysTime: 3.000ms
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 56 (56)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 14.316ms
- OpenTime: 4.737ms
- ExecTreeOpenTime: 4.731ms
- PrepareTime: 23.129ms
- ExecTreePrepareTime: 36.097us
DataStreamSender (dst_id=3):(Total: 29.438ms, non-child: 29.438ms, % non-child: 100.00%)
- BytesSent: 2.94 MB (3081958)
- NetworkThroughput(*): 590.63 MB/sec
- OverallThroughput: 99.84 MB/sec
- PeakMemoryUsage: 169.12 KB (173184)
- RowsReturned: 6.14K (6144)
- SerializeBatchTime: 22.743ms
- TransmitDataRPCTime: 4.976ms
- UncompressedRowBatchSize: 6.35 MB (6654222)
CodeGen:(Total: 21.184ms, non-child: 21.184ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 20.716ms
KUDU_SCAN_NODE (id=0):(Total: 21.352ms, non-child: 21.352ms, % non-child: 100.00%)
BytesRead(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 18 (18)
- PeakMemoryUsage: 485.34 MB (508911616)
- RowsRead: 313.80K (313801)
- RowsReturned: 7.17K (7168)
- RowsReturnedRate: 335.69 K/sec
- ScanRangesComplete: 6 (6)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 18.614ms
- MaterializeTupleTime(*): 14.289ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 0.000ns
- ScannerThreadsVoluntaryContextSwitches: 31 (31)
- TotalKuduScanRoundTrips: 287 (287)
- TotalReadThroughput: 0.00 /sec
Averaged Fragment F01:(Total: 16m53s, non-child: 14m40s, % non-child: 86.92%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:14m25s max:20m6s mean: 16m53s stddev:1m49s
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 5
- AverageThreadTokens: 2.20
- BloomFilterBytes: 0
- PeakMemoryUsage: 82.76 MB (86782848)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 10.65 GB (11439881561)
- RowsProduced: 95.43M (95433077)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 12m16s
- TotalStorageWaitTime: 2m28s
- TotalThreadsInvoluntaryContextSwitches: 12.15K (12150)
- TotalThreadsTotalWallClockTime: 37m46s
- TotalThreadsSysTime: 31s514ms
- TotalThreadsUserTime: 3m11s
- TotalThreadsVoluntaryContextSwitches: 327.25K (327255)
Fragment Instance Lifecycle Timings:
- ExecTime: 16m53s
- ExecTreeExecTime: 3s138ms
- OpenTime: 9.091ms
- ExecTreeOpenTime: 9.078ms
- PrepareTime: 33.778ms
- ExecTreePrepareTime: 46.329us
DataStreamSender (dst_id=4):(Total: 2m9s, non-child: 2m9s, % non-child: 100.00%)
- BytesSent: 9.99 GB (10731064067)
- NetworkThroughput(*): 451.89 MB/sec
- OverallThroughput: 79.85 MB/sec
- PeakMemoryUsage: 367.88 KB (376704)
- RowsReturned: 95.43M (95433077)
- SerializeBatchTime: 1m46s
- TransmitDataRPCTime: 22s870ms
- UncompressedRowBatchSize: 16.58 GB (17807298747)
CodeGen:(Total: 30.864ms, non-child: 30.864ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 30.069ms
KUDU_SCAN_NODE (id=1):(Total: 2s983ms, non-child: 2s983ms, % non-child: 100.00%)
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 82.40 MB (86402867)
- RowsRead: 95.43M (95433077)
- RowsReturned: 95.43M (95433077)
- RowsReturnedRate: 41.80 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 3.79K (3791)
- ScannerThreadsTotalWallClockTime: 20m52s
- MaterializeTupleTime(*): 2s936ms
- ScannerThreadsSysTime: 20s714ms
- ScannerThreadsUserTime: 41s357ms
- ScannerThreadsVoluntaryContextSwitches: 97.21K (97211)
- TotalKuduScanRoundTrips: 16.45K (16455)
- TotalReadThroughput: 0.00 /sec
Fragment F01:
Instance f947e489a0df32c4:58c89d830000000b (host=zp-stg-hadoop-33:22000):(Total: 20m6s, non-child: 18m, % non-child: 89.60%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(32s000ms): 82.16 MB, 82.84 MB, 82.82 MB, 82.82 MB, 82.84 MB, 82.83 MB, 82.84 MB, 82.84 MB, 82.83 MB, 82.84 MB, 82.82 MB, 82.82 MB, 82.83 MB, 82.84 MB, 82.83 MB, 82.83 MB, 82.82 MB, 82.82 MB, 82.84 MB, 82.84 MB, 82.83 MB, 82.83 MB, 82.85 MB, 82.82 MB, 82.82 MB, 82.82 MB, 82.83 MB, 82.82 MB, 82.83 MB, 82.75 MB, 82.81 MB, 82.83 MB, 82.78 MB, 82.78 MB, 82.83 MB, 82.75 MB, 82.83 MB
ThreadUsage(32s000ms): 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3
- AverageThreadTokens: 3.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 83.00 MB (87032704)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 10.04 GB (10783223533)
- RowsProduced: 158.73M (158727562)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 16m4s
- TotalStorageWaitTime: 2m29s
- TotalThreadsInvoluntaryContextSwitches: 18.42K (18416)
- TotalThreadsTotalWallClockTime: 1h
- TotalThreadsSysTime: 33s596ms
- TotalThreadsUserTime: 3m9s
- TotalThreadsVoluntaryContextSwitches: 654.55K (654548)
Fragment Instance Lifecycle Timings:
- ExecTime: 20m6s
- ExecTreeExecTime: 3s318ms
- OpenTime: 3.910ms
- ExecTreeOpenTime: 3.903ms
- PrepareTime: 20.848ms
- ExecTreePrepareTime: 39.206us
DataStreamSender (dst_id=4):(Total: 2m2s, non-child: 2m2s, % non-child: 100.00%)
- BytesSent: 16.63 GB (17861139463)
- NetworkThroughput(*): 769.83 MB/sec
- OverallThroughput: 139.33 MB/sec
- PeakMemoryUsage: 367.88 KB (376704)
- RowsReturned: 158.73M (158727562)
- SerializeBatchTime: 1m39s
- TransmitDataRPCTime: 22s126ms
- UncompressedRowBatchSize: 27.57 GB (29600071456)
CodeGen:(Total: 19.042ms, non-child: 19.042ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 18.581ms
KUDU_SCAN_NODE (id=1):(Total: 3s163ms, non-child: 3s163ms, % non-child: 100.00%)
BytesRead(32s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 2 (2)
- PeakMemoryUsage: 82.63 MB (86647808)
- RowsRead: 158.73M (158727562)
- RowsReturned: 158.73M (158727562)
- RowsReturnedRate: 50.17 M/sec
- ScanRangesComplete: 2 (2)
- ScannerThreadsInvoluntaryContextSwitches: 5.96K (5965)
- ScannerThreadsTotalWallClockTime: 40m7s
- MaterializeTupleTime(*): 3s132ms
- ScannerThreadsSysTime: 22s560ms
- ScannerThreadsUserTime: 42s342ms
- ScannerThreadsVoluntaryContextSwitches: 206.69K (206692)
- TotalKuduScanRoundTrips: 27.37K (27369)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000009 (host=zp-stg-hadoop-32:22000):(Total: 16m46s, non-child: 14m28s, % non-child: 86.32%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 76.17 MB, 82.54 MB, 82.54 MB, 82.53 MB, 82.55 MB, 82.54 MB, 82.55 MB, 82.51 MB, 82.55 MB, 82.54 MB, 82.57 MB, 82.52 MB, 82.54 MB, 82.52 MB, 82.51 MB, 82.52 MB, 82.52 MB, 82.56 MB, 82.51 MB, 82.52 MB, 82.52 MB, 82.54 MB, 82.54 MB, 82.48 MB, 82.54 MB, 82.53 MB, 82.54 MB, 82.57 MB, 82.52 MB, 82.53 MB, 82.53 MB, 82.50 MB, 82.53 MB, 82.57 MB, 82.54 MB, 82.51 MB, 82.54 MB, 82.54 MB, 82.51 MB, 82.52 MB, 82.50 MB, 82.53 MB, 82.52 MB, 82.50 MB, 82.50 MB, 82.48 MB, 82.50 MB, 82.53 MB, 82.44 MB, 82.27 MB, 81.51 MB, 82.03 MB, 81.82 MB, 81.91 MB, 80.62 MB, 81.92 MB, 81.68 MB, 81.45 MB, 82.03 MB, 81.70 MB, 82.23 MB, 82.05 MB
ThreadUsage(16s000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 82.70 MB (86718336)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 9.33 GB (10012970865)
- RowsProduced: 81.29M (81290494)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 12m43s
- TotalStorageWaitTime: 2m12s
- TotalThreadsInvoluntaryContextSwitches: 13.38K (13385)
- TotalThreadsTotalWallClockTime: 33m30s
- TotalThreadsSysTime: 31s501ms
- TotalThreadsUserTime: 3m17s
- TotalThreadsVoluntaryContextSwitches: 283.41K (283414)
Fragment Instance Lifecycle Timings:
- ExecTime: 16m46s
- ExecTreeExecTime: 1s404ms
- OpenTime: 6.314ms
- ExecTreeOpenTime: 6.298ms
- PrepareTime: 40.470ms
- ExecTreePrepareTime: 55.920us
DataStreamSender (dst_id=4):(Total: 2m16s, non-child: 2m16s, % non-child: 100.00%)
- BytesSent: 8.50 GB (9130094942)
- NetworkThroughput(*): 350.97 MB/sec
- OverallThroughput: 63.88 MB/sec
- PeakMemoryUsage: 367.88 KB (376704)
- RowsReturned: 81.29M (81290494)
- SerializeBatchTime: 1m51s
- TransmitDataRPCTime: 24s808ms
- UncompressedRowBatchSize: 14.16 GB (15209248416)
CodeGen:(Total: 38.438ms, non-child: 38.438ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 37.488ms
KUDU_SCAN_NODE (id=1):(Total: 1s229ms, non-child: 1s229ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 82.34 MB (86341632)
- RowsRead: 81.29M (81290494)
- RowsReturned: 81.29M (81290494)
- RowsReturnedRate: 66.09 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 3.62K (3616)
- ScannerThreadsTotalWallClockTime: 16m44s
- MaterializeTupleTime(*): 1s183ms
- ScannerThreadsSysTime: 20s738ms
- ScannerThreadsUserTime: 42s216ms
- ScannerThreadsVoluntaryContextSwitches: 76.51K (76509)
- TotalKuduScanRoundTrips: 14.02K (14016)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000008 (host=zp-stg-hadoop-34:22000):(Total: 16m34s, non-child: 14m21s, % non-child: 86.61%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 79.93 MB, 82.54 MB, 82.53 MB, 82.53 MB, 82.55 MB, 82.55 MB, 82.56 MB, 82.50 MB, 82.54 MB, 82.53 MB, 82.53 MB, 82.51 MB, 82.52 MB, 82.54 MB, 82.50 MB, 82.51 MB, 82.53 MB, 82.53 MB, 82.49 MB, 82.53 MB, 82.55 MB, 82.53 MB, 82.53 MB, 82.51 MB, 82.53 MB, 82.51 MB, 82.51 MB, 82.52 MB, 82.53 MB, 82.52 MB, 82.48 MB, 82.47 MB, 82.56 MB, 82.52 MB, 82.55 MB, 82.49 MB, 82.55 MB, 82.54 MB, 82.51 MB, 82.55 MB, 82.51 MB, 82.53 MB, 82.54 MB, 82.53 MB, 82.53 MB, 82.53 MB, 82.49 MB, 81.97 MB, 82.17 MB, 82.23 MB, 81.60 MB, 82.19 MB, 81.74 MB, 82.11 MB, 80.92 MB, 82.34 MB, 81.27 MB, 81.90 MB, 82.40 MB, 82.07 MB, 82.06 MB, 82.17 MB
ThreadUsage(16s000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 82.70 MB (86718336)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 5.51 GB (5915642685)
- RowsProduced: 79.03M (79033069)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 13m3s
- TotalStorageWaitTime: 2m1s
- TotalThreadsInvoluntaryContextSwitches: 10.97K (10969)
- TotalThreadsTotalWallClockTime: 33m8s
- TotalThreadsSysTime: 31s530ms
- TotalThreadsUserTime: 3m12s
- TotalThreadsVoluntaryContextSwitches: 284.63K (284626)
Fragment Instance Lifecycle Timings:
- ExecTime: 16m34s
- ExecTreeExecTime: 1s591ms
- OpenTime: 6.256ms
- ExecTreeOpenTime: 6.241ms
- PrepareTime: 41.556ms
- ExecTreePrepareTime: 44.691us
DataStreamSender (dst_id=4):(Total: 2m11s, non-child: 2m11s, % non-child: 100.00%)
- BytesSent: 8.28 GB (8886194606)
- NetworkThroughput(*): 355.11 MB/sec
- OverallThroughput: 64.33 MB/sec
- PeakMemoryUsage: 367.88 KB (376704)
- RowsReturned: 79.03M (79033069)
- SerializeBatchTime: 1m47s
- TransmitDataRPCTime: 23s864ms
- UncompressedRowBatchSize: 13.73 GB (14739828747)
CodeGen:(Total: 38.875ms, non-child: 38.875ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 37.906ms
KUDU_SCAN_NODE (id=1):(Total: 1s429ms, non-child: 1s429ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 82.34 MB (86341632)
- RowsRead: 79.03M (79033069)
- RowsReturned: 79.03M (79033069)
- RowsReturnedRate: 55.27 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 3.44K (3445)
- ScannerThreadsTotalWallClockTime: 16m33s
- MaterializeTupleTime(*): 1s387ms
- ScannerThreadsSysTime: 20s305ms
- ScannerThreadsUserTime: 41s362ms
- ScannerThreadsVoluntaryContextSwitches: 75.90K (75903)
- TotalKuduScanRoundTrips: 13.63K (13627)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d830000000a (host=zp-stg-hadoop-31:22000):(Total: 16m34s, non-child: 14m20s, % non-child: 86.47%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 80.48 MB, 82.42 MB, 82.53 MB, 82.53 MB, 82.51 MB, 82.54 MB, 82.55 MB, 82.54 MB, 82.56 MB, 82.51 MB, 82.54 MB, 82.49 MB, 82.53 MB, 82.54 MB, 82.49 MB, 82.51 MB, 82.49 MB, 82.48 MB, 82.49 MB, 82.48 MB, 82.55 MB, 82.44 MB, 82.51 MB, 82.42 MB, 82.51 MB, 82.47 MB, 82.52 MB, 82.47 MB, 82.47 MB, 82.55 MB, 82.51 MB, 82.50 MB, 82.54 MB, 82.49 MB, 82.52 MB, 82.50 MB, 82.52 MB, 82.49 MB, 82.53 MB, 82.51 MB, 82.53 MB, 82.55 MB, 82.32 MB, 82.55 MB, 82.41 MB, 82.51 MB, 82.53 MB, 82.43 MB, 82.38 MB, 82.36 MB, 82.16 MB, 82.08 MB, 81.61 MB, 81.97 MB, 81.43 MB, 81.58 MB, 81.12 MB, 81.58 MB, 81.96 MB, 82.33 MB, 81.93 MB, 82.06 MB
ThreadUsage(16s000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 82.70 MB (86718336)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 7.82 GB (8395987935)
- RowsProduced: 79.11M (79106626)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 10m21s
- TotalStorageWaitTime: 2m31s
- TotalThreadsInvoluntaryContextSwitches: 12.34K (12342)
- TotalThreadsTotalWallClockTime: 33m7s
- TotalThreadsSysTime: 30s843ms
- TotalThreadsUserTime: 3m10s
- TotalThreadsVoluntaryContextSwitches: 278.24K (278241)
Fragment Instance Lifecycle Timings:
- ExecTime: 16m34s
- ExecTreeExecTime: 3s494ms
- OpenTime: 8.622ms
- ExecTreeOpenTime: 8.606ms
- PrepareTime: 41.463ms
- ExecTreePrepareTime: 43.927us
DataStreamSender (dst_id=4):(Total: 2m11s, non-child: 2m11s, % non-child: 100.00%)
- BytesSent: 8.28 GB (8890728058)
- NetworkThroughput(*): 360.45 MB/sec
- OverallThroughput: 64.63 MB/sec
- PeakMemoryUsage: 367.88 KB (376704)
- RowsReturned: 79.11M (79106626)
- SerializeBatchTime: 1m47s
- TransmitDataRPCTime: 23s522ms
- UncompressedRowBatchSize: 13.74 GB (14749864896)
CodeGen:(Total: 37.716ms, non-child: 37.716ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 36.665ms
KUDU_SCAN_NODE (id=1):(Total: 3s328ms, non-child: 3s328ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 82.34 MB (86341632)
- RowsRead: 79.11M (79106626)
- RowsReturned: 79.11M (79106626)
- RowsReturnedRate: 23.77 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 3.44K (3445)
- ScannerThreadsTotalWallClockTime: 16m33s
- MaterializeTupleTime(*): 3s283ms
- ScannerThreadsSysTime: 20s257ms
- ScannerThreadsUserTime: 40s830ms
- ScannerThreadsVoluntaryContextSwitches: 71.89K (71886)
- TotalKuduScanRoundTrips: 13.64K (13640)
- TotalReadThroughput: 0.00 /sec
Instance f947e489a0df32c4:58c89d8300000007 (host=zp-stg-hadoop-36:22000):(Total: 14m25s, non-child: 12m13s, % non-child: 84.74%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>):
MemoryUsage(16s000ms): 71.61 MB, 82.42 MB, 82.39 MB, 82.30 MB, 82.40 MB, 82.36 MB, 82.37 MB, 82.30 MB, 82.38 MB, 82.39 MB, 82.26 MB, 82.43 MB, 82.49 MB, 82.43 MB, 82.37 MB, 82.36 MB, 82.50 MB, 82.45 MB, 82.49 MB, 82.52 MB, 82.46 MB, 82.49 MB, 82.45 MB, 82.15 MB, 82.47 MB, 82.51 MB, 82.49 MB, 82.46 MB, 82.51 MB, 82.38 MB, 82.43 MB, 82.29 MB, 82.52 MB, 82.52 MB, 82.47 MB, 82.54 MB, 82.53 MB, 82.49 MB, 82.50 MB, 82.40 MB, 82.47 MB, 82.42 MB, 82.45 MB, 82.50 MB, 82.51 MB, 82.47 MB, 82.34 MB, 82.25 MB, 82.28 MB, 82.26 MB, 82.35 MB, 81.35 MB, 73.37 MB, 65.92 MB
ThreadUsage(16s000ms): 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 82.71 MB (86726528)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 20.57 GB (22091582788)
- RowsProduced: 79.01M (79007636)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 9m8s
- TotalStorageWaitTime: 3m10s
- TotalThreadsInvoluntaryContextSwitches: 5.64K (5640)
- TotalThreadsTotalWallClockTime: 28m51s
- TotalThreadsSysTime: 30s101ms
- TotalThreadsUserTime: 3m5s
- TotalThreadsVoluntaryContextSwitches: 135.45K (135449)
Fragment Instance Lifecycle Timings:
- ExecTime: 14m25s
- ExecTreeExecTime: 5s881ms
- OpenTime: 20.354ms
- ExecTreeOpenTime: 20.342ms
- PrepareTime: 24.555ms
- ExecTreePrepareTime: 47.904us
DataStreamSender (dst_id=4):(Total: 2m6s, non-child: 2m6s, % non-child: 100.00%)
- BytesSent: 8.28 GB (8887163268)
- NetworkThroughput(*): 423.12 MB/sec
- OverallThroughput: 67.07 MB/sec
- PeakMemoryUsage: 367.88 KB (376704)
- RowsReturned: 79.01M (79007636)
- SerializeBatchTime: 1m46s
- TransmitDataRPCTime: 20s030ms
- UncompressedRowBatchSize: 13.73 GB (14737480220)
CodeGen:(Total: 20.249ms, non-child: 20.249ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.96 MB (2050180)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 19.703ms
KUDU_SCAN_NODE (id=1):(Total: 5s767ms, non-child: 5s767ms, % non-child: 100.00%)
BytesRead(16s000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesRead: 0
- KuduRemoteScanTokens: 0 (0)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 82.34 MB (86341632)
- RowsRead: 79.01M (79007636)
- RowsReturned: 79.01M (79007636)
- RowsReturnedRate: 13.70 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 2.49K (2486)
- ScannerThreadsTotalWallClockTime: 14m25s
- MaterializeTupleTime(*): 5s696ms
- ScannerThreadsSysTime: 19s711ms
- ScannerThreadsUserTime: 40s034ms
- ScannerThreadsVoluntaryContextSwitches: 55.06K (55065)
- TotalKuduScanRoundTrips: 13.62K (13623)
- TotalReadThroughput: 0.00 /sec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment