Skip to content

Instantly share code, notes, and snippets.

@lekv
Last active December 5, 2017 19:01
Show Gist options
  • Save lekv/886926d8ce1f76c1d033ed01ae389732 to your computer and use it in GitHub Desktop.
Save lekv/886926d8ce1f76c1d033ed01ae389732 to your computer and use it in GitHub Desktop.
Preview for IMPALA-6190/IMPALA-6246
Query (id=7a4b153d9175fa9a:ab326f0d00000000):
DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance.
Summary:
Session ID: 324dcca803f2fbf5:7a6bf3b925f397a3
Session Type: BEESWAX
Start Time: 2017-12-04 16:46:45.530864000
End Time:
Query Type: QUERY
Query State: RUNNING
Query Status: OK
Impala Version: impalad version 2.11.0-SNAPSHOT DEBUG (build 3aa263a47c0792b1e778a40b6e449b016e93651e)
User: lv
Connected User: lv
Delegated User:
Network Address: ::ffff:127.0.0.1:49386
Default Db: tpch_parquet
Sql Statement: select count(*) from lineitem l1 join lineitem l2 join lineitem l3 where l1.l_orderkey = l2.l_orderkey and l2.l_orderkey < l3.l_orderkey
Coordinator: lv-desktop:22000
Query Options (set by configuration):
Query Options (set by configuration and planner): MT_DOP=0
Plan:
----------------
Max Per-Host Resource Reservation: Memory=17.00MB
Per-Host Resource Estimates: Memory=322.79MB
F04:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
PLAN-ROOT SINK
| mem-estimate=0B mem-reservation=0B
|
10:AGGREGATE [FINALIZE]
| output: count:merge(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=3 row-size=8B cardinality=1
|
09:EXCHANGE [UNPARTITIONED]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=3 row-size=8B cardinality=1
|
F02:PLAN FRAGMENT [HASH(l1.l_orderkey)] hosts=3 instances=3
Per-Host Resources: mem-estimate=72.79MB mem-reservation=17.00MB
05:AGGREGATE
| output: count(*)
| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
| tuple-ids=3 row-size=8B cardinality=1
|
04:NESTED LOOP JOIN [INNER JOIN, BROADCAST]
| predicates: l2.l_orderkey < l3.l_orderkey
| mem-estimate=45.79MB mem-reservation=0B
| tuple-ids=0,1,2 row-size=24B cardinality=23035503
|
|--08:EXCHANGE [BROADCAST]
| | mem-estimate=0B mem-reservation=0B
| | tuple-ids=2 row-size=8B cardinality=6001215
| |
| F03:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
| Per-Host Resources: mem-estimate=80.00MB mem-reservation=0B
| 02:SCAN HDFS [tpch_parquet.lineitem l3, RANDOM]
| partitions=1/1 files=3 size=193.71MB
| stored statistics:
| table: rows=6001215 size=193.71MB
| columns: all
| extrapolated-rows=disabled
| mem-estimate=80.00MB mem-reservation=0B
| tuple-ids=2 row-size=8B cardinality=6001215
|
03:HASH JOIN [INNER JOIN, PARTITIONED]
| hash predicates: l1.l_orderkey = l2.l_orderkey
| fk/pk conjuncts: none
| runtime filters: RF000[bloom] <- l2.l_orderkey
| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB
| tuple-ids=0,1 row-size=16B cardinality=23035503
|
|--07:EXCHANGE [HASH(l2.l_orderkey)]
| | mem-estimate=0B mem-reservation=0B
| | tuple-ids=1 row-size=8B cardinality=6001215
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
| Per-Host Resources: mem-estimate=80.00MB mem-reservation=0B
| 01:SCAN HDFS [tpch_parquet.lineitem l2, RANDOM]
| partitions=1/1 files=3 size=193.71MB
| stored statistics:
| table: rows=6001215 size=193.71MB
| columns: all
| extrapolated-rows=disabled
| mem-estimate=80.00MB mem-reservation=0B
| tuple-ids=1 row-size=8B cardinality=6001215
|
06:EXCHANGE [HASH(l1.l_orderkey)]
| mem-estimate=0B mem-reservation=0B
| tuple-ids=0 row-size=8B cardinality=6001215
|
F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
Per-Host Resources: mem-estimate=80.00MB mem-reservation=0B
00:SCAN HDFS [tpch_parquet.lineitem l1, RANDOM]
partitions=1/1 files=3 size=193.71MB
runtime filters: RF000[bloom] -> l1.l_orderkey
stored statistics:
table: rows=6001215 size=193.71MB
columns: all
extrapolated-rows=disabled
mem-estimate=80.00MB mem-reservation=0B
tuple-ids=0 row-size=8B cardinality=6001215
----------------
Estimated Per-Host Mem: 338465272
Per Host Min Reservation: lv-desktop:22000(17.00 MB) lv-desktop:22001(17.00 MB) lv-desktop:22002(17.00 MB)
Request Pool: default-pool
Admission result: Admitted immediately
Planner Timeline: 4s748ms
- Metadata load started: 32.953ms (32.953ms)
- Metadata load finished: 4s641ms (4s608ms)
- Analysis finished: 4s649ms (7.949ms)
- Value transfer graph computed: 4s665ms (16.538ms)
- Single node plan created: 4s693ms (27.778ms)
- Runtime filters computed: 4s698ms (4.956ms)
- Distributed plan created: 4s698ms (245.441us)
- Planning finished: 4s748ms (50.098ms)
Query Timeline: 18h8m
- Query submitted: 37.302us (37.302us)
- Planning finished: 4s760ms (4s760ms)
- Submit for admission: 4s760ms (455.280us)
- Completed admission: 4s761ms (136.147us)
- Ready to start on 3 backends: 4s761ms (417.953us)
- All 3 execution backends (13 fragment instances) started: 4s762ms (1.381ms)
- First dynamic filter received: 7s673ms (2s911ms)
- ComputeScanRangeAssignmentTimer: 147.294us
ImpalaServer:
- ClientFetchWaitTimer: 0.000ns
- RowMaterializationTimer: 0.000ns
Execution Profile 7a4b153d9175fa9a:ab326f0d00000000:(Total: 76.537ms, non-child: 0.000ns, % non-child: 0.00%)
Number of filters: 1
Filter routing table:
ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
-------------------------------------------------------------------------------------------------------------------
0 3 0 REMOTE false 3 (3) N/A N/A true
Backend startup latencies: Count: 3, min / max: 0 / 1ms, 25th %-ile: 0, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms
- FiltersReceived: 3 (3)
- FinalizationTimer: 0.000ns
- NumBackends: 3 (3)
- NumFragmentInstances: 13 (13)
- NumFragments: 5 (5)
Averaged Fragment F04:
Coordinator Fragment F04:
Instance 7a4b153d9175fa9a:ab326f0d00000000 (host=lv-desktop:22000):(Total: 16.543ms, non-child: 443.530us, % non-child: 2.68%)
Fragment Instance Lifecycle Event Timeline: 16.503ms
- Codegen Finished: 16.289ms (16.289ms)
- Prepare Finished: 16.503ms (213.299us)
MemoryUsage(500.000ms): 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB
- AverageThreadTokens: 0.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 12.00 KB (12288)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 257.72 MB (270239503)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 0 (0)
- TotalThreadsTotalWallClockTime: 0.000ns
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 0.000ns
- TotalThreadsVoluntaryContextSwitches: 0 (0)
Fragment Instance Lifecycle Timings:
- OpenTime: 0.000ns
- ExecTreeOpenTime: 0.000ns
- PrepareTime: 16.502ms
- ExecTreePrepareTime: 112.581us
PLAN_ROOT_SINK:
- PeakMemoryUsage: 0
CodeGen:(Total: 15.988ms, non-child: 15.988ms, % non-child: 100.00%)
- CodegenTime: 0.000ns
- CompileTime: 0.000ns
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 0 (0)
- NumInstructions: 0 (0)
- OptimizationTime: 0.000ns
- PeakMemoryUsage: 0
- PrepareTime: 15.419ms
AGGREGATION_NODE (id=10):(Total: 111.696us, non-child: 111.696us, % non-child: 100.00%)
- BuildTime: 0.000ns
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 4.00 KB (4096)
- RowsRepartitioned: 0 (0)
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=9):
- ConvertRowBatchTime: 0.000ns
- PeakMemoryUsage: 0
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
DataStreamReceiver:
BytesReceived(500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- BytesReceived: 0
- DeserializeRowBatchTimer: 0.000ns
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 0
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
Averaged Fragment F02:
Fragment F02:
Instance 7a4b153d9175fa9a:ab326f0d0000000b (host=lv-desktop:22000):(Total: 59.819ms, non-child: 0.000ns, % non-child: 0.00%)
Fragment Instance Lifecycle Event Timeline: 59.779ms
- Codegen Finished: 52.766ms (52.766ms)
- Prepare Finished: 59.779ms (7.012ms)
MemoryUsage(500.000ms): 27.16 MB, 55.03 MB, 80.47 MB, 105.75 MB, 125.41 MB, 167.42 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB
ThreadUsage(500.000ms): 3, 3, 3, 3, 3, 2, 1, 1, 1, 1
- AverageThreadTokens: 1.00
- BloomFilterBytes: 2.00 MB (2097152)
- PeakMemoryUsage: 250.47 MB (262639497)
- PeakReservation: 141.00 MB (147849216)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 257.72 MB (270239503)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 2s749ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 3.37K (3366)
- TotalThreadsTotalWallClockTime: 4s983ms
- TotalThreadsSysTime: 104.000ms
- TotalThreadsUserTime: 1s764ms
- TotalThreadsVoluntaryContextSwitches: 1.27K (1271)
Fragment Instance Lifecycle Timings:
- OpenTime: 0.000ns
- ExecTreeOpenTime: 0.000ns
- PrepareTime: 59.778ms
- ExecTreePrepareTime: 4.119ms
DataStreamSender (dst_id=9):(Total: 93.340us, non-child: 93.340us, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 301.567ms, non-child: 301.567ms, % non-child: 100.00%)
- CodegenTime: 639.726us
- CompileTime: 22.154ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 82 (82)
- NumInstructions: 2.74K (2741)
- OptimizationTime: 242.964ms
- PeakMemoryUsage: 1.34 MB (1403392)
- PrepareTime: 35.548ms
AGGREGATION_NODE (id=5):(Total: 4.119ms, non-child: 0.000ns, % non-child: 0.00%)
ExecOption: Codegen Enabled
- BuildTime: 9m36s
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 48.00 KB (49148)
- RowsRepartitioned: 0 (0)
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
NESTED_LOOP_JOIN_NODE (id=4):(Total: 17h56m, non-child: 17h56m, % non-child: 100.00%)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildRows: 6.00M (6001215)
- BuildTime: 82.036ms
- PeakMemoryUsage: 97.34 MB (102067887)
- ProbeRows: 1.02K (1024)
- ProbeTime: 0.000ns
- RowsReturned: 253.93B (253928239878)
- RowsReturnedRate: 3.93 M/sec
Nested Loop Join Builder:
- PeakMemoryUsage: 97.30 MB (102026931)
EXCHANGE_NODE (id=8):(Total: 2s058ms, non-child: 2s058ms, % non-child: 100.00%)
- ConvertRowBatchTime: 1s838ms
- PeakMemoryUsage: 0
- RowsReturned: 6.00M (6001215)
- RowsReturnedRate: 2.92 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 2.70 MB, 12.06 MB, 19.45 MB, 26.52 MB, 32.39 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB
- BytesReceived: 32.73 MB (34323965)
- DeserializeRowBatchTimer: 131.368ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 6.36 MB (6667264)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
HASH_JOIN_NODE (id=3):(Total: 2s828ms, non-child: 494.662ms, % non-child: 17.49%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 2.01M (2005526)
- BuildTime: 585.116ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 141.06 MB (147908860)
- ProbeRows: 14.34K (14336)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 3.164ms
- RowsReturned: 69.63K (69632)
- RowsReturnedRate: 24.61 K/sec
Buffer pool:
- AllocTime: 5.559ms
- CumulativeAllocationBytes: 141.00 MB (147849216)
- CumulativeAllocations: 93 (93)
- PeakReservation: 141.00 MB (147849216)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 141.00 MB (147849216)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=3):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 340.456ms
- BuildRowsPartitioned: 2.01M (2005526)
- HashBuckets: 4.19M (4194304)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 240.441ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=7):(Total: 2s229ms, non-child: 2s229ms, % non-child: 100.00%)
- ConvertRowBatchTime: 343.600ms
- PeakMemoryUsage: 0
- RowsReturned: 2.01M (2005526)
- RowsReturnedRate: 899.59 K/sec
DataStreamReceiver:
BytesReceived(500.000ms): 590.38 KB, 2.34 MB, 3.75 MB, 5.19 MB, 7.30 MB, 10.95 MB
- BytesReceived: 10.95 MB (11485320)
- DeserializeRowBatchTimer: 37.183ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 755.37 KB (773500)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=6):(Total: 704.162ms, non-child: 704.162ms, % non-child: 100.00%)
- ConvertRowBatchTime: 1.251ms
- PeakMemoryUsage: 0
- RowsReturned: 14.34K (14336)
- RowsReturnedRate: 20.36 K/sec
DataStreamReceiver:
BytesReceived(500.000ms): 0, 0, 1.18 MB, 2.62 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB
- BytesReceived: 3.46 MB (3627180)
- DeserializeRowBatchTimer: 10.398ms
- FirstBatchArrivalWaitTime: 702.755ms
- PeakMemoryUsage: 10.00 MB (10488660)
- SendersBlockedTimer: 15h49m
- SendersBlockedTotalTimer(*): 38h2m
Instance 7a4b153d9175fa9a:ab326f0d0000000c (host=lv-desktop:22001):(Total: 29.301ms, non-child: 0.000ns, % non-child: 0.00%)
Fragment Instance Lifecycle Event Timeline: 29.255ms
- Codegen Finished: 29.171ms (29.171ms)
- Prepare Finished: 29.255ms (83.591us)
MemoryUsage(500.000ms): 27.05 MB, 54.93 MB, 80.36 MB, 105.63 MB, 125.36 MB, 159.42 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB
ThreadUsage(500.000ms): 3, 3, 3, 3, 3, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.00
- BloomFilterBytes: 2.00 MB (2097152)
- PeakMemoryUsage: 250.47 MB (262639497)
- PeakReservation: 141.00 MB (147849216)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 257.74 MB (270259983)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 3s010ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 2.87K (2871)
- TotalThreadsTotalWallClockTime: 5s106ms
- TotalThreadsSysTime: 124.000ms
- TotalThreadsUserTime: 1s768ms
- TotalThreadsVoluntaryContextSwitches: 1.48K (1481)
Fragment Instance Lifecycle Timings:
- OpenTime: 0.000ns
- ExecTreeOpenTime: 0.000ns
- PrepareTime: 29.255ms
- ExecTreePrepareTime: 330.234us
DataStreamSender (dst_id=9):(Total: 80.333us, non-child: 80.333us, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 264.888ms, non-child: 264.888ms, % non-child: 100.00%)
- CodegenTime: 273.136us
- CompileTime: 28.217ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 82 (82)
- NumInstructions: 2.74K (2741)
- OptimizationTime: 216.942ms
- PeakMemoryUsage: 1.34 MB (1403392)
- PrepareTime: 19.184ms
AGGREGATION_NODE (id=5):(Total: 329.672us, non-child: 0.000ns, % non-child: 0.00%)
ExecOption: Codegen Enabled
- BuildTime: 9m19s
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 48.00 KB (49148)
- RowsRepartitioned: 0 (0)
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
NESTED_LOOP_JOIN_NODE (id=4):(Total: 17h57m, non-child: 17h57m, % non-child: 100.00%)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildRows: 6.00M (6001215)
- BuildTime: 83.925ms
- PeakMemoryUsage: 97.34 MB (102067887)
- ProbeRows: 1.02K (1024)
- ProbeTime: 0.000ns
- RowsReturned: 249.52B (249518458691)
- RowsReturnedRate: 3.86 M/sec
Nested Loop Join Builder:
- PeakMemoryUsage: 97.30 MB (102026931)
EXCHANGE_NODE (id=8):(Total: 2s112ms, non-child: 2s112ms, % non-child: 100.00%)
- ConvertRowBatchTime: 1s759ms
- PeakMemoryUsage: 0
- RowsReturned: 6.00M (6001215)
- RowsReturnedRate: 2.84 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 2.66 MB, 12.04 MB, 19.43 MB, 26.49 MB, 32.37 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB
- BytesReceived: 32.73 MB (34323965)
- DeserializeRowBatchTimer: 130.844ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 2.14 MB (2245632)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
HASH_JOIN_NODE (id=3):(Total: 2s908ms, non-child: 539.303ms, % non-child: 18.54%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 1.99M (1994429)
- BuildTime: 597.322ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 141.06 MB (147908860)
- ProbeRows: 14.34K (14336)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 3.193ms
- RowsReturned: 69.63K (69632)
- RowsReturnedRate: 23.94 K/sec
Buffer pool:
- AllocTime: 6.634ms
- CumulativeAllocationBytes: 141.00 MB (147849216)
- CumulativeAllocations: 93 (93)
- PeakReservation: 141.00 MB (147849216)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 141.00 MB (147849216)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=3):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 319.824ms
- BuildRowsPartitioned: 1.99M (1994429)
- HashBuckets: 4.19M (4194304)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 272.741ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=7):(Total: 2s288ms, non-child: 2s288ms, % non-child: 100.00%)
- ConvertRowBatchTime: 325.948ms
- PeakMemoryUsage: 0
- RowsReturned: 1.99M (1994429)
- RowsReturnedRate: 871.47 K/sec
DataStreamReceiver:
BytesReceived(500.000ms): 580.44 KB, 2.31 MB, 3.72 MB, 5.18 MB, 7.24 MB, 10.89 MB
- BytesReceived: 10.89 MB (11423629)
- DeserializeRowBatchTimer: 32.698ms
- FirstBatchArrivalWaitTime: 0.000ns
- PeakMemoryUsage: 271.93 KB (278460)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=6):(Total: 743.719ms, non-child: 743.719ms, % non-child: 100.00%)
- ConvertRowBatchTime: 1.236ms
- PeakMemoryUsage: 0
- RowsReturned: 14.34K (14336)
- RowsReturnedRate: 19.28 K/sec
DataStreamReceiver:
BytesReceived(500.000ms): 0, 0, 1.17 MB, 2.61 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB
- BytesReceived: 3.46 MB (3629040)
- DeserializeRowBatchTimer: 11.180ms
- FirstBatchArrivalWaitTime: 742.334ms
- PeakMemoryUsage: 10.00 MB (10488660)
- SendersBlockedTimer: 15h48m
- SendersBlockedTotalTimer(*): 37h16m
Instance 7a4b153d9175fa9a:ab326f0d0000000a (host=lv-desktop:22002):(Total: 28.927ms, non-child: 0.000ns, % non-child: 0.00%)
Fragment Instance Lifecycle Event Timeline: 28.893ms
- Codegen Finished: 28.769ms (28.769ms)
- Prepare Finished: 28.893ms (123.737us)
MemoryUsage(500.000ms): 27.49 MB, 55.35 MB, 80.83 MB, 106.18 MB, 125.62 MB, 160.42 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB, 250.44 MB
ThreadUsage(500.000ms): 3, 3, 3, 3, 3, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.00
- BloomFilterBytes: 2.00 MB (2097152)
- PeakMemoryUsage: 250.47 MB (262639497)
- PeakReservation: 141.00 MB (147849216)
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 257.71 MB (270227215)
- RowsProduced: 0 (0)
- TotalNetworkReceiveTime: 3s069ms
- TotalNetworkSendTime: 0.000ns
- TotalStorageWaitTime: 0.000ns
- TotalThreadsInvoluntaryContextSwitches: 3.41K (3412)
- TotalThreadsTotalWallClockTime: 5s196ms
- TotalThreadsSysTime: 132.000ms
- TotalThreadsUserTime: 1s772ms
- TotalThreadsVoluntaryContextSwitches: 1.45K (1450)
Fragment Instance Lifecycle Timings:
- OpenTime: 0.000ns
- ExecTreeOpenTime: 0.000ns
- PrepareTime: 28.892ms
- ExecTreePrepareTime: 364.727us
DataStreamSender (dst_id=9):(Total: 4.319ms, non-child: 4.319ms, % non-child: 100.00%)
- BytesSent: 0
- NetworkThroughput(*): 0.00 /sec
- OverallThroughput: 0.00 /sec
- PeakMemoryUsage: 16.00 KB (16384)
- RowsReturned: 0 (0)
- SerializeBatchTime: 0.000ns
- TransmitDataRPCTime: 0.000ns
- UncompressedRowBatchSize: 0
CodeGen:(Total: 224.439ms, non-child: 224.439ms, % non-child: 100.00%)
- CodegenTime: 363.180us
- CompileTime: 20.070ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 82 (82)
- NumInstructions: 2.74K (2741)
- OptimizationTime: 186.986ms
- PeakMemoryUsage: 1.34 MB (1403392)
- PrepareTime: 16.975ms
AGGREGATION_NODE (id=5):(Total: 364.243us, non-child: 0.000ns, % non-child: 0.00%)
ExecOption: Codegen Enabled
- BuildTime: 9m43s
- GetResultsTime: 0.000ns
- HTResizeTime: 0.000ns
- HashBuckets: 0 (0)
- LargestPartitionPercent: 0 (0)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 0 (0)
- PeakMemoryUsage: 48.00 KB (49148)
- RowsRepartitioned: 0 (0)
- RowsReturned: 0 (0)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
NESTED_LOOP_JOIN_NODE (id=4):(Total: 17h56m, non-child: 17h56m, % non-child: 100.00%)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildRows: 6.00M (6001215)
- BuildTime: 81.563ms
- PeakMemoryUsage: 97.34 MB (102067887)
- ProbeRows: 1.02K (1024)
- ProbeTime: 0.000ns
- RowsReturned: 251.39B (251390346485)
- RowsReturnedRate: 3.89 M/sec
Nested Loop Join Builder:
- PeakMemoryUsage: 97.30 MB (102026931)
EXCHANGE_NODE (id=8):(Total: 2s155ms, non-child: 2s155ms, % non-child: 100.00%)
- ConvertRowBatchTime: 1s812ms
- PeakMemoryUsage: 0
- RowsReturned: 6.00M (6001215)
- RowsReturnedRate: 2.78 M/sec
DataStreamReceiver:
BytesReceived(500.000ms): 2.81 MB, 12.18 MB, 19.55 MB, 26.64 MB, 32.46 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB, 32.73 MB
- BytesReceived: 32.73 MB (34323965)
- DeserializeRowBatchTimer: 134.890ms
- FirstBatchArrivalWaitTime: 18.328ms
- PeakMemoryUsage: 10.01 MB (10497024)
- SendersBlockedTimer: 25.702ms
- SendersBlockedTotalTimer(*): 53.226ms
HASH_JOIN_NODE (id=3):(Total: 2s949ms, non-child: 541.959ms, % non-child: 18.37%)
ExecOption: Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildRows: 2.00M (2001260)
- BuildTime: 612.159ms
- NumHashTableBuildsSkipped: 0 (0)
- PeakMemoryUsage: 141.06 MB (147908860)
- ProbeRows: 14.34K (14336)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 3.073ms
- RowsReturned: 69.63K (69632)
- RowsReturnedRate: 23.61 K/sec
Buffer pool:
- AllocTime: 490.914us
- CumulativeAllocationBytes: 141.00 MB (147849216)
- CumulativeAllocations: 93 (93)
- PeakReservation: 141.00 MB (147849216)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 141.00 MB (147849216)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
Hash Join Builder (join_node_id=3):
ExecOption: Build Side Codegen Enabled, Hash Table Construction Codegen Enabled
Runtime filters: 1 of 1 Runtime Filter Published
- BuildRowsPartitionTime: 332.285ms
- BuildRowsPartitioned: 2.00M (2001260)
- HashBuckets: 4.19M (4194304)
- HashCollisions: 0 (0)
- HashTablesBuildTime: 275.160ms
- LargestPartitionPercent: 6 (6)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.12 KB (21632)
- RepartitionTime: 0.000ns
- SpilledPartitions: 0 (0)
EXCHANGE_NODE (id=7):(Total: 2s326ms, non-child: 2s326ms, % non-child: 100.00%)
- ConvertRowBatchTime: 338.472ms
- PeakMemoryUsage: 0
- RowsReturned: 2.00M (2001260)
- RowsReturnedRate: 860.38 K/sec
DataStreamReceiver:
BytesReceived(500.000ms): 620.31 KB, 2.37 MB, 3.77 MB, 5.21 MB, 7.32 MB, 10.93 MB
- BytesReceived: 10.93 MB (11461707)
- DeserializeRowBatchTimer: 41.502ms
- FirstBatchArrivalWaitTime: 19.795ms
- PeakMemoryUsage: 120.86 KB (123760)
- SendersBlockedTimer: 0.000ns
- SendersBlockedTotalTimer(*): 0.000ns
EXCHANGE_NODE (id=6):(Total: 792.966ms, non-child: 792.966ms, % non-child: 100.00%)
- ConvertRowBatchTime: 1.221ms
- PeakMemoryUsage: 0
- RowsReturned: 14.34K (14336)
- RowsReturnedRate: 18.08 K/sec
DataStreamReceiver:
BytesReceived(500.000ms): 0, 0, 1.20 MB, 2.65 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB, 3.40 MB
- BytesReceived: 3.46 MB (3628409)
- DeserializeRowBatchTimer: 12.813ms
- FirstBatchArrivalWaitTime: 791.515ms
- PeakMemoryUsage: 10.00 MB (10488660)
- SendersBlockedTimer: 13h25m
- SendersBlockedTotalTimer(*): 36h25m
Averaged Fragment F00:
Fragment F00:
Instance 7a4b153d9175fa9a:ab326f0d00000002 (host=lv-desktop:22000):(Total: 108.255ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/68.86 MB
Filter 0 arrival: 2s916ms
Fragment Instance Lifecycle Event Timeline: 1s036ms
- Codegen Finished: 66.574ms (66.574ms)
- Prepare Finished: 69.308ms (2.734ms)
- Open Finished: 108.228ms (38.919ms)
- First Batch Received: 1s035ms (927.538ms)
- First Batch Sent: 1s036ms (457.853us)
MemoryUsage(500.000ms): 51.86 KB, 51.86 KB, 5.20 MB, 5.19 MB, 5.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB, 7.24 MB
ThreadUsage(500.000ms): 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 2.00 MB (2097152)
- PeakMemoryUsage: 7.25 MB (7604098)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 257.72 MB (270239503)
- RowsProduced: 633.86K (633856)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 10h53m
- TotalStorageWaitTime: 11.691ms
- TotalThreadsInvoluntaryContextSwitches: 21 (21)
- TotalThreadsTotalWallClockTime: 38.894ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 36.000ms
- TotalThreadsVoluntaryContextSwitches: 1 (1)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s174ms
- OpenTime: 38.896ms
- ExecTreeOpenTime: 105.069us
- PrepareTime: 69.306ms
- ExecTreePrepareTime: 22.474ms
DataStreamSender (dst_id=6):(Total: 416.333ms, non-child: 416.333ms, % non-child: 100.00%)
- BytesSent: 3.44 MB (3607328)
- NetworkThroughput(*): 84.95 MB/sec
- OverallThroughput: 8.26 MB/sec
- PeakMemoryUsage: 222.66 KB (228000)
- RowsReturned: 632.83K (632832)
- SerializeBatchTime: 352.586ms
- TransmitDataRPCTime: 40.498ms
- UncompressedRowBatchSize: 10.21 MB (10705240)
CodeGen:(Total: 58.738ms, non-child: 58.738ms, % non-child: 100.00%)
- CodegenTime: 1.775ms
- CompileTime: 8.704ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 29 (29)
- NumInstructions: 445 (445)
- OptimizationTime: 28.795ms
- PeakMemoryUsage: 222.50 KB (227840)
- PrepareTime: 20.686ms
HDFS_SCAN_NODE (id=0):(Total: 1s218ms, non-child: 1s218ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/68.86 MB
ExecOption: PARQUET Codegen Enabled
Runtime filters: Not all filters arrived (arrived: [], missing [0]), waited for 906ms
BytesRead(500.000ms): 0, 0, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB
- FooterProcessingTime: (Avg: 132.464us ; Min: 132.464us ; Max: 132.464us ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 3.42 MB (3587693)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 0
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 0
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 0 (0)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 3.412ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 0 (0)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 286.12 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 0.000ns
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 696.32K (696320)
- RowsReturned: 633.86K (633856)
- RowsReturnedRate: 519.99 K/sec
- ScanRangesComplete: 0 (0)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 0.000ns
- MaterializeTupleTime(*): 0.000ns
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 0.000ns
- ScannerThreadsVoluntaryContextSwitches: 0 (0)
- TotalRawHdfsReadTime(*): 11.958ms
- TotalReadThroughput: 54.00 B/sec
Filter 0 (2.00 MB):
- Files processed: 0 (0)
- Files rejected: 0 (0)
- Files total: 0 (0)
- RowGroups processed: 0 (0)
- RowGroups rejected: 0 (0)
- RowGroups total: 0 (0)
- Rows processed: 0 (0)
- Rows rejected: 0 (0)
- Rows total: 0 (0)
- Splits processed: 0 (0)
- Splits rejected: 0 (0)
- Splits total: 0 (0)
Instance 7a4b153d9175fa9a:ab326f0d00000003 (host=lv-desktop:22001):(Total: 114.190ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/55.73 MB
Filter 0 arrival: 2s919ms
Fragment Instance Lifecycle Event Timeline: 1s037ms
- Codegen Finished: 64.647ms (64.647ms)
- Prepare Finished: 74.834ms (10.186ms)
- Open Finished: 114.153ms (39.318ms)
- First Batch Received: 1s037ms (923.292ms)
- First Batch Sent: 1s037ms (455.838us)
MemoryUsage(500.000ms): 51.86 KB, 51.86 KB, 5.19 MB, 5.20 MB, 5.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB, 7.23 MB
ThreadUsage(500.000ms): 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 2.00 MB (2097152)
- PeakMemoryUsage: 7.28 MB (7636866)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 257.74 MB (270259983)
- RowsProduced: 616.45K (616448)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 13h14m
- TotalStorageWaitTime: 7.533ms
- TotalThreadsInvoluntaryContextSwitches: 22 (22)
- TotalThreadsTotalWallClockTime: 39.297ms
- TotalThreadsSysTime: 0.000ns
- TotalThreadsUserTime: 28.000ms
- TotalThreadsVoluntaryContextSwitches: 1 (1)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s096ms
- OpenTime: 39.299ms
- ExecTreeOpenTime: 73.012us
- PrepareTime: 74.833ms
- ExecTreePrepareTime: 20.289ms
DataStreamSender (dst_id=6):(Total: 387.829ms, non-child: 387.829ms, % non-child: 100.00%)
- BytesSent: 3.35 MB (3514341)
- NetworkThroughput(*): 92.96 MB/sec
- OverallThroughput: 8.64 MB/sec
- PeakMemoryUsage: 190.66 KB (195232)
- RowsReturned: 615.42K (615424)
- SerializeBatchTime: 351.354ms
- TransmitDataRPCTime: 36.053ms
- UncompressedRowBatchSize: 9.94 MB (10426780)
CodeGen:(Total: 78.100ms, non-child: 78.100ms, % non-child: 100.00%)
- CodegenTime: 8.083ms
- CompileTime: 3.849ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 29 (29)
- NumInstructions: 445 (445)
- OptimizationTime: 34.509ms
- PeakMemoryUsage: 222.50 KB (227840)
- PrepareTime: 32.519ms
HDFS_SCAN_NODE (id=0):(Total: 1s136ms, non-child: 1s136ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/55.73 MB
ExecOption: PARQUET Codegen Enabled
Runtime filters: Not all filters arrived (arrived: [], missing [0]), waited for 906ms
BytesRead(500.000ms): 0, 0, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB
- FooterProcessingTime: (Avg: 2.807ms ; Min: 2.807ms ; Max: 2.807ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 2.81 MB (2945000)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 0
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 0
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 0 (0)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 3.368ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 0 (0)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 513.75 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 0.000ns
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 678.91K (678912)
- RowsReturned: 616.45K (616448)
- RowsReturnedRate: 542.47 K/sec
- ScanRangesComplete: 0 (0)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 0.000ns
- MaterializeTupleTime(*): 0.000ns
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 0.000ns
- ScannerThreadsVoluntaryContextSwitches: 0 (0)
- TotalRawHdfsReadTime(*): 5.466ms
- TotalReadThroughput: 45.00 B/sec
Filter 0 (2.00 MB):
- Files processed: 0 (0)
- Files rejected: 0 (0)
- Files total: 0 (0)
- RowGroups processed: 0 (0)
- RowGroups rejected: 0 (0)
- RowGroups total: 0 (0)
- Rows processed: 0 (0)
- Rows rejected: 0 (0)
- Rows total: 0 (0)
- Splits processed: 0 (0)
- Splits rejected: 0 (0)
- Splits total: 0 (0)
Instance 7a4b153d9175fa9a:ab326f0d00000001 (host=lv-desktop:22002):(Total: 74.633ms, non-child: 0.000ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/69.12 MB
Filter 0 arrival: 2s921ms
Fragment Instance Lifecycle Event Timeline: 1s024ms
- Codegen Finished: 37.053ms (37.053ms)
- Prepare Finished: 39.441ms (2.387ms)
- Open Finished: 74.603ms (35.162ms)
- First Batch Received: 1s023ms (949.193ms)
- First Batch Sent: 1s024ms (465.504us)
MemoryUsage(500.000ms): 51.86 KB, 51.86 KB, 5.20 MB, 5.17 MB, 5.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB, 7.21 MB
ThreadUsage(500.000ms): 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 2.00 MB (2097152)
- PeakMemoryUsage: 7.25 MB (7604098)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 257.71 MB (270227215)
- RowsProduced: 659.46K (659456)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 13h25m
- TotalStorageWaitTime: 5.258ms
- TotalThreadsInvoluntaryContextSwitches: 24 (24)
- TotalThreadsTotalWallClockTime: 35.138ms
- TotalThreadsSysTime: 4.000ms
- TotalThreadsUserTime: 28.000ms
- TotalThreadsVoluntaryContextSwitches: 2 (2)
Fragment Instance Lifecycle Timings:
- ExecTime: 0.000ns
- ExecTreeExecTime: 1s181ms
- OpenTime: 35.140ms
- ExecTreeOpenTime: 75.230us
- PrepareTime: 39.440ms
- ExecTreePrepareTime: 18.056ms
DataStreamSender (dst_id=6):(Total: 424.873ms, non-child: 424.873ms, % non-child: 100.00%)
- BytesSent: 3.59 MB (3762960)
- NetworkThroughput(*): 85.88 MB/sec
- OverallThroughput: 8.45 MB/sec
- PeakMemoryUsage: 190.66 KB (195232)
- RowsReturned: 658.43K (658432)
- SerializeBatchTime: 382.255ms
- TransmitDataRPCTime: 41.788ms
- UncompressedRowBatchSize: 10.65 MB (11169340)
CodeGen:(Total: 52.088ms, non-child: 52.088ms, % non-child: 100.00%)
- CodegenTime: 1.707ms
- CompileTime: 7.324ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 29 (29)
- NumInstructions: 445 (445)
- OptimizationTime: 26.503ms
- PeakMemoryUsage: 222.50 KB (227840)
- PrepareTime: 17.715ms
HDFS_SCAN_NODE (id=0):(Total: 1s217ms, non-child: 1s217ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/69.12 MB
ExecOption: PARQUET Codegen Enabled
Runtime filters: Not all filters arrived (arrived: [], missing [0]), waited for 926ms
BytesRead(500.000ms): 0, 0, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB
- FooterProcessingTime: (Avg: 1.267ms ; Min: 1.267ms ; Max: 1.267ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 3.43 MB (3592722)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 0
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 0
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 0 (0)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 5.272ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 0 (0)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 724.29 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 0.000ns
- RowBatchQueuePutWaitTime: 0.000ns
- RowsRead: 721.92K (721920)
- RowsReturned: 659.46K (659456)
- RowsReturnedRate: 541.84 K/sec
- ScanRangesComplete: 0 (0)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 0.000ns
- MaterializeTupleTime(*): 0.000ns
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 0.000ns
- ScannerThreadsVoluntaryContextSwitches: 0 (0)
- TotalRawHdfsReadTime(*): 4.730ms
- TotalReadThroughput: 55.00 B/sec
Filter 0 (2.00 MB):
- Files processed: 0 (0)
- Files rejected: 0 (0)
- Files total: 0 (0)
- RowGroups processed: 0 (0)
- RowGroups rejected: 0 (0)
- RowGroups total: 0 (0)
- Rows processed: 0 (0)
- Rows rejected: 0 (0)
- Rows total: 0 (0)
- Splits processed: 0 (0)
- Splits rejected: 0 (0)
- Splits total: 0 (0)
Averaged Fragment F01:
Fragment F01:
Instance 7a4b153d9175fa9a:ab326f0d00000005 (host=lv-desktop:22000):(Total: 2s872ms, non-child: 1s366ms, % non-child: 47.58%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/68.86 MB
Fragment Instance Lifecycle Event Timeline: 2s872ms
- Codegen Finished: 65.665ms (65.665ms)
- Prepare Finished: 70.637ms (4.971ms)
- Open Finished: 83.861ms (13.224ms)
- First Batch Received: 246.805ms (162.944ms)
- First Batch Sent: 247.222ms (416.937us)
- ExecInternal Finished: 2s872ms (2s624ms)
MemoryUsage(500.000ms): 5.19 MB, 5.18 MB, 5.23 MB, 5.22 MB, 5.20 MB
ThreadUsage(500.000ms): 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 5.31 MB (5571664)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 152.06 MB (159448783)
- RowsProduced: 2.13M (2133979)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 26.733ms
- TotalStorageWaitTime: 158.085ms
- TotalThreadsInvoluntaryContextSwitches: 3.38K (3382)
- TotalThreadsTotalWallClockTime: 5s555ms
- TotalThreadsSysTime: 24.000ms
- TotalThreadsUserTime: 1s696ms
- TotalThreadsVoluntaryContextSwitches: 1.98K (1982)
Fragment Instance Lifecycle Timings:
- ExecTime: 2s788ms
- ExecTreeExecTime: 268.498ms
- OpenTime: 13.200ms
- ExecTreeOpenTime: 109.904us
- PrepareTime: 70.636ms
- ExecTreePrepareTime: 19.023ms
DataStreamSender (dst_id=7):(Total: 1s171ms, non-child: 1s171ms, % non-child: 100.00%)
- BytesSent: 11.66 MB (12223008)
- NetworkThroughput(*): 93.28 MB/sec
- OverallThroughput: 9.95 MB/sec
- PeakMemoryUsage: 222.66 KB (228000)
- RowsReturned: 2.13M (2133979)
- SerializeBatchTime: 1s017ms
- TransmitDataRPCTime: 124.970ms
- UncompressedRowBatchSize: 34.60 MB (36277643)
CodeGen:(Total: 29.759ms, non-child: 29.759ms, % non-child: 100.00%)
- CodegenTime: 614.672us
- CompileTime: 1.944ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 9.699ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 17.595ms
HDFS_SCAN_NODE (id=1):(Total: 304.202ms, non-child: 304.202ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/68.86 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0%
File Formats: PARQUET/SNAPPY:1
BytesRead(500.000ms): 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB
- FooterProcessingTime: (Avg: 152.459ms ; Min: 152.459ms ; Max: 152.459ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 3.42 MB (3587693)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 3.42 MB (3587693)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 3.42 MB (3587693)
- CachedFileHandlesHitCount: 1 (1)
- CachedFileHandlesMissCount: 1 (1)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 10.991ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 21.66 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 165.974ms
- RowBatchQueuePutWaitTime: 2s380ms
- RowsRead: 2.13M (2133979)
- RowsReturned: 2.13M (2133979)
- RowsReturnedRate: 7.01 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 147 (147)
- ScannerThreadsTotalWallClockTime: 2s754ms
- MaterializeTupleTime(*): 176.158ms
- ScannerThreadsSysTime: 4.000ms
- ScannerThreadsUserTime: 192.000ms
- ScannerThreadsVoluntaryContextSwitches: 1.96K (1964)
- TotalRawHdfsReadTime(*): 157.966ms
- TotalReadThroughput: 1.37 MB/sec
Instance 7a4b153d9175fa9a:ab326f0d00000006 (host=lv-desktop:22001):(Total: 2s733ms, non-child: 1s295ms, % non-child: 47.39%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/55.73 MB
Fragment Instance Lifecycle Event Timeline: 2s733ms
- Codegen Finished: 79.312ms (79.312ms)
- Prepare Finished: 79.454ms (141.949us)
- Open Finished: 93.287ms (13.832ms)
- First Batch Received: 259.751ms (166.464ms)
- First Batch Sent: 260.219ms (467.917us)
- ExecInternal Finished: 2s733ms (2s473ms)
MemoryUsage(500.000ms): 5.22 MB, 5.20 MB, 5.24 MB, 5.20 MB, 5.18 MB
ThreadUsage(500.000ms): 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 5.28 MB (5538896)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 136.98 MB (143637787)
- RowsProduced: 1.72M (1724693)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 20.198ms
- TotalStorageWaitTime: 144.585ms
- TotalThreadsInvoluntaryContextSwitches: 2.44K (2438)
- TotalThreadsTotalWallClockTime: 5s238ms
- TotalThreadsSysTime: 8.000ms
- TotalThreadsUserTime: 1s428ms
- TotalThreadsVoluntaryContextSwitches: 1.58K (1584)
Fragment Instance Lifecycle Timings:
- ExecTime: 2s640ms
- ExecTreeExecTime: 324.882ms
- OpenTime: 13.806ms
- ExecTreeOpenTime: 104.708us
- PrepareTime: 79.452ms
- ExecTreePrepareTime: 52.437ms
DataStreamSender (dst_id=7):(Total: 975.453ms, non-child: 975.453ms, % non-child: 100.00%)
- BytesSent: 9.42 MB (9877871)
- NetworkThroughput(*): 90.79 MB/sec
- OverallThroughput: 9.66 MB/sec
- PeakMemoryUsage: 190.66 KB (195232)
- RowsReturned: 1.72M (1724693)
- SerializeBatchTime: 867.498ms
- TransmitDataRPCTime: 103.755ms
- UncompressedRowBatchSize: 27.96 MB (29319781)
CodeGen:(Total: 35.940ms, non-child: 35.940ms, % non-child: 100.00%)
- CodegenTime: 670.050us
- CompileTime: 2.939ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 9.342ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 22.984ms
HDFS_SCAN_NODE (id=1):(Total: 426.848ms, non-child: 426.848ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/55.73 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0%
File Formats: PARQUET/SNAPPY:1
BytesRead(500.000ms): 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB
- FooterProcessingTime: (Avg: 138.872ms ; Min: 138.872ms ; Max: 138.872ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 2.81 MB (2945000)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 2.81 MB (2945000)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 2.81 MB (2945000)
- CachedFileHandlesHitCount: 2 (2)
- CachedFileHandlesMissCount: 0 (0)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 9.018ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 366.96 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 149.473ms
- RowBatchQueuePutWaitTime: 2s249ms
- RowsRead: 1.72M (1724693)
- RowsReturned: 1.72M (1724693)
- RowsReturnedRate: 4.04 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 134 (134)
- ScannerThreadsTotalWallClockTime: 2s584ms
- MaterializeTupleTime(*): 161.652ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 160.000ms
- ScannerThreadsVoluntaryContextSwitches: 1.55K (1550)
- TotalRawHdfsReadTime(*): 7.653ms
- TotalReadThroughput: 1.12 MB/sec
Instance 7a4b153d9175fa9a:ab326f0d00000004 (host=lv-desktop:22002):(Total: 2s887ms, non-child: 1s471ms, % non-child: 50.97%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/69.12 MB
Fragment Instance Lifecycle Event Timeline: 2s887ms
- Codegen Finished: 27.823ms (27.823ms)
- Prepare Finished: 46.184ms (18.360ms)
- Open Finished: 60.076ms (13.891ms)
- First Batch Received: 307.610ms (247.534ms)
- First Batch Sent: 308.060ms (450.889us)
- ExecInternal Finished: 2s887ms (2s579ms)
MemoryUsage(500.000ms): 5.21 MB, 5.17 MB, 5.22 MB, 5.18 MB, 5.24 MB
ThreadUsage(500.000ms): 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 5.28 MB (5538896)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 151.94 MB (159323639)
- RowsProduced: 2.14M (2142543)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 15.264ms
- TotalStorageWaitTime: 228.203ms
- TotalThreadsInvoluntaryContextSwitches: 3.13K (3130)
- TotalThreadsTotalWallClockTime: 5s638ms
- TotalThreadsSysTime: 24.000ms
- TotalThreadsUserTime: 1s660ms
- TotalThreadsVoluntaryContextSwitches: 2.06K (2061)
Fragment Instance Lifecycle Timings:
- ExecTime: 2s827ms
- ExecTreeExecTime: 296.507ms
- OpenTime: 13.867ms
- ExecTreeOpenTime: 75.142us
- PrepareTime: 46.184ms
- ExecTreePrepareTime: 1.965ms
DataStreamSender (dst_id=7):(Total: 1s088ms, non-child: 1s088ms, % non-child: 100.00%)
- BytesSent: 11.70 MB (12269777)
- NetworkThroughput(*): 97.11 MB/sec
- OverallThroughput: 10.75 MB/sec
- PeakMemoryUsage: 190.66 KB (195232)
- RowsReturned: 2.14M (2142543)
- SerializeBatchTime: 958.338ms
- TransmitDataRPCTime: 120.496ms
- UncompressedRowBatchSize: 34.74 MB (36423231)
CodeGen:(Total: 29.827ms, non-child: 29.827ms, % non-child: 100.00%)
- CodegenTime: 643.469us
- CompileTime: 1.938ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 10.554ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 16.782ms
HDFS_SCAN_NODE (id=1):(Total: 297.821ms, non-child: 297.821ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/69.12 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0%
File Formats: PARQUET/SNAPPY:1
BytesRead(500.000ms): 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB
- FooterProcessingTime: (Avg: 206.252ms ; Min: 206.252ms ; Max: 206.252ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 3.43 MB (3592722)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 3.43 MB (3592722)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 3.43 MB (3592722)
- CachedFileHandlesHitCount: 0 (0)
- CachedFileHandlesMissCount: 2 (2)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 11.076ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 15.04 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 241.852ms
- RowBatchQueuePutWaitTime: 2s349ms
- RowsRead: 2.14M (2142543)
- RowsReturned: 2.14M (2142543)
- RowsReturnedRate: 7.19 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 113 (113)
- ScannerThreadsTotalWallClockTime: 2s797ms
- MaterializeTupleTime(*): 171.430ms
- ScannerThreadsSysTime: 12.000ms
- ScannerThreadsUserTime: 184.000ms
- ScannerThreadsVoluntaryContextSwitches: 1.98K (1979)
- TotalRawHdfsReadTime(*): 227.790ms
- TotalReadThroughput: 1.37 MB/sec
Averaged Fragment F03:
Fragment F03:
Instance 7a4b153d9175fa9a:ab326f0d00000008 (host=lv-desktop:22000):(Total: 2s478ms, non-child: 777.643ms, % non-child: 31.38%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/68.86 MB
Fragment Instance Lifecycle Event Timeline: 2s478ms
- Codegen Finished: 44.561ms (44.561ms)
- Prepare Finished: 44.696ms (134.416us)
- Open Finished: 63.121ms (18.425ms)
- First Batch Received: 239.228ms (176.106ms)
- First Batch Sent: 239.627ms (398.911us)
- ExecInternal Finished: 2s478ms (2s238ms)
MemoryUsage(500.000ms): 5.14 MB, 5.14 MB, 5.12 MB, 5.14 MB, 1.07 MB
ThreadUsage(500.000ms): 2, 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 5.15 MB (5395536)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 141.06 MB (147912655)
- RowsProduced: 2.13M (2133979)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 272.491ms
- TotalStorageWaitTime: 164.692ms
- TotalThreadsInvoluntaryContextSwitches: 4.48K (4479)
- TotalThreadsTotalWallClockTime: 4s817ms
- TotalThreadsSysTime: 40.000ms
- TotalThreadsUserTime: 952.000ms
- TotalThreadsVoluntaryContextSwitches: 2.39K (2386)
Fragment Instance Lifecycle Timings:
- ExecTime: 2s415ms
- ExecTreeExecTime: 263.769ms
- OpenTime: 18.404ms
- ExecTreeOpenTime: 84.301us
- PrepareTime: 44.694ms
- ExecTreePrepareTime: 11.540ms
DataStreamSender (dst_id=8):(Total: 1s382ms, non-child: 1s382ms, % non-child: 100.00%)
- BytesSent: 34.92 MB (36615942)
- NetworkThroughput(*): 110.25 MB/sec
- OverallThroughput: 25.26 MB/sec
- PeakMemoryUsage: 42.66 KB (43680)
- RowsReturned: 2.13M (2133979)
- SerializeBatchTime: 1s048ms
- TransmitDataRPCTime: 316.734ms
- UncompressedRowBatchSize: 103.79 MB (108832929)
CodeGen:(Total: 34.767ms, non-child: 34.767ms, % non-child: 100.00%)
- CodegenTime: 778.810us
- CompileTime: 7.307ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 9.871ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 17.044ms
HDFS_SCAN_NODE (id=2):(Total: 283.555ms, non-child: 283.555ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/68.86 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0%
File Formats: PARQUET/SNAPPY:1
BytesRead(500.000ms): 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB, 3.42 MB
- FooterProcessingTime: (Avg: 161.946ms ; Min: 161.946ms ; Max: 161.946ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 3.42 MB (3587693)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 3.42 MB (3587693)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 3.42 MB (3587693)
- CachedFileHandlesHitCount: 1 (1)
- CachedFileHandlesMissCount: 1 (1)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 11.178ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 20.44 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 174.782ms
- RowBatchQueuePutWaitTime: 2s000ms
- RowsRead: 2.13M (2133979)
- RowsReturned: 2.13M (2133979)
- RowsReturnedRate: 7.53 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 175 (175)
- ScannerThreadsTotalWallClockTime: 2s384ms
- MaterializeTupleTime(*): 177.478ms
- ScannerThreadsSysTime: 12.000ms
- ScannerThreadsUserTime: 192.000ms
- ScannerThreadsVoluntaryContextSwitches: 1.99K (1993)
- TotalRawHdfsReadTime(*): 167.364ms
- TotalReadThroughput: 1.37 MB/sec
Instance 7a4b153d9175fa9a:ab326f0d00000009 (host=lv-desktop:22001):(Total: 1s951ms, non-child: 535.913ms, % non-child: 27.46%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/55.73 MB
Fragment Instance Lifecycle Event Timeline: 1s956ms
- Codegen Finished: 46.994ms (46.994ms)
- Prepare Finished: 52.774ms (5.779ms)
- Open Finished: 84.603ms (31.829ms)
- First Batch Received: 254.804ms (170.200ms)
- First Batch Sent: 255.168ms (364.019us)
- ExecInternal Finished: 1s956ms (1s701ms)
MemoryUsage(500.000ms): 5.10 MB, 5.14 MB, 5.14 MB, 229.06 KB
ThreadUsage(500.000ms): 2, 2, 2, 1
- AverageThreadTokens: 1.75
- BloomFilterBytes: 0
- PeakMemoryUsage: 5.15 MB (5395536)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 118.74 MB (124506456)
- RowsProduced: 1.72M (1724693)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 318.095ms
- TotalStorageWaitTime: 164.574ms
- TotalThreadsInvoluntaryContextSwitches: 2.97K (2969)
- TotalThreadsTotalWallClockTime: 3s725ms
- TotalThreadsSysTime: 12.000ms
- TotalThreadsUserTime: 792.000ms
- TotalThreadsVoluntaryContextSwitches: 2.11K (2110)
Fragment Instance Lifecycle Timings:
- ExecTime: 1s872ms
- ExecTreeExecTime: 226.112ms
- OpenTime: 31.803ms
- ExecTreeOpenTime: 75.252us
- PrepareTime: 47.767ms
- ExecTreePrepareTime: 15.513ms
DataStreamSender (dst_id=8):(Total: 1s098ms, non-child: 1s098ms, % non-child: 100.00%)
- BytesSent: 28.23 MB (29598276)
- NetworkThroughput(*): 99.10 MB/sec
- OverallThroughput: 25.68 MB/sec
- PeakMemoryUsage: 42.66 KB (43680)
- RowsReturned: 1.72M (1724693)
- SerializeBatchTime: 812.785ms
- TransmitDataRPCTime: 284.845ms
- UncompressedRowBatchSize: 83.88 MB (87959343)
CodeGen:(Total: 61.653ms, non-child: 61.653ms, % non-child: 100.00%)
- CodegenTime: 670.432us
- CompileTime: 18.885ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 9.628ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 32.616ms
HDFS_SCAN_NODE (id=2):(Total: 255.109ms, non-child: 255.109ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/55.73 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0%
File Formats: PARQUET/SNAPPY:1
BytesRead(500.000ms): 2.81 MB, 2.81 MB, 2.81 MB, 2.81 MB
- FooterProcessingTime: (Avg: 157.149ms ; Min: 157.149ms ; Max: 157.149ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 0.75
- BytesRead: 2.81 MB (2945000)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 2.81 MB (2945000)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 2.81 MB (2945000)
- CachedFileHandlesHitCount: 1 (1)
- CachedFileHandlesMissCount: 1 (1)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 9.264ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 17.92 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 169.819ms
- RowBatchQueuePutWaitTime: 1s474ms
- RowsRead: 1.72M (1724693)
- RowsReturned: 1.72M (1724693)
- RowsReturnedRate: 6.76 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 95 (95)
- ScannerThreadsTotalWallClockTime: 1s821ms
- MaterializeTupleTime(*): 150.587ms
- ScannerThreadsSysTime: 4.000ms
- ScannerThreadsUserTime: 164.000ms
- ScannerThreadsVoluntaryContextSwitches: 1.63K (1634)
- TotalRawHdfsReadTime(*): 156.770ms
- TotalReadThroughput: 1.40 MB/sec
Instance 7a4b153d9175fa9a:ab326f0d00000007 (host=lv-desktop:22002):(Total: 2s421ms, non-child: 823.424ms, % non-child: 34.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/69.12 MB
Fragment Instance Lifecycle Event Timeline: 2s421ms
- Codegen Finished: 31.688ms (31.688ms)
- Prepare Finished: 36.425ms (4.736ms)
- Open Finished: 55.885ms (19.460ms)
- First Batch Received: 303.808ms (247.923ms)
- First Batch Sent: 304.182ms (373.682us)
- ExecInternal Finished: 2s421ms (2s117ms)
MemoryUsage(500.000ms): 5.14 MB, 5.14 MB, 5.14 MB, 5.14 MB
ThreadUsage(500.000ms): 2, 2, 2, 2
- AverageThreadTokens: 2.00
- BloomFilterBytes: 0
- PeakMemoryUsage: 5.15 MB (5395536)
- PeakReservation: 0
- PeakUsedReservation: 0
- PerHostPeakMemUsage: 138.21 MB (144924116)
- RowsProduced: 2.14M (2142543)
- TotalNetworkReceiveTime: 0.000ns
- TotalNetworkSendTime: 518.677ms
- TotalStorageWaitTime: 214.639ms
- TotalThreadsInvoluntaryContextSwitches: 3.05K (3047)
- TotalThreadsTotalWallClockTime: 4s680ms
- TotalThreadsSysTime: 20.000ms
- TotalThreadsUserTime: 972.000ms
- TotalThreadsVoluntaryContextSwitches: 2.77K (2772)
Fragment Instance Lifecycle Timings:
- ExecTime: 2s365ms
- ExecTreeExecTime: 280.598ms
- OpenTime: 19.437ms
- ExecTreeOpenTime: 100.204us
- PrepareTime: 36.414ms
- ExecTreePrepareTime: 1.965ms
DataStreamSender (dst_id=8):(Total: 1s276ms, non-child: 1s276ms, % non-child: 100.00%)
- BytesSent: 35.05 MB (36757677)
- NetworkThroughput(*): 106.95 MB/sec
- OverallThroughput: 27.46 MB/sec
- PeakMemoryUsage: 42.66 KB (43680)
- RowsReturned: 2.14M (2142543)
- SerializeBatchTime: 942.476ms
- TransmitDataRPCTime: 327.764ms
- UncompressedRowBatchSize: 104.21 MB (109269693)
CodeGen:(Total: 40.180ms, non-child: 40.180ms, % non-child: 100.00%)
- CodegenTime: 3.357ms
- CompileTime: 3.847ms
- LoadTime: 0.000ns
- ModuleBitcodeSize: 1.89 MB (1980068)
- NumFunctions: 7 (7)
- NumInstructions: 105 (105)
- OptimizationTime: 14.055ms
- PeakMemoryUsage: 52.50 KB (53760)
- PrepareTime: 21.775ms
HDFS_SCAN_NODE (id=2):(Total: 281.837ms, non-child: 281.837ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/69.12 MB
ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0%
File Formats: PARQUET/SNAPPY:1
BytesRead(500.000ms): 3.43 MB, 3.43 MB, 3.43 MB, 3.43 MB
- FooterProcessingTime: (Avg: 205.742ms ; Min: 205.742ms ; Max: 205.742ms ; Number of samples: 1)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 3.43 MB (3592722)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 3.43 MB (3592722)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 3.43 MB (3592722)
- CachedFileHandlesHitCount: 2 (2)
- CachedFileHandlesMissCount: 0 (0)
- CollectionItemsRead: 0 (0)
- DecompressionTime: 11.318ms
- MaxCompressedTextFileLength: 0
- NumColumns: 1 (1)
- NumDictFilteredRowGroups: 0 (0)
- NumDisksAccessed: 1 (1)
- NumRowGroups: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- NumScannersWithNoReads: 0 (0)
- NumStatsFilteredRowGroups: 0 (0)
- PeakMemoryUsage: 5.10 MB (5343249)
- PerReadThreadRawHdfsThroughput: 15.96 MB/sec
- RemoteScanRanges: 0 (0)
- RowBatchQueueGetWaitTime: 226.375ms
- RowBatchQueuePutWaitTime: 1s866ms
- RowsRead: 2.14M (2142543)
- RowsReturned: 2.14M (2142543)
- RowsReturnedRate: 7.60 M/sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 118 (118)
- ScannerThreadsTotalWallClockTime: 2s295ms
- MaterializeTupleTime(*): 173.390ms
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 204.000ms
- ScannerThreadsVoluntaryContextSwitches: 1.99K (1993)
- TotalRawHdfsReadTime(*): 214.688ms
- TotalReadThroughput: 1.71 MB/sec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment