Skip to content

Instantly share code, notes, and snippets.

@sobrien05

sobrien05/gist:4cdeeb85d314d08d4e2a Secret

Created January 21, 2016 00:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sobrien05/4cdeeb85d314d08d4e2a to your computer and use it in GitHub Desktop.
Save sobrien05/4cdeeb85d314d08d4e2a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
Query (id=4242a648b2529ccb:b503ec709811c7ba)
Summary
Session ID: b84fe6993f1ae43c:4e7916dd584f0ad
Session Type: BEESWAX
Start Time: 2016-01-20 12:22:15.825683000
End Time: 2016-01-20 12:33:34.061228000
Query Type: DML
Query State: FINISHED
Query Status: OK
Impala Version: impalad version 2.3.0-cdh5.5.1 RELEASE (build 73bf5bc5afbb47aa7eab06cfbf6023ba8cb74f3c)
User: hadoop
Connected User: hadoop
Delegated User:
Network Address: 192.168.30.134:28054
Default Db: default
Sql Statement: insert OVERWRITE `logs`.`buzz_house`
SELECT user_id, B.house_id, style, category_id
FROM c2.buzz_p B, c2.houses_p H
WHERE B.house_id=H.house_id AND B.created > from_unixtime(unix_timestamp(now() - interval 365 days), 'yyyy-MM-dd HH:mm:ss.S')
Coordinator: hdws70.houzz.com:22000
Plan:
----------------
Estimated Per-Host Requirements: Memory=303.46MB VCores=2
F02:PLAN FRAGMENT [HASH(B.house_id)]
WRITE TO HDFS [logs.buzz_house, OVERWRITE=true]
| partitions=1
| hosts=15 per-host-mem=100.00KB
|
02:HASH JOIN [INNER JOIN, PARTITIONED]
| hash predicates: B.house_id = H.house_id
| hosts=15 per-host-mem=39.37MB
| tuple-ids=0,1 row-size=57B cardinality=81411008
|
|--04:EXCHANGE [HASH(H.house_id)]
| hosts=15 per-host-mem=0B
| tuple-ids=1 row-size=12B cardinality=46906905
|
03:EXCHANGE [HASH(B.house_id)]
hosts=15 per-host-mem=0B
tuple-ids=0 row-size=45B cardinality=81411008
F01:PLAN FRAGMENT [RANDOM]
DATASTREAM SINK [FRAGMENT=F02, EXCHANGE=04, HASH(H.house_id)]
01:SCAN HDFS [c2.houses_p h, RANDOM]
partitions=1/1 files=23 size=3.53GB
table stats: 46906905 rows total
column stats: all
hosts=15 per-host-mem=264.00MB
tuple-ids=1 row-size=12B cardinality=46906905
F00:PLAN FRAGMENT [RANDOM]
DATASTREAM SINK [FRAGMENT=F02, EXCHANGE=03, HASH(B.house_id)]
00:SCAN HDFS [c2.buzz_p b, RANDOM]
partitions=1/1 files=79 size=17.87GB
predicates: B.created > from_unixtime(unix_timestamp(now() - INTERVAL 365 days), 'yyyy-MM-dd HH:mm:ss.S')
table stats: 814110075 rows total
column stats: all
hosts=15 per-host-mem=264.00MB
tuple-ids=0 row-size=45B cardinality=81411008
----------------
Estimated Per-Host Mem: 318204537
Estimated Per-Host VCores: 2
Request Pool: default-pool
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
-----------------------------------------------------------------------------------------------------------------------
02:HASH JOIN 15 9s158ms 10s863ms 203.06M 81.41M 267.02 MB 39.37 MB INNER JOIN, PARTITIONED
|--04:EXCHANGE 15 313.219ms 788.728ms 46.91M 46.91M 0 0 HASH(H.house_id)
| 01:SCAN HDFS 15 207.593ms 960.248ms 46.91M 46.91M 50.67 MB 264.00 MB c2.houses_p h
03:EXCHANGE 15 2s738ms 4s115ms 203.06M 81.41M 0 0 HASH(B.house_id)
00:SCAN HDFS 15 5s019ms 9s324ms 203.06M 81.41M 443.88 MB 264.00 MB c2.buzz_p b
Planner Timeline
Analysis finished: 6976261
Equivalence classes computed: 7203074
Single node plan created: 13406110
Distributed plan created: 14632817
Lineage info computed: 15831085
Planning finished: 18280140
Query Timeline
Start execution: 95279
Planning finished: 24351737
Ready to start remote fragments: 25436092
Remote fragments started: 420868419
DML data written: 657098694091
DML Metastore update finished: 677153105110
Request finished: 677153267711
Unregister query: 678235716954
ImpalaServer
- ClientFetchWaitTimer: 1081308644
- InactiveTotalTime: 0
- MetastoreUpdateTimer: 20054526653
- RowMaterializationTimer: 0
- TotalTime: 0
Execution Profile 4242a648b2529ccb:b503ec709811c7ba
Fragment start latencies: count: 45, last: 0.051709ns, min: 0.001798ns, max: 0.272138ns, mean: 0.017740ns, stddev: 0.041229ns
Insert Stats:
Partition: Default
BytesWritten: 4.54 GB
Per Node Peak Memory Usage: hdws82.houzz.com:22000(633.25 MB) hdws70.houzz.com:22000(664.92 MB) hdws81.houzz.com:22000(632.42 MB) hdws74.houzz.com:22000(612.82 MB) hdws86.houzz.com:22000(639.79 MB) hdws72.houzz.com:22000(597.04 MB) hdws83.houzz.com:22000(710.63 MB) hdws75.houzz.com:22000(606.34 MB) hdws73.houzz.com:22000(617.18 MB) hdws67.houzz.com:22000(623.99 MB) hdws80.houzz.com:22000(670.54 MB) hdws78.houzz.com:22000(631.45 MB) hdws71.houzz.com:22000(603.24 MB) hdws69.houzz.com:22000(677.64 MB) hdws84.houzz.com:22000(561.74 MB)
- FinalizationTimer: 1297403460
- FileDeletionTimer: 598546752
- FileMoveTimer: 605166446
- Overwrite/PartitionCreationTimer: 56494384
- InactiveTotalTime: 0
- TotalTime: 657072967420
Averaged Fragment F02
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:10m54s max:10m55s mean: 10m55s stddev:161.814203.814.202746ms
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 15
- AverageThreadTokens: 1.1365603480457342
- InactiveTotalTime: 0
- PeakMemoryUsage: 300003396
- PerHostPeakMemUsage: 662908821
- PrepareTime: 365869
- RowsProduced: 13537168
- TotalCpuTime: 127540262193
- TotalNetworkReceiveTime: 615465772418
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 652654535672
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 325094744
- CompressTimer: 0
- EncodeTimer: 13788395677
- FilesCreated: 1
- FinalizePartitionFileTimer: 115972656
- HdfsWriteTimer: 13092276142
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13537168
- TmpFileCreateTimer: 116753795
- TotalTime: 27580849562
HASH_JOIN_NODE (id=2)
- BuildPartitionTime: 1249340985
- BuildRows: 3127127
- BuildRowsPartitioned: 3127127
- BuildTime: 390657881
- GetNewBlockTime: 3527219
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 9158539602
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13537175
- ProbeRowsPartitioned: 0
- ProbeTime: 6826082859
- RowsReturned: 13537168
- RowsReturnedRate: 21674
- SpilledPartitions: 0
- TotalTime: 624737879489
- UnpinTime: 2303
EXCHANGE_NODE (id=4)
- BytesReceived: 39340736
- ConvertRowBatchTime: 138813037
- DeserializeRowBatchTimer: 606773268
- FirstBatchArrivalWaitTime: 265249088
- InactiveTotalTime: 0
- PeakMemoryUsage: 0
- RowsReturned: 3127127
- RowsReturnedRate: 36441
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 85816807836
EXCHANGE_NODE (id=3)
- BytesReceived: 479653242
- ConvertRowBatchTime: 448834298
- DeserializeRowBatchTimer: 7659599333
- FirstBatchArrivalWaitTime: 644789462
- InactiveTotalTime: 0
- PeakMemoryUsage: 0
- RowsReturned: 13537175
- RowsReturnedRate: 25537
- SendersBlockedTimer: 7042720781
- SendersBlockedTotalTimer(*): 95192734639
- TotalTime: 530323166607
Averaged Fragment F01
split sizes: min: 4.06 MB, max: 535.88 MB, avg: 240.97 MB, stddev: 131.40 MB
completion times: min:113.409ms max:1m26s mean: 37s001ms stddev:21s791ms
execution rates: min:5.57 MB/sec max:35.84 MB/sec mean:8.59 MB/sec stddev:7.33 MB/sec
num instances: 15
- AverageThreadTokens: 1.9544692419539305
- InactiveTotalTime: 0
- PeakMemoryUsage: 30143811
- PerHostPeakMemUsage: 462854761
- PrepareTime: 2693729
- RowsProduced: 3127127
- TotalCpuTime: 51047052878
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 34863029046
- TotalStorageWaitTime: 61641856
- TotalTime: 36383374763
DataStreamSender (dst_id=4)
- BytesSent: 39340736
- InactiveTotalTime: 0
- NetworkThroughput(*): 1441837
- OverallThroughput: 4765506
- PeakMemoryUsage: 552960
- SerializeBatchTime: 593440627
- ThriftTransmitTime(*): 58027201353
- TotalTime: 36165848867
- UncompressedRowBatchSize: 62554784
HDFS_SCAN_NODE (id=1)
- AverageHdfsReadThreadConcurrency: 8.547008547008547E-4
- AverageScannerThreadConcurrency: 1.0337532886038543
- BytesRead: 17702286
- BytesReadDataNodeCache: 0
- BytesReadLocal: 17702286
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 17702286
- DecompressionTime: 63168935
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 4
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 29669455
- PerReadThreadRawHdfsThroughput: 413237972
- RemoteScanRanges: 0
- RowsRead: 3127127
- RowsReturned: 3127127
- RowsReturnedRate: 22989627
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 996
- ScannerThreadsTotalWallClockTime: 48967884751
- MaterializeTupleTime(*): 48712706565
- ScannerThreadsSysTime: 105606000
- ScannerThreadsUserTime: 583235933
- ScannerThreadsVoluntaryContextSwitches: 1948
- TotalRawHdfsReadTime(*): 92248443
- TotalReadThroughput: 502141
- TotalTime: 207593189
Averaged Fragment F00
split sizes: min: 636.19 MB, max: 1.79 GB, avg: 1.19 GB, stddev: 257.13 MB
completion times: min:2m32s max:10m54s mean: 7m9s stddev:2m20s
execution rates: min:1.85 MB/sec max:6.56 MB/sec mean:3.20 MB/sec stddev:1.34 MB/sec
num instances: 15
- AverageThreadTokens: 4.194296125073884
- InactiveTotalTime: 0
- PeakMemoryUsage: 379056806
- PerHostPeakMemUsage: 662908821
- PrepareTime: 407720
- RowsProduced: 13537175
- TotalCpuTime: 1391027494168
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 408367940837
- TotalStorageWaitTime: 6765197015
- TotalTime: 427563066444
DataStreamSender (dst_id=3)
- BytesSent: 479653242
- InactiveTotalTime: 0
- NetworkThroughput(*): 684752
- OverallThroughput: 1077149
- PeakMemoryUsage: 552686
- SerializeBatchTime: 7535011965
- ThriftTransmitTime(*): 674310388808
- TotalTime: 422481749833
- UncompressedRowBatchSize: 771724796
HDFS_SCAN_NODE (id=0)
- AverageHdfsReadThreadConcurrency: 0.06741814437808509
- AverageScannerThreadConcurrency: 3.197699640204059
- BytesRead: 758279825
- BytesReadDataNodeCache: 0
- BytesReadLocal: 286135576
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 286135576
- DecompressionTime: 12481212178
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 30
- NumDisksAccessed: 5
- NumRowGroups: 5
- NumScannerThreadsStarted: 5
- PeakMemoryUsage: 378519300
- PerReadThreadRawHdfsThroughput: 38270588
- RemoteScanRanges: 17
- RowsRead: 54274005
- RowsReturned: 13537175
- RowsReturnedRate: 3104688
- ScanRangesComplete: 10
- ScannerThreadsInvoluntaryContextSwitches: 37189
- ScannerThreadsTotalWallClockTime: 1376327192424
- MaterializeTupleTime(*): 1338287915800
- ScannerThreadsSysTime: 19788701400
- ScannerThreadsUserTime: 182721817400
- ScannerThreadsVoluntaryContextSwitches: 429277
- TotalRawHdfsReadTime(*): 28443239718
- TotalReadThroughput: 2198005
- TotalTime: 5019549112
Fragment F02
Instance 4242a648b2529ccb:b503ec709811c7bd (host=hdws71.houzz.com:22000)
- AverageThreadTokens: 1.13520197856554
- InactiveTotalTime: 0
- PeakMemoryUsage: 299936768
- PerHostPeakMemUsage: 632540416
- PrepareTime: 168795
- RowsProduced: 13090456
- TotalCpuTime: 121452473263
- TotalNetworkReceiveTime: 621940226415
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 654690856555
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 313921246
- CompressTimer: 0
- EncodeTimer: 11746005380
- FilesCreated: 1
- FinalizePartitionFileTimer: 648367369
- HdfsWriteTimer: 12974509248
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13090456
- TmpFileCreateTimer: 34681616
- TotalTime: 25577828547
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 786201946
- BuildRows: 3126829
- BuildRowsPartitioned: 3126829
- BuildTime: 379918735
- GetNewBlockTime: 1884761
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 7587140227
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13090459
- ProbeRowsPartitioned: 0
- ProbeTime: 6217530037
- RowsReturned: 13090456
- RowsReturnedRate: 20809
- SpilledPartitions: 0
- TotalTime: 629069435090
- UnpinTime: 820
EXCHANGE_NODE (id=4)
- BytesReceived: 39334117
- ConvertRowBatchTime: 139436854
- DeserializeRowBatchTimer: 376953888
- FirstBatchArrivalWaitTime: 290918151
- InactiveTotalTime: 85984316031
- PeakMemoryUsage: 0
- RowsReturned: 3126829
- RowsReturnedRate: 36229
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86305559907
EXCHANGE_NODE (id=3)
- BytesReceived: 464195098
- ConvertRowBatchTime: 442414610
- DeserializeRowBatchTimer: 6391120555
- FirstBatchArrivalWaitTime: 1154024115
- InactiveTotalTime: 533566962783
- PeakMemoryUsage: 0
- RowsReturned: 13090459
- RowsReturnedRate: 24409
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 536295051001
Instance 4242a648b2529ccb:b503ec709811c7bb (host=hdws84.houzz.com:22000)
- AverageThreadTokens: 1.1289256198347108
- InactiveTotalTime: 0
- PeakMemoryUsage: 301217280
- PerHostPeakMemUsage: 589027256
- PrepareTime: 136957
- RowsProduced: 14026234
- TotalCpuTime: 129935137904
- TotalNetworkReceiveTime: 613449339205
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 654414641292
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 337777162
- CompressTimer: 0
- EncodeTimer: 15747160726
- FilesCreated: 1
- FinalizePartitionFileTimer: 71415901
- HdfsWriteTimer: 15105265200
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 14026234
- TmpFileCreateTimer: 37021722
- TotalTime: 31142200937
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 743904594
- BuildRows: 3129358
- BuildRowsPartitioned: 3129358
- BuildTime: 330108841
- GetNewBlockTime: 32762619
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 9568418945
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 14026241
- ProbeRowsPartitioned: 0
- ProbeTime: 6918113360
- RowsReturned: 14026234
- RowsReturnedRate: 22506
- SpilledPartitions: 0
- TotalTime: 623212886854
- UnpinTime: 1433
EXCHANGE_NODE (id=4)
- BytesReceived: 39372618
- ConvertRowBatchTime: 148405930
- DeserializeRowBatchTimer: 356537739
- FirstBatchArrivalWaitTime: 290614383
- InactiveTotalTime: 86042402230
- PeakMemoryUsage: 0
- RowsReturned: 3129358
- RowsReturnedRate: 36256
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86312224625
EXCHANGE_NODE (id=3)
- BytesReceived: 496533356
- ConvertRowBatchTime: 442086646
- DeserializeRowBatchTimer: 8054324426
- FirstBatchArrivalWaitTime: 565719292
- InactiveTotalTime: 525482459147
- PeakMemoryUsage: 0
- RowsReturned: 14026241
- RowsReturnedRate: 26573
- SendersBlockedTimer: 79810220057
- SendersBlockedTotalTimer(*): 1195770566869
- TotalTime: 527823067013
Instance 4242a648b2529ccb:b503ec709811c7c6 (host=hdws74.houzz.com:22000)
- AverageThreadTokens: 1.1348122866894197
- InactiveTotalTime: 0
- PeakMemoryUsage: 300561408
- PerHostPeakMemUsage: 642588048
- PrepareTime: 236557
- RowsProduced: 13306315
- TotalCpuTime: 128474321618
- TotalNetworkReceiveTime: 614354574020
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 654335515118
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 319730871
- CompressTimer: 0
- EncodeTimer: 12123166140
- FilesCreated: 1
- FinalizePartitionFileTimer: 76649656
- HdfsWriteTimer: 14601775110
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13306315
- TmpFileCreateTimer: 54686505
- TotalTime: 27679960557
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 1371726425
- BuildRows: 3125714
- BuildRowsPartitioned: 3125714
- BuildTime: 375141004
- GetNewBlockTime: 1125576
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 9391436575
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13306324
- ProbeRowsPartitioned: 0
- ProbeTime: 7432914172
- RowsReturned: 13306315
- RowsReturnedRate: 21327
- SpilledPartitions: 0
- TotalTime: 623916277895
- UnpinTime: 2208
EXCHANGE_NODE (id=4)
- BytesReceived: 39320348
- ConvertRowBatchTime: 158486618
- DeserializeRowBatchTimer: 673466378
- FirstBatchArrivalWaitTime: 289191794
- InactiveTotalTime: 85414967944
- PeakMemoryUsage: 0
- RowsReturned: 3125714
- RowsReturnedRate: 36468
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 85710413227
EXCHANGE_NODE (id=3)
- BytesReceived: 471799697
- ConvertRowBatchTime: 444716218
- DeserializeRowBatchTimer: 8430430586
- FirstBatchArrivalWaitTime: 563210685
- InactiveTotalTime: 525214602383
- PeakMemoryUsage: 0
- RowsReturned: 13306324
- RowsReturnedRate: 25138
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 529330222677
Instance 4242a648b2529ccb:b503ec709811c7be (host=hdws78.houzz.com:22000)
- AverageThreadTokens: 1.1372226787181594
- InactiveTotalTime: 0
- PeakMemoryUsage: 299999232
- PerHostPeakMemUsage: 662126184
- PrepareTime: 784195
- RowsProduced: 13390744
- TotalCpuTime: 164184165610
- TotalNetworkReceiveTime: 579168943288
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 653696672417
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 322004424
- CompressTimer: 0
- EncodeTimer: 16519457491
- FilesCreated: 1
- FinalizePartitionFileTimer: 88073207
- HdfsWriteTimer: 47378716823
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13390744
- TmpFileCreateTimer: 39590039
- TotalTime: 64197860772
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 2730045764
- BuildRows: 3130065
- BuildRowsPartitioned: 3130065
- BuildTime: 368554597
- GetNewBlockTime: 1989737
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 10089688093
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13390753
- ProbeRowsPartitioned: 0
- ProbeTime: 6134955202
- RowsReturned: 13390744
- RowsReturnedRate: 22717
- SpilledPartitions: 0
- TotalTime: 589441977281
- UnpinTime: 1250
EXCHANGE_NODE (id=4)
- BytesReceived: 39365673
- ConvertRowBatchTime: 130817851
- DeserializeRowBatchTimer: 1162610930
- FirstBatchArrivalWaitTime: 232302883
- InactiveTotalTime: 84024813121
- PeakMemoryUsage: 0
- RowsReturned: 3130065
- RowsReturnedRate: 37128
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 84304362705
EXCHANGE_NODE (id=3)
- BytesReceived: 474735291
- ConvertRowBatchTime: 430222717
- DeserializeRowBatchTimer: 7774491346
- FirstBatchArrivalWaitTime: 503696150
- InactiveTotalTime: 493620564463
- PeakMemoryUsage: 0
- RowsReturned: 13390753
- RowsReturnedRate: 27024
- SendersBlockedTimer: 21881781710
- SendersBlockedTotalTimer(*): 196856849757
- TotalTime: 495508735018
Instance 4242a648b2529ccb:b503ec709811c7c3 (host=hdws83.houzz.com:22000)
- AverageThreadTokens: 1.1398963730569949
- InactiveTotalTime: 0
- PeakMemoryUsage: 300249088
- PerHostPeakMemUsage: 745151136
- PrepareTime: 232307
- RowsProduced: 14215771
- TotalCpuTime: 124674921945
- TotalNetworkReceiveTime: 618114659965
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 653499143247
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 341333777
- CompressTimer: 0
- EncodeTimer: 13188507983
- FilesCreated: 1
- FinalizePartitionFileTimer: 78160680
- HdfsWriteTimer: 13410146998
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 14215771
- TmpFileCreateTimer: 45834186
- TotalTime: 26925140239
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 981041836
- BuildRows: 3126051
- BuildRowsPartitioned: 3126051
- BuildTime: 335547224
- GetNewBlockTime: 945051
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 8256311769
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 14215781
- ProbeRowsPartitioned: 0
- ProbeTime: 6730208696
- RowsReturned: 14215771
- RowsReturnedRate: 22689
- SpilledPartitions: 0
- TotalTime: 626525977616
- UnpinTime: 3619
EXCHANGE_NODE (id=4)
- BytesReceived: 39329360
- ConvertRowBatchTime: 126576044
- DeserializeRowBatchTimer: 318433514
- FirstBatchArrivalWaitTime: 290263983
- InactiveTotalTime: 85865854557
- PeakMemoryUsage: 0
- RowsReturned: 3126051
- RowsReturnedRate: 36300
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86115794370
EXCHANGE_NODE (id=3)
- BytesReceived: 503038946
- ConvertRowBatchTime: 469033439
- DeserializeRowBatchTimer: 7338413405
- FirstBatchArrivalWaitTime: 563256305
- InactiveTotalTime: 530892670936
- PeakMemoryUsage: 0
- RowsReturned: 14215781
- RowsReturnedRate: 26687
- SendersBlockedTimer: 512607951
- SendersBlockedTotalTimer(*): 2860698095
- TotalTime: 532683034259
Instance 4242a648b2529ccb:b503ec709811c7c1 (host=hdws73.houzz.com:22000)
- AverageThreadTokens: 1.1355060034305318
- InactiveTotalTime: 0
- PeakMemoryUsage: 298468864
- PerHostPeakMemUsage: 647160336
- PrepareTime: 187054
- RowsProduced: 12993990
- TotalCpuTime: 116311474675
- TotalNetworkReceiveTime: 626473054906
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 653134725331
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 311982982
- CompressTimer: 0
- EncodeTimer: 12395532549
- FilesCreated: 1
- FinalizePartitionFileTimer: 74586859
- HdfsWriteTimer: 5734387390
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 12993990
- TmpFileCreateTimer: 50792334
- TotalTime: 18455605361
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 754496509
- BuildRows: 3126018
- BuildRowsPartitioned: 3126018
- BuildTime: 337570698
- GetNewBlockTime: 1419727
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 8014198336
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 12993995
- ProbeRowsPartitioned: 0
- ProbeTime: 6713545655
- RowsReturned: 12993990
- RowsReturnedRate: 20475
- SpilledPartitions: 0
- TotalTime: 634623336461
- UnpinTime: 4522
EXCHANGE_NODE (id=4)
- BytesReceived: 39323025
- ConvertRowBatchTime: 120802674
- DeserializeRowBatchTimer: 342529358
- FirstBatchArrivalWaitTime: 289068124
- InactiveTotalTime: 86074025820
- PeakMemoryUsage: 0
- RowsReturned: 3126018
- RowsReturnedRate: 36205
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86340889846
EXCHANGE_NODE (id=3)
- BytesReceived: 461188667
- ConvertRowBatchTime: 452439229
- DeserializeRowBatchTimer: 7739451826
- FirstBatchArrivalWaitTime: 559924672
- InactiveTotalTime: 538280205927
- PeakMemoryUsage: 0
- RowsReturned: 12993995
- RowsReturnedRate: 24027
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 540794298756
Instance 4242a648b2529ccb:b503ec709811c7c2 (host=hdws75.houzz.com:22000)
- AverageThreadTokens: 1.1341156747694887
- InactiveTotalTime: 0
- PeakMemoryUsage: 299249664
- PerHostPeakMemUsage: 635793208
- PrepareTime: 166731
- RowsProduced: 13408781
- TotalCpuTime: 122532735158
- TotalNetworkReceiveTime: 620248432386
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 653032859373
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 322268515
- CompressTimer: 0
- EncodeTimer: 14181345416
- FilesCreated: 1
- FinalizePartitionFileTimer: 77718454
- HdfsWriteTimer: 8525687664
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13408781
- TmpFileCreateTimer: 54089203
- TotalTime: 23036396635
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 810437390
- BuildRows: 3129121
- BuildRowsPartitioned: 3129121
- BuildTime: 339167232
- GetNewBlockTime: 1538513
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 9488339274
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13408787
- ProbeRowsPartitioned: 0
- ProbeTime: 6508021061
- RowsReturned: 13408781
- RowsReturnedRate: 21285
- SpilledPartitions: 0
- TotalTime: 629942696785
- UnpinTime: 2387
EXCHANGE_NODE (id=4)
- BytesReceived: 39369213
- ConvertRowBatchTime: 166275024
- DeserializeRowBatchTimer: 457273415
- FirstBatchArrivalWaitTime: 288856802
- InactiveTotalTime: 85835797083
- PeakMemoryUsage: 0
- RowsReturned: 3129121
- RowsReturnedRate: 36267
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86278028705
EXCHANGE_NODE (id=3)
- BytesReceived: 475458941
- ConvertRowBatchTime: 447384532
- DeserializeRowBatchTimer: 6841029438
- FirstBatchArrivalWaitTime: 564755876
- InactiveTotalTime: 530991440000
- PeakMemoryUsage: 0
- RowsReturned: 13408787
- RowsReturnedRate: 25078
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 534679889154
Instance 4242a648b2529ccb:b503ec709811c7c5 (host=hdws86.houzz.com:22000)
- AverageThreadTokens: 1.135483870967742
- InactiveTotalTime: 0
- PeakMemoryUsage: 300655104
- PerHostPeakMemUsage: 670865104
- PrepareTime: 194258
- RowsProduced: 13302295
- TotalCpuTime: 117833219765
- TotalNetworkReceiveTime: 624902633715
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 653005895760
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 319073299
- CompressTimer: 0
- EncodeTimer: 12734362076
- FilesCreated: 1
- FinalizePartitionFileTimer: 78679313
- HdfsWriteTimer: 4735630468
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13302295
- TmpFileCreateTimer: 45910473
- TotalTime: 17781789640
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 1329346767
- BuildRows: 3127033
- BuildRowsPartitioned: 3127033
- BuildTime: 358514500
- GetNewBlockTime: 1079136
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 10136219686
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13302301
- ProbeRowsPartitioned: 0
- ProbeTime: 8241160214
- RowsReturned: 13302295
- RowsReturnedRate: 20942
- SpilledPartitions: 0
- TotalTime: 635175581360
- UnpinTime: 1134
EXCHANGE_NODE (id=4)
- BytesReceived: 39336259
- ConvertRowBatchTime: 125171909
- DeserializeRowBatchTimer: 1464647159
- FirstBatchArrivalWaitTime: 249611291
- InactiveTotalTime: 84939752705
- PeakMemoryUsage: 0
- RowsReturned: 3127033
- RowsReturnedRate: 36476
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 85728481105
EXCHANGE_NODE (id=3)
- BytesReceived: 471452740
- ConvertRowBatchTime: 430594547
- DeserializeRowBatchTimer: 8022253914
- FirstBatchArrivalWaitTime: 523855084
- InactiveTotalTime: 536857823120
- PeakMemoryUsage: 0
- RowsReturned: 13302301
- RowsReturnedRate: 24642
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 539804132939
Instance 4242a648b2529ccb:b503ec709811c7c9 (host=hdws82.houzz.com:22000)
- AverageThreadTokens: 1.1368852459016394
- InactiveTotalTime: 0
- PeakMemoryUsage: 298874880
- PerHostPeakMemUsage: 664006024
- PrepareTime: 212919
- RowsProduced: 13275814
- TotalCpuTime: 117763723165
- TotalNetworkReceiveTime: 625019455077
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 652845741411
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 319167487
- CompressTimer: 0
- EncodeTimer: 14085461031
- FilesCreated: 1
- FinalizePartitionFileTimer: 75941462
- HdfsWriteTimer: 5317336864
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13275814
- TmpFileCreateTimer: 48915877
- TotalTime: 19709624027
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 584008667
- BuildRows: 3125129
- BuildRowsPartitioned: 3125129
- BuildTime: 334913148
- GetNewBlockTime: 1570197
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 7978293868
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13275823
- ProbeRowsPartitioned: 0
- ProbeTime: 6112822463
- RowsReturned: 13275814
- RowsReturnedRate: 20970
- SpilledPartitions: 0
- TotalTime: 633084019420
- UnpinTime: 1559
EXCHANGE_NODE (id=4)
- BytesReceived: 39310072
- ConvertRowBatchTime: 122729974
- DeserializeRowBatchTimer: 861315539
- FirstBatchArrivalWaitTime: 288732060
- InactiveTotalTime: 86279674103
- PeakMemoryUsage: 0
- RowsReturned: 3125129
- RowsReturnedRate: 36123
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86513356676
EXCHANGE_NODE (id=3)
- BytesReceived: 470929618
- ConvertRowBatchTime: 414454577
- DeserializeRowBatchTimer: 9926429923
- FirstBatchArrivalWaitTime: 563229353
- InactiveTotalTime: 535517472711
- PeakMemoryUsage: 0
- RowsReturned: 13275823
- RowsReturnedRate: 24624
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 539130391448
Instance 4242a648b2529ccb:b503ec709811c7c0 (host=hdws67.houzz.com:22000)
- AverageThreadTokens: 1.136443661971831
- InactiveTotalTime: 0
- PeakMemoryUsage: 299530752
- PerHostPeakMemUsage: 654302608
- PrepareTime: 135553
- RowsProduced: 13661258
- TotalCpuTime: 129897905889
- TotalNetworkReceiveTime: 612964928612
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 652423070874
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 328218652
- CompressTimer: 0
- EncodeTimer: 12812035434
- FilesCreated: 1
- FinalizePartitionFileTimer: 76178747
- HdfsWriteTimer: 14375470955
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13661258
- TmpFileCreateTimer: 48483074
- TotalTime: 27512783164
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 2110449550
- BuildRows: 3127966
- BuildRowsPartitioned: 3127966
- BuildTime: 400150234
- GetNewBlockTime: 1660155
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 10863572180
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13661261
- ProbeRowsPartitioned: 0
- ProbeTime: 7313284642
- RowsReturned: 13661258
- RowsReturnedRate: 21893
- SpilledPartitions: 0
- TotalTime: 624000132976
- UnpinTime: 1333
EXCHANGE_NODE (id=4)
- BytesReceived: 39355486
- ConvertRowBatchTime: 140739992
- DeserializeRowBatchTimer: 298962099
- FirstBatchArrivalWaitTime: 290948620
- InactiveTotalTime: 84724325742
- PeakMemoryUsage: 0
- RowsReturned: 3127966
- RowsReturnedRate: 36805
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 84985647443
EXCHANGE_NODE (id=3)
- BytesReceived: 484218393
- ConvertRowBatchTime: 473030327
- DeserializeRowBatchTimer: 6532446385
- FirstBatchArrivalWaitTime: 564915329
- InactiveTotalTime: 526383447774
- PeakMemoryUsage: 0
- RowsReturned: 13661261
- RowsReturnedRate: 25840
- SendersBlockedTimer: 1666265796
- SendersBlockedTotalTimer(*): 21450815288
- TotalTime: 528673587721
Instance 4242a648b2529ccb:b503ec709811c7c7 (host=hdws81.houzz.com:22000)
- AverageThreadTokens: 1.143222506393862
- InactiveTotalTime: 0
- PeakMemoryUsage: 299374592
- PerHostPeakMemUsage: 663143536
- PrepareTime: 197219
- RowsProduced: 13403441
- TotalCpuTime: 136789510476
- TotalNetworkReceiveTime: 605989871493
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 651633476652
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 321343641
- CompressTimer: 0
- EncodeTimer: 13683785711
- FilesCreated: 1
- FinalizePartitionFileTimer: 77009334
- HdfsWriteTimer: 21305906487
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13403441
- TmpFileCreateTimer: 28643290
- TotalTime: 35303455994
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 2547569220
- BuildRows: 3126012
- BuildRowsPartitioned: 3126012
- BuildTime: 331477024
- GetNewBlockTime: 1555852
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 10634766336
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13403449
- ProbeRowsPartitioned: 0
- ProbeTime: 6825268745
- RowsReturned: 13403441
- RowsReturnedRate: 21749
- SpilledPartitions: 0
- TotalTime: 616272600178
- UnpinTime: 2598
EXCHANGE_NODE (id=4)
- BytesReceived: 39338192
- ConvertRowBatchTime: 165266611
- DeserializeRowBatchTimer: 417623475
- FirstBatchArrivalWaitTime: 289781326
- InactiveTotalTime: 84249196626
- PeakMemoryUsage: 0
- RowsReturned: 3126012
- RowsReturnedRate: 36974
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 84544226015
EXCHANGE_NODE (id=3)
- BytesReceived: 474550956
- ConvertRowBatchTime: 455124861
- DeserializeRowBatchTimer: 5976866649
- FirstBatchArrivalWaitTime: 1119752036
- InactiveTotalTime: 520584802625
- PeakMemoryUsage: 0
- RowsReturned: 13403449
- RowsReturnedRate: 25669
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 522154743841
Instance 4242a648b2529ccb:b503ec709811c7bc (host=hdws69.houzz.com:22000)
- AverageThreadTokens: 1.1395730706075533
- InactiveTotalTime: 0
- PeakMemoryUsage: 300780032
- PerHostPeakMemUsage: 710555928
- PrepareTime: 173249
- RowsProduced: 13655914
- TotalCpuTime: 133584829463
- TotalNetworkReceiveTime: 609512103385
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 651283541973
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 327592037
- CompressTimer: 0
- EncodeTimer: 14558362364
- FilesCreated: 1
- FinalizePartitionFileTimer: 85959997
- HdfsWriteTimer: 17602040382
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13655914
- TmpFileCreateTimer: 32236057
- TotalTime: 32470791286
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 554846464
- BuildRows: 3126410
- BuildRowsPartitioned: 3126410
- BuildTime: 398598200
- GetNewBlockTime: 1095136
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 9106044534
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13655928
- ProbeRowsPartitioned: 0
- ProbeTime: 7121306174
- RowsReturned: 13655914
- RowsReturnedRate: 22069
- SpilledPartitions: 0
- TotalTime: 618756453948
- UnpinTime: 761
EXCHANGE_NODE (id=4)
- BytesReceived: 39336241
- ConvertRowBatchTime: 122274853
- DeserializeRowBatchTimer: 261493455
- FirstBatchArrivalWaitTime: 289027611
- InactiveTotalTime: 86342459919
- PeakMemoryUsage: 0
- RowsReturned: 3126410
- RowsReturnedRate: 36128
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86535989105
EXCHANGE_NODE (id=3)
- BytesReceived: 483655804
- ConvertRowBatchTime: 464409100
- DeserializeRowBatchTimer: 9067388775
- FirstBatchArrivalWaitTime: 563908532
- InactiveTotalTime: 521027579323
- PeakMemoryUsage: 0
- RowsReturned: 13655928
- RowsReturnedRate: 26078
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 523645378511
Instance 4242a648b2529ccb:b503ec709811c7bf (host=hdws80.houzz.com:22000)
- AverageThreadTokens: 1.1411764705882352
- InactiveTotalTime: 0
- PeakMemoryUsage: 300623872
- PerHostPeakMemUsage: 703116096
- PrepareTime: 214077
- RowsProduced: 13229303
- TotalCpuTime: 129493611999
- TotalNetworkReceiveTime: 614991972388
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 651122145632
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 317016414
- CompressTimer: 0
- EncodeTimer: 15781716024
- FilesCreated: 1
- FinalizePartitionFileTimer: 81460259
- HdfsWriteTimer: 4522191870
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 13229303
- TmpFileCreateTimer: 1120484860
- TotalTime: 25214252294
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 862170169
- BuildRows: 3128839
- BuildRowsPartitioned: 3128839
- BuildTime: 917061378
- GetNewBlockTime: 1393361
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 9960321300
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 13229309
- ProbeRowsPartitioned: 0
- ProbeTime: 6865201447
- RowsReturned: 13229303
- RowsReturnedRate: 21161
- SpilledPartitions: 0
- TotalTime: 625147500610
- UnpinTime: 6085
EXCHANGE_NODE (id=4)
- BytesReceived: 39363310
- ConvertRowBatchTime: 151908579
- DeserializeRowBatchTimer: 516799094
- FirstBatchArrivalWaitTime: 290232603
- InactiveTotalTime: 85905705159
- PeakMemoryUsage: 0
- RowsReturned: 3128839
- RowsReturnedRate: 36285
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 86228228335
EXCHANGE_NODE (id=3)
- BytesReceived: 469109850
- ConvertRowBatchTime: 459047731
- DeserializeRowBatchTimer: 10059044790
- FirstBatchArrivalWaitTime: 564742366
- InactiveTotalTime: 526656117208
- PeakMemoryUsage: 0
- RowsReturned: 13229309
- RowsReturnedRate: 24986
- SendersBlockedTimer: 838894875
- SendersBlockedTotalTimer(*): 6749738306
- TotalTime: 529468103424
Instance 4242a648b2529ccb:b503ec709811c7c4 (host=hdws72.houzz.com:22000)
- AverageThreadTokens: 1.1352459016393444
- InactiveTotalTime: 0
- PeakMemoryUsage: 299936768
- PerHostPeakMemUsage: 626039368
- PrepareTime: 2258652
- RowsProduced: 14090262
- TotalCpuTime: 121197188051
- TotalNetworkReceiveTime: 621050419644
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 650486928148
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 338559233
- CompressTimer: 0
- EncodeTimer: 14630673702
- FilesCreated: 1
- FinalizePartitionFileTimer: 73820461
- HdfsWriteTimer: 5488007299
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 14090262
- TmpFileCreateTimer: 61075958
- TotalTime: 20449321380
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 1304616349
- BuildRows: 3126035
- BuildRowsPartitioned: 3126035
- BuildTime: 325503934
- GetNewBlockTime: 1359189
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 8337496259
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 14090270
- ProbeRowsPartitioned: 0
- ProbeTime: 7100475185
- RowsReturned: 14090262
- RowsReturnedRate: 22365
- SpilledPartitions: 0
- TotalTime: 629987651914
- UnpinTime: 3190
EXCHANGE_NODE (id=4)
- BytesReceived: 39325205
- ConvertRowBatchTime: 137998273
- DeserializeRowBatchTimer: 1326605396
- FirstBatchArrivalWaitTime: 19707521
- InactiveTotalTime: 85244364825
- PeakMemoryUsage: 0
- RowsReturned: 3126035
- RowsReturnedRate: 36553
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 85519046402
EXCHANGE_NODE (id=3)
- BytesReceived: 498360063
- ConvertRowBatchTime: 445461502
- DeserializeRowBatchTimer: 6436928525
- FirstBatchArrivalWaitTime: 732688356
- InactiveTotalTime: 532780998160
- PeakMemoryUsage: 0
- RowsReturned: 14090270
- RowsReturnedRate: 26277
- SendersBlockedTimer: 931041327
- SendersBlockedTotalTimer(*): 4202351274
- TotalTime: 536210440812
Instance 4242a648b2529ccb:b503ec709811c7c8 (host=hdws70.houzz.com:22000)
- AverageThreadTokens: 1.1346938775510205
- InactiveTotalTime: 0
- PeakMemoryUsage: 300592640
- PerHostPeakMemUsage: 697217072
- PrepareTime: 189515
- RowsProduced: 14006951
- TotalCpuTime: 118978713916
- TotalNetworkReceiveTime: 623805971775
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 650212821305
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 80
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 85899345920
- PeakMemoryUsage: 268435456
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
HdfsTableSink
- BytesWritten: 336431434
- CompressTimer: 0
- EncodeTimer: 12638363134
- FilesCreated: 1
- FinalizePartitionFileTimer: 75568152
- HdfsWriteTimer: 5307069380
- InactiveTotalTime: 0
- PartitionsCreated: 1
- PeakMemoryUsage: 51200
- RowsInserted: 14006951
- TmpFileCreateTimer: 48861737
- TotalTime: 18255732604
HASH_JOIN_NODE (id=2)
ExecOption: Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 1269253130
- BuildRows: 3126325
- BuildRowsPartitioned: 3126325
- BuildTime: 327641466
- GetNewBlockTime: 1529277
- HashBuckets: 8388608
- InactiveTotalTime: 0
- LargestPartitionPercent: 6
- LocalTime: 7965846662
- MaxPartitionLevel: 0
- NumRepartitions: 0
- PartitionsCreated: 16
- PeakMemoryUsage: 279994368
- PinTime: 0
- ProbeRows: 14006953
- ProbeRowsPartitioned: 0
- ProbeTime: 6156435845
- RowsReturned: 14006951
- RowsReturnedRate: 22165
- SpilledPartitions: 0
- TotalTime: 631911663957
- UnpinTime: 1655
EXCHANGE_NODE (id=4)
- BytesReceived: 39331927
- ConvertRowBatchTime: 125304383
- DeserializeRowBatchTimer: 266347589
- FirstBatchArrivalWaitTime: 289479173
- InactiveTotalTime: 85626169019
- PeakMemoryUsage: 0
- RowsReturned: 3126325
- RowsReturnedRate: 36424
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 85829869088
EXCHANGE_NODE (id=3)
- BytesReceived: 495571221
- ConvertRowBatchTime: 462094441
- DeserializeRowBatchTimer: 6303369466
- FirstBatchArrivalWaitTime: 564163785
- InactiveTotalTime: 535906547093
- PeakMemoryUsage: 0
- RowsReturned: 14006953
- RowsReturnedRate: 26003
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 538646422542
Fragment F01
Instance 4242a648b2529ccb:b503ec709811c7d4 (host=hdws75.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 12:1/63.36 MB 10:1/237.88 MB 11:1/234.64 MB
- AverageThreadTokens: 3.207547169811321
- InactiveTotalTime: 0
- PeakMemoryUsage: 53608672
- PerHostPeakMemUsage: 554678312
- PrepareTime: 159885
- RowsProduced: 6941731
- TotalCpuTime: 195034417436
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 83348791657
- TotalStorageWaitTime: 140759479
- TotalTime: 86752689079
DataStreamSender (dst_id=4)
- BytesSent: 88137736
- InactiveTotalTime: 0
- NetworkThroughput(*): 643644
- OverallThroughput: 1021484
- PeakMemoryUsage: 552960
- SerializeBatchTime: 1269445876
- ThriftTransmitTime(*): 136935448068
- TotalTime: 86284000551
- UncompressedRowBatchSize: 138861768
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 12:1/63.36 MB 10:1/237.88 MB 11:1/234.64 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:9
ExecOption: Codegen enabled: 0 out of 3
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 2.207547169811321
- BytesRead: 39103789
- BytesReadDataNodeCache: 0
- BytesReadLocal: 39103789
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 39103789
- DecompressionTime: 253587649
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 9
- NumDisksAccessed: 3
- NumRowGroups: 3
- NumScannerThreadsStarted: 3
- PeakMemoryUsage: 53129440
- PerReadThreadRawHdfsThroughput: 279045327
- RemoteScanRanges: 0
- RowsRead: 6941731
- RowsReturned: 6941731
- RowsReturnedRate: 15625180
- ScanRangesComplete: 3
- ScannerThreadsInvoluntaryContextSwitches: 2027
- ScannerThreadsTotalWallClockTime: 191685420651
- MaterializeTupleTime(*): 190662068483
- ScannerThreadsSysTime: 268016000
- ScannerThreadsUserTime: 1348083000
- ScannerThreadsVoluntaryContextSwitches: 7129
- TotalRawHdfsReadTime(*): 140134183
- TotalReadThroughput: 491871
- TotalTime: 444265664
Instance 4242a648b2529ccb:b503ec709811c7cf (host=hdws81.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 10:1/233.04 MB 11:1/237.60 MB
- AverageThreadTokens: 2.8472222222222223
- InactiveTotalTime: 0
- PeakMemoryUsage: 51578568
- PerHostPeakMemUsage: 483138320
- PrepareTime: 182387
- RowsProduced: 6112966
- TotalCpuTime: 144320037890
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 69671292209
- TotalStorageWaitTime: 103487908
- TotalTime: 71969415505
DataStreamSender (dst_id=4)
- BytesSent: 77363842
- InactiveTotalTime: 0
- NetworkThroughput(*): 635623
- OverallThroughput: 1089801
- PeakMemoryUsage: 552960
- SerializeBatchTime: 1706049783
- ThriftTransmitTime(*): 121713322950
- TotalTime: 70988903348
- UncompressedRowBatchSize: 122283220
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 10:1/233.04 MB 11:1/237.60 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:6
ExecOption: Codegen enabled: 0 out of 2
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 1.9275362318840579
- BytesRead: 34516947
- BytesReadDataNodeCache: 0
- BytesReadLocal: 34516947
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 34516947
- DecompressionTime: 112495646
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 6
- NumDisksAccessed: 2
- NumRowGroups: 2
- NumScannerThreadsStarted: 2
- PeakMemoryUsage: 51176128
- PerReadThreadRawHdfsThroughput: 320440603
- RemoteScanRanges: 0
- RowsRead: 6112966
- RowsReturned: 6112966
- RowsReturnedRate: 6366023
- ScanRangesComplete: 2
- ScannerThreadsInvoluntaryContextSwitches: 2228
- ScannerThreadsTotalWallClockTime: 138981334805
- MaterializeTupleTime(*): 138737844713
- ScannerThreadsSysTime: 268016000
- ScannerThreadsUserTime: 1220076000
- ScannerThreadsVoluntaryContextSwitches: 5690
- TotalRawHdfsReadTime(*): 107717145
- TotalReadThroughput: 500245
- TotalTime: 960248752
Instance 4242a648b2529ccb:b503ec709811c7d0 (host=hdws80.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/59.00 MB 1:1/214.78 MB
- AverageThreadTokens: 2.3294117647058825
- InactiveTotalTime: 0
- PeakMemoryUsage: 34331400
- PerHostPeakMemUsage: 477430336
- PrepareTime: 301635
- RowsProduced: 3554370
- TotalCpuTime: 62395412162
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 43534044042
- TotalStorageWaitTime: 120740484
- TotalTime: 45459841543
DataStreamSender (dst_id=4)
- BytesSent: 44443046
- InactiveTotalTime: 0
- NetworkThroughput(*): 630631
- OverallThroughput: 982182
- PeakMemoryUsage: 552960
- SerializeBatchTime: 728253125
- ThriftTransmitTime(*): 70473834397
- TotalTime: 45249279641
- UncompressedRowBatchSize: 71101316
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/59.00 MB 1:1/214.78 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:6
ExecOption: Codegen enabled: 0 out of 2
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 1.3294117647058823
- BytesRead: 20127323
- BytesReadDataNodeCache: 0
- BytesReadLocal: 20127323
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 20127323
- DecompressionTime: 67809189
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 6
- NumDisksAccessed: 2
- NumRowGroups: 2
- NumScannerThreadsStarted: 2
- PeakMemoryUsage: 33847736
- PerReadThreadRawHdfsThroughput: 229242768
- RemoteScanRanges: 0
- RowsRead: 3554370
- RowsReturned: 3554370
- RowsReturnedRate: 18285663
- ScanRangesComplete: 2
- ScannerThreadsInvoluntaryContextSwitches: 1735
- ScannerThreadsTotalWallClockTime: 60519030006
- MaterializeTupleTime(*): 60310648219
- ScannerThreadsSysTime: 148008000
- ScannerThreadsUserTime: 684042000
- ScannerThreadsVoluntaryContextSwitches: 2405
- TotalRawHdfsReadTime(*): 87799162
- TotalReadThroughput: 473572
- TotalTime: 194380148
Instance 4242a648b2529ccb:b503ec709811c7d7 (host=hdws74.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:1/40.17 MB 3:1/213.26 MB
- AverageThreadTokens: 2.1728395061728394
- InactiveTotalTime: 0
- PeakMemoryUsage: 34274472
- PerHostPeakMemUsage: 524888760
- PrepareTime: 121499
- RowsProduced: 3481586
- TotalCpuTime: 56492709904
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 42251363942
- TotalStorageWaitTime: 47349552
- TotalTime: 44908556941
DataStreamSender (dst_id=4)
- BytesSent: 44768211
- InactiveTotalTime: 0
- NetworkThroughput(*): 654557
- OverallThroughput: 999820
- PeakMemoryUsage: 552960
- SerializeBatchTime: 700753220
- ThriftTransmitTime(*): 68394611419
- TotalTime: 44776252275
- UncompressedRowBatchSize: 69645348
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:1/40.17 MB 3:1/213.26 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:6
ExecOption: Codegen enabled: 0 out of 2
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 1.1875
- BytesRead: 19853009
- BytesReadDataNodeCache: 0
- BytesReadLocal: 19853009
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 19853009
- DecompressionTime: 56776762
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 6
- NumDisksAccessed: 2
- NumRowGroups: 2
- NumScannerThreadsStarted: 2
- PeakMemoryUsage: 33830696
- PerReadThreadRawHdfsThroughput: 430079070
- RemoteScanRanges: 0
- RowsRead: 3481586
- RowsReturned: 3481586
- RowsReturnedRate: 29208507
- ScanRangesComplete: 2
- ScannerThreadsInvoluntaryContextSwitches: 1685
- ScannerThreadsTotalWallClockTime: 53324938858
- MaterializeTupleTime(*): 52265884847
- ScannerThreadsSysTime: 156009000
- ScannerThreadsUserTime: 644040000
- ScannerThreadsVoluntaryContextSwitches: 1953
- TotalRawHdfsReadTime(*): 46161300
- TotalReadThroughput: 496325
- TotalTime: 119197669
Instance 4242a648b2529ccb:b503ec709811c7ca (host=hdws84.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/40.73 MB 17:1/239.38 MB
- AverageThreadTokens: 2.210526315789474
- InactiveTotalTime: 0
- PeakMemoryUsage: 34292432
- PerHostPeakMemUsage: 487979648
- PrepareTime: 38284013
- RowsProduced: 3627997
- TotalCpuTime: 54603911619
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 41865570454
- TotalStorageWaitTime: 127818874
- TotalTime: 43355035433
DataStreamSender (dst_id=4)
- BytesSent: 45606133
- InactiveTotalTime: 0
- NetworkThroughput(*): 689201
- OverallThroughput: 1057165
- PeakMemoryUsage: 552960
- SerializeBatchTime: 678090277
- ThriftTransmitTime(*): 66172414551
- TotalTime: 43140030897
- UncompressedRowBatchSize: 72574148
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/40.73 MB 17:1/239.38 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:6
ExecOption: Codegen enabled: 0 out of 2
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 1.2105263157894737
- BytesRead: 20668060
- BytesReadDataNodeCache: 0
- BytesReadLocal: 20668060
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 20668060
- DecompressionTime: 76865913
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 6
- NumDisksAccessed: 2
- NumRowGroups: 2
- NumScannerThreadsStarted: 2
- PeakMemoryUsage: 33867848
- PerReadThreadRawHdfsThroughput: 275082978
- RemoteScanRanges: 0
- RowsRead: 3627997
- RowsReturned: 3627997
- RowsReturnedRate: 15699504
- ScanRangesComplete: 2
- ScannerThreadsInvoluntaryContextSwitches: 1502
- ScannerThreadsTotalWallClockTime: 52885466336
- MaterializeTupleTime(*): 52657448708
- ScannerThreadsSysTime: 132007000
- ScannerThreadsUserTime: 716044000
- ScannerThreadsVoluntaryContextSwitches: 1976
- TotalRawHdfsReadTime(*): 75133911
- TotalReadThroughput: 543896
- TotalTime: 231089910
Instance 4242a648b2529ccb:b503ec709811c7d5 (host=hdws72.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/42.31 MB 17:1/239.83 MB
- AverageThreadTokens: 2.08974358974359
- InactiveTotalTime: 0
- PeakMemoryUsage: 38559064
- PerHostPeakMemUsage: 479775664
- PrepareTime: 207246
- RowsProduced: 3760074
- TotalCpuTime: 49365220279
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 37569105333
- TotalStorageWaitTime: 96333474
- TotalTime: 41483454025
DataStreamSender (dst_id=4)
- BytesSent: 47068519
- InactiveTotalTime: 0
- NetworkThroughput(*): 702828
- OverallThroughput: 1150372
- PeakMemoryUsage: 552960
- SerializeBatchTime: 649749680
- ThriftTransmitTime(*): 66970154168
- TotalTime: 40915904593
- UncompressedRowBatchSize: 75216192
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/42.31 MB 17:1/239.83 MB
Hdfs Read Thread Concurrency Bucket: 0:98.72% 1:1.282% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:6
ExecOption: Codegen enabled: 0 out of 2
- AverageHdfsReadThreadConcurrency: 0.01282051282051282
- AverageScannerThreadConcurrency: 1.0897435897435896
- BytesRead: 21424117
- BytesReadDataNodeCache: 0
- BytesReadLocal: 21424117
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 21424117
- DecompressionTime: 67417177
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 6
- NumDisksAccessed: 2
- NumRowGroups: 2
- NumScannerThreadsStarted: 2
- PeakMemoryUsage: 38030680
- PerReadThreadRawHdfsThroughput: 29573682
- RemoteScanRanges: 0
- RowsRead: 3760074
- RowsReturned: 3760074
- RowsReturnedRate: 6953137
- ScanRangesComplete: 2
- ScannerThreadsInvoluntaryContextSwitches: 1251
- ScannerThreadsTotalWallClockTime: 45474366782
- MaterializeTupleTime(*): 45291292239
- ScannerThreadsSysTime: 104006000
- ScannerThreadsUserTime: 700043000
- ScannerThreadsVoluntaryContextSwitches: 1940
- TotalRawHdfsReadTime(*): 724431843
- TotalReadThroughput: 549336
- TotalTime: 540773747
Instance 4242a648b2529ccb:b503ec709811c7cc (host=hdws71.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 8:1/33.71 MB 12:1/235.57 MB
- AverageThreadTokens: 2.0625
- InactiveTotalTime: 0
- PeakMemoryUsage: 31383608
- PerHostPeakMemUsage: 483196680
- PrepareTime: 151879
- RowsProduced: 3415399
- TotalCpuTime: 48544291364
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 39883565793
- TotalStorageWaitTime: 54096594
- TotalTime: 40963500580
DataStreamSender (dst_id=4)
- BytesSent: 42657741
- InactiveTotalTime: 0
- NetworkThroughput(*): 650177
- OverallThroughput: 1044119
- PeakMemoryUsage: 552960
- SerializeBatchTime: 612719563
- ThriftTransmitTime(*): 65609385727
- TotalTime: 40855213631
- UncompressedRowBatchSize: 68321352
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 8:1/33.71 MB 12:1/235.57 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:6
ExecOption: Codegen enabled: 0 out of 2
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 1.0897435897435896
- BytesRead: 19261389
- BytesReadDataNodeCache: 0
- BytesReadLocal: 19261389
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 19261389
- DecompressionTime: 68998020
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 6
- NumDisksAccessed: 2
- NumRowGroups: 2
- NumScannerThreadsStarted: 2
- PeakMemoryUsage: 30896184
- PerReadThreadRawHdfsThroughput: 520135263
- RemoteScanRanges: 0
- RowsRead: 3415399
- RowsReturned: 3415399
- RowsReturnedRate: 35067420
- ScanRangesComplete: 2
- ScannerThreadsInvoluntaryContextSwitches: 1206
- ScannerThreadsTotalWallClockTime: 45693314035
- MaterializeTupleTime(*): 45335473938
- ScannerThreadsSysTime: 108006000
- ScannerThreadsUserTime: 620038000
- ScannerThreadsVoluntaryContextSwitches: 1815
- TotalRawHdfsReadTime(*): 37031500
- TotalReadThroughput: 493881
- TotalTime: 97395216
Instance 4242a648b2529ccb:b503ec709811c7d3 (host=hdws73.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/238.98 MB
- AverageThreadTokens: 1.9107142857142858
- InactiveTotalTime: 0
- PeakMemoryUsage: 28118392
- PerHostPeakMemUsage: 472500960
- PrepareTime: 151408
- RowsProduced: 2919456
- TotalCpuTime: 30187724987
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 30481175151
- TotalStorageWaitTime: 34317429
- TotalTime: 31531176776
DataStreamSender (dst_id=4)
- BytesSent: 36508408
- InactiveTotalTime: 0
- NetworkThroughput(*): 723562
- OverallThroughput: 1161456
- PeakMemoryUsage: 552960
- SerializeBatchTime: 507361314
- ThriftTransmitTime(*): 50456485824
- TotalTime: 31433288907
- UncompressedRowBatchSize: 58400560
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/238.98 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.9107142857142857
- BytesRead: 16400310
- BytesReadDataNodeCache: 0
- BytesReadLocal: 16400310
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 16400310
- DecompressionTime: 53858760
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 27578080
- PerReadThreadRawHdfsThroughput: 638232965
- RemoteScanRanges: 0
- RowsRead: 2919456
- RowsReturned: 2919456
- RowsReturnedRate: 32737694
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 719
- ScannerThreadsTotalWallClockTime: 28911405717
- MaterializeTupleTime(*): 28815664735
- ScannerThreadsSysTime: 64004000
- ScannerThreadsUserTime: 560035000
- ScannerThreadsVoluntaryContextSwitches: 1187
- TotalRawHdfsReadTime(*): 25696432
- TotalReadThroughput: 585725
- TotalTime: 89177203
Instance 4242a648b2529ccb:b503ec709811c7d2 (host=hdws83.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 4:1/214.90 MB
- AverageThreadTokens: 1.8771929824561404
- InactiveTotalTime: 0
- PeakMemoryUsage: 28062392
- PerHostPeakMemUsage: 509444032
- PrepareTime: 114717
- RowsProduced: 2808058
- TotalCpuTime: 28492216698
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 29385820491
- TotalStorageWaitTime: 27525874
- TotalTime: 30458214738
DataStreamSender (dst_id=4)
- BytesSent: 36003683
- InactiveTotalTime: 0
- NetworkThroughput(*): 778942
- OverallThroughput: 1185652
- PeakMemoryUsage: 552960
- SerializeBatchTime: 460674167
- ThriftTransmitTime(*): 46221238138
- TotalTime: 30366134143
- UncompressedRowBatchSize: 56172160
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 4:1/214.90 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.8771929824561403
- BytesRead: 15889756
- BytesReadDataNodeCache: 0
- BytesReadLocal: 15889756
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 15889756
- DecompressionTime: 44389696
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 27587312
- PerReadThreadRawHdfsThroughput: 660620434
- RemoteScanRanges: 0
- RowsRead: 2808058
- RowsReturned: 2808058
- RowsReturnedRate: 33306573
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 429
- ScannerThreadsTotalWallClockTime: 26861910813
- MaterializeTupleTime(*): 26781479894
- ScannerThreadsSysTime: 56003000
- ScannerThreadsUserTime: 516032000
- ScannerThreadsVoluntaryContextSwitches: 1162
- TotalRawHdfsReadTime(*): 24052777
- TotalReadThroughput: 557535
- TotalTime: 84309423
Instance 4242a648b2529ccb:b503ec709811c7d6 (host=hdws67.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 10:1/181.32 MB
- AverageThreadTokens: 1.8035714285714286
- InactiveTotalTime: 0
- PeakMemoryUsage: 24978960
- PerHostPeakMemUsage: 515270336
- PrepareTime: 136271
- RowsProduced: 2333879
- TotalCpuTime: 27348598399
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 28053915646
- TotalStorageWaitTime: 47655051
- TotalTime: 30246609995
DataStreamSender (dst_id=4)
- BytesSent: 28805344
- InactiveTotalTime: 0
- NetworkThroughput(*): 619446
- OverallThroughput: 956061
- PeakMemoryUsage: 552960
- SerializeBatchTime: 409853338
- ThriftTransmitTime(*): 46501747446
- TotalTime: 30129188487
- UncompressedRowBatchSize: 46686720
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 10:1/181.32 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.8035714285714286
- BytesRead: 13150763
- BytesReadDataNodeCache: 0
- BytesReadLocal: 13150763
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 13150763
- DecompressionTime: 38548403
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 24454520
- PerReadThreadRawHdfsThroughput: 396775395
- RemoteScanRanges: 0
- RowsRead: 2333879
- RowsReturned: 2333879
- RowsReturnedRate: 21826314
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 720
- ScannerThreadsTotalWallClockTime: 24425270334
- MaterializeTupleTime(*): 24329328079
- ScannerThreadsSysTime: 76004000
- ScannerThreadsUserTime: 408025000
- ScannerThreadsVoluntaryContextSwitches: 1026
- TotalRawHdfsReadTime(*): 33144099
- TotalReadThroughput: 469670
- TotalTime: 106929595
Instance 4242a648b2529ccb:b503ec709811c7cb (host=hdws69.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 9:1/236.48 MB
- AverageThreadTokens: 1.8070175438596492
- InactiveTotalTime: 0
- PeakMemoryUsage: 28086776
- PerHostPeakMemUsage: 530264856
- PrepareTime: 128846
- RowsProduced: 2735017
- TotalCpuTime: 25683010287
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 28898601043
- TotalStorageWaitTime: 28369358
- TotalTime: 29705703993
DataStreamSender (dst_id=4)
- BytesSent: 34352618
- InactiveTotalTime: 0
- NetworkThroughput(*): 684378
- OverallThroughput: 1159356
- PeakMemoryUsage: 552960
- SerializeBatchTime: 422838526
- ThriftTransmitTime(*): 50195378332
- TotalTime: 29630768680
- UncompressedRowBatchSize: 54711052
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 9:1/236.48 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.8214285714285714
- BytesRead: 15419551
- BytesReadDataNodeCache: 0
- BytesReadLocal: 15419551
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 15419551
- DecompressionTime: 36172148
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 27578496
- PerReadThreadRawHdfsThroughput: 705271559
- RemoteScanRanges: 0
- RowsRead: 2735017
- RowsReturned: 2735017
- RowsReturnedRate: 40774756
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 433
- ScannerThreadsTotalWallClockTime: 24370300346
- MaterializeTupleTime(*): 24298783776
- ScannerThreadsSysTime: 84005000
- ScannerThreadsUserTime: 460028000
- ScannerThreadsVoluntaryContextSwitches: 1160
- TotalRawHdfsReadTime(*): 21863282
- TotalReadThroughput: 550698
- TotalTime: 67076232
Instance 4242a648b2529ccb:b503ec709811c7ce (host=hdws70.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/167.98 MB
- AverageThreadTokens: 1.8695652173913044
- InactiveTotalTime: 0
- PeakMemoryUsage: 25966864
- PerHostPeakMemUsage: 594453728
- PrepareTime: 96585
- RowsProduced: 2355996
- TotalCpuTime: 22414828868
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 23522696387
- TotalStorageWaitTime: 31818638
- TotalTime: 24331607946
DataStreamSender (dst_id=4)
- BytesSent: 29133173
- InactiveTotalTime: 0
- NetworkThroughput(*): 729612
- OverallThroughput: 1201046
- PeakMemoryUsage: 552960
- SerializeBatchTime: 355488987
- ThriftTransmitTime(*): 39929672920
- TotalTime: 24256500288
- UncompressedRowBatchSize: 47129148
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/167.98 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.8695652173913043
- BytesRead: 13308560
- BytesReadDataNodeCache: 0
- BytesReadLocal: 13308560
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 13308560
- DecompressionTime: 29249598
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 25511672
- PerReadThreadRawHdfsThroughput: 556896967
- RemoteScanRanges: 0
- RowsRead: 2355996
- RowsReturned: 2355996
- RowsReturnedRate: 34584784
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 465
- ScannerThreadsTotalWallClockTime: 21559998682
- MaterializeTupleTime(*): 21493601924
- ScannerThreadsSysTime: 68004000
- ScannerThreadsUserTime: 372023000
- ScannerThreadsVoluntaryContextSwitches: 891
- TotalRawHdfsReadTime(*): 23897706
- TotalReadThroughput: 578633
- TotalTime: 68122328
Instance 4242a648b2529ccb:b503ec709811c7cd (host=hdws82.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 13:1/173.88 MB
- AverageThreadTokens: 1.9473684210526316
- InactiveTotalTime: 0
- PeakMemoryUsage: 24974208
- PerHostPeakMemUsage: 553655360
- PrepareTime: 137213
- RowsProduced: 2332357
- TotalCpuTime: 19733216499
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 19087656283
- TotalStorageWaitTime: 34889140
- TotalTime: 19034034585
DataStreamSender (dst_id=4)
- BytesSent: 28645775
- InactiveTotalTime: 0
- NetworkThroughput(*): 916074
- OverallThroughput: 1511350
- PeakMemoryUsage: 552960
- SerializeBatchTime: 325647875
- ThriftTransmitTime(*): 31270139062
- TotalTime: 18953758555
- UncompressedRowBatchSize: 46656284
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 13:1/173.88 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 1.0
- BytesRead: 13112240
- BytesReadDataNodeCache: 0
- BytesReadLocal: 13112240
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 13112240
- DecompressionTime: 32896330
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 24451352
- PerReadThreadRawHdfsThroughput: 547665603
- RemoteScanRanges: 0
- RowsRead: 2332357
- RowsReturned: 2332357
- RowsReturnedRate: 31617349
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 382
- ScannerThreadsTotalWallClockTime: 18930348012
- MaterializeTupleTime(*): 18856714780
- ScannerThreadsSysTime: 28001000
- ScannerThreadsUserTime: 428026000
- ScannerThreadsVoluntaryContextSwitches: 833
- TotalRawHdfsReadTime(*): 23942055
- TotalReadThroughput: 728457
- TotalTime: 73768264
Instance 4242a648b2529ccb:b503ec709811c7d1 (host=hdws78.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 15:1/31.70 MB
- AverageThreadTokens: 1.1818181818181819
- InactiveTotalTime: 0
- PeakMemoryUsage: 12053424
- PerHostPeakMemUsage: 272101992
- PrepareTime: 109777
- RowsProduced: 474458
- TotalCpuTime: 1045878534
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 5361170988
- TotalStorageWaitTime: 22513409
- TotalTime: 5526769528
DataStreamSender (dst_id=4)
- BytesSent: 5929823
- InactiveTotalTime: 0
- NetworkThroughput(*): 623753
- OverallThroughput: 1078891
- PeakMemoryUsage: 552960
- SerializeBatchTime: 67718350
- ThriftTransmitTime(*): 9506674880
- TotalTime: 5496215772
- UncompressedRowBatchSize: 9491036
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 15:1/31.70 MB
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.18181818181818182
- BytesRead: 2817494
- BytesReadDataNodeCache: 0
- BytesReadLocal: 2817494
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 2817494
- DecompressionTime: 7952780
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 11541424
- PerReadThreadRawHdfsThroughput: 247292546
- RemoteScanRanges: 0
- RowsRead: 474458
- RowsReturned: 474458
- RowsReturnedRate: 16917551
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 166
- ScannerThreadsTotalWallClockTime: 878279579
- MaterializeTupleTime(*): 846096913
- ScannerThreadsSysTime: 24001000
- ScannerThreadsUserTime: 64004000
- ScannerThreadsVoluntaryContextSwitches: 55
- TotalRawHdfsReadTime(*): 11393364
- TotalReadThroughput: 512271
- TotalTime: 28045312
Instance 4242a648b2529ccb:b503ec709811c7d8 (host=hdws86.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 14:1/4.06 MB
- AverageThreadTokens: 0.0
- InactiveTotalTime: 0
- PeakMemoryUsage: 1887944
- PerHostPeakMemUsage: 4042440
- PrepareTime: 122588
- RowsProduced: 53561
- TotalCpuTime: 44318254
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 30666280
- TotalStorageWaitTime: 6952589
- TotalTime: 24010785
DataStreamSender (dst_id=4)
- BytesSent: 686994
- InactiveTotalTime: 0
- NetworkThroughput(*): 11945141
- OverallThroughput: 55883843
- PeakMemoryUsage: 552960
- SerializeBatchTime: 6965330
- ThriftTransmitTime(*): 57512422
- TotalTime: 12293249
- UncompressedRowBatchSize: 1071460
HDFS_SCAN_NODE (id=1)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 14:1/4.06 MB
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.0
- BytesRead: 480982
- BytesReadDataNodeCache: 0
- BytesReadLocal: 480982
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 480982
- DecompressionTime: 515967
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 3
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 1560264
- PerReadThreadRawHdfsThroughput: 362214425
- RemoteScanRanges: 0
- RowsRead: 53561
- RowsReturned: 53561
- RowsReturnedRate: 5873963
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 2
- ScannerThreadsTotalWallClockTime: 16886317
- MaterializeTupleTime(*): 8267234
- ScannerThreadsSysTime: 0
- ScannerThreadsUserTime: 8000000
- ScannerThreadsVoluntaryContextSwitches: 11
- TotalRawHdfsReadTime(*): 1327893
- TotalReadThroughput: 0
- TotalTime: 9118374
Fragment F00
Instance 4242a648b2529ccb:b503ec709811c7e3 (host=hdws86.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 11:2/252.24 MB 19:1/128.00 MB 8:1/128.00 MB 4:1/128.00 MB 1:1/126.16 MB 0:3/382.15 MB 22:1/126.19 MB
- AverageThreadTokens: 3.7788539144471347
- InactiveTotalTime: 0
- PeakMemoryUsage: 375456912
- PerHostPeakMemUsage: 670865104
- PrepareTime: 221085
- RowsProduced: 25151977
- TotalCpuTime: 1849455646321
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 624640258925
- TotalStorageWaitTime: 2313302848
- TotalTime: 652992407752
DataStreamSender (dst_id=3)
- BytesSent: 891452013
- InactiveTotalTime: 0
- NetworkThroughput(*): 827684
- OverallThroughput: 1375153
- PeakMemoryUsage: 540672
- SerializeBatchTime: 14797537428
- ThriftTransmitTime(*): 1077043268304
- TotalTime: 648256237853
- UncompressedRowBatchSize: 1433859217
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 11:2/252.24 MB 19:1/128.00 MB 8:1/128.00 MB 4:1/128.00 MB 1:1/126.16 MB 0:3/382.15 MB 22:1/126.19 MB
Hdfs Read Thread Concurrency Bucket: 0:98.71% 1:1.049% 2:0.08071% 3:0% 4:0.08071% 5:0% 6:0% 7:0% 8:0.08071% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:15 PARQUET/SNAPPY:15
ExecOption: Codegen enabled: 0 out of 10
- AverageHdfsReadThreadConcurrency: 0.021791767554479417
- AverageScannerThreadConcurrency: 2.7788539144471347
- BytesRead: 736473782
- BytesReadDataNodeCache: 0
- BytesReadLocal: 268742444
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 268742444
- DecompressionTime: 11398111566
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 30
- NumDisksAccessed: 5
- NumRowGroups: 5
- NumScannerThreadsStarted: 5
- PeakMemoryUsage: 374890056
- PerReadThreadRawHdfsThroughput: 55948253
- RemoteScanRanges: 18
- RowsRead: 54232888
- RowsReturned: 25151977
- RowsReturnedRate: 5401949
- ScanRangesComplete: 10
- ScannerThreadsInvoluntaryContextSwitches: 41476
- ScannerThreadsTotalWallClockTime: 1821634283856
- MaterializeTupleTime(*): 1798700054073
- ScannerThreadsSysTime: 13248826000
- ScannerThreadsUserTime: 185423587000
- ScannerThreadsVoluntaryContextSwitches: 290759
- TotalRawHdfsReadTime(*): 13163481114
- TotalReadThroughput: 1188819
- TotalTime: 4656092264
Instance 4242a648b2529ccb:b503ec709811c7e7 (host=hdws82.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 14:1/128.00 MB 13:1/126.02 MB 12:1/126.23 MB 6:1/126.07 MB 1:3/380.37 MB 5:1/128.00 MB 0:3/361.78 MB 2:1/126.21 MB
- AverageThreadTokens: 4.497109826589595
- InactiveTotalTime: 0
- PeakMemoryUsage: 380853944
- PerHostPeakMemUsage: 664006024
- PrepareTime: 348247
- RowsProduced: 20315387
- TotalCpuTime: 1973180039752
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 532700003314
- TotalStorageWaitTime: 9149261257
- TotalTime: 557451117182
DataStreamSender (dst_id=3)
- BytesSent: 714523965
- InactiveTotalTime: 0
- NetworkThroughput(*): 742854
- OverallThroughput: 1292062
- PeakMemoryUsage: 606208
- SerializeBatchTime: 12166753009
- ThriftTransmitTime(*): 961862932605
- TotalTime: 553010224797
- UncompressedRowBatchSize: 1158135799
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 14:1/128.00 MB 13:1/126.02 MB 12:1/126.23 MB 6:1/126.07 MB 1:3/380.37 MB 5:1/128.00 MB 0:3/361.78 MB 2:1/126.21 MB
Hdfs Read Thread Concurrency Bucket: 0:96.82% 1:1.638% 2:1.349% 3:0.1927% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:24 PARQUET/SNAPPY:12
ExecOption: Codegen enabled: 0 out of 12
- AverageHdfsReadThreadConcurrency: 0.049132947976878616
- AverageScannerThreadConcurrency: 3.4971098265895955
- BytesRead: 634212238
- BytesReadDataNodeCache: 0
- BytesReadLocal: 268844846
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 268844846
- DecompressionTime: 6918325462
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 36
- NumDisksAccessed: 7
- NumRowGroups: 4
- NumScannerThreadsStarted: 5
- PeakMemoryUsage: 380419768
- PerReadThreadRawHdfsThroughput: 24440536
- RemoteScanRanges: 16
- RowsRead: 47376338
- RowsReturned: 20315387
- RowsReturnedRate: 4637311
- ScanRangesComplete: 12
- ScannerThreadsInvoluntaryContextSwitches: 30226
- ScannerThreadsTotalWallClockTime: 1954985595111
- MaterializeTupleTime(*): 1929670224934
- ScannerThreadsSysTime: 16061003000
- ScannerThreadsUserTime: 165766358000
- ScannerThreadsVoluntaryContextSwitches: 484590
- TotalRawHdfsReadTime(*): 25949194219
- TotalReadThroughput: 1221984
- TotalTime: 4380854312
Instance 4242a648b2529ccb:b503ec709811c7df (host=hdws73.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 20:1/126.06 MB 19:1/128.00 MB 17:1/126.14 MB 1:1/128.00 MB 13:1/128.00 MB 0:4/492.54 MB
- AverageThreadTokens: 4.2782874617737
- InactiveTotalTime: 0
- PeakMemoryUsage: 368019472
- PerHostPeakMemUsage: 647160336
- PrepareTime: 358598
- RowsProduced: 19806441
- TotalCpuTime: 1839875590374
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 525852577672
- TotalStorageWaitTime: 3945484376
- TotalTime: 548658391117
DataStreamSender (dst_id=3)
- BytesSent: 707131448
- InactiveTotalTime: 0
- NetworkThroughput(*): 808183
- OverallThroughput: 1297884
- PeakMemoryUsage: 557056
- SerializeBatchTime: 9791375986
- ThriftTransmitTime(*): 874964211138
- TotalTime: 544833939804
- UncompressedRowBatchSize: 1129121909
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 20:1/126.06 MB 19:1/128.00 MB 17:1/126.14 MB 1:1/128.00 MB 13:1/128.00 MB 0:4/492.54 MB
Hdfs Read Thread Concurrency Bucket: 0:97.15% 1:2.243% 2:0.4077% 3:0% 4:0.2039% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:9 PARQUET/SNAPPY:18
ExecOption: Codegen enabled: 0 out of 9
- AverageHdfsReadThreadConcurrency: 0.03873598369011213
- AverageScannerThreadConcurrency: 3.2782874617737003
- BytesRead: 823307469
- BytesReadDataNodeCache: 0
- BytesReadLocal: 268640042
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 268640042
- DecompressionTime: 14940297425
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 27
- NumDisksAccessed: 5
- NumRowGroups: 6
- NumScannerThreadsStarted: 6
- PeakMemoryUsage: 367593488
- PerReadThreadRawHdfsThroughput: 35594653
- RemoteScanRanges: 21
- RowsRead: 59437546
- RowsReturned: 19806441
- RowsReturnedRate: 5290165
- ScanRangesComplete: 9
- ScannerThreadsInvoluntaryContextSwitches: 54929
- ScannerThreadsTotalWallClockTime: 1816828943820
- MaterializeTupleTime(*): 1786102176840
- ScannerThreadsSysTime: 17297079000
- ScannerThreadsUserTime: 206976933000
- ScannerThreadsVoluntaryContextSwitches: 394168
- TotalRawHdfsReadTime(*): 23130087720
- TotalReadThroughput: 1678503
- TotalTime: 3744011950
Instance 4242a648b2529ccb:b503ec709811c7e5 (host=hdws81.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:2/254.11 MB 15:1/125.82 MB 11:1/128.00 MB 0:3/277.31 MB 14:1/128.00 MB 8:1/128.00 MB 13:1/56.14 MB 4:1/126.18 MB 3:1/126.10 MB
- AverageThreadTokens: 4.672858617131063
- InactiveTotalTime: 0
- PeakMemoryUsage: 383631984
- PerHostPeakMemUsage: 663143536
- PrepareTime: 557466
- RowsProduced: 18343648
- TotalCpuTime: 2002322401012
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 506679447246
- TotalStorageWaitTime: 17672318318
- TotalTime: 536433983695
DataStreamSender (dst_id=3)
- BytesSent: 658095464
- InactiveTotalTime: 0
- NetworkThroughput(*): 756860
- OverallThroughput: 1248685
- PeakMemoryUsage: 557056
- SerializeBatchTime: 9159368658
- ThriftTransmitTime(*): 869506959777
- TotalTime: 527030453742
- UncompressedRowBatchSize: 1045731272
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:2/254.11 MB 15:1/125.82 MB 11:1/128.00 MB 0:3/277.31 MB 14:1/128.00 MB 8:1/128.00 MB 13:1/56.14 MB 4:1/126.18 MB 3:1/126.10 MB
Hdfs Read Thread Concurrency Bucket: 0:94.94% 1:4.334% 2:0.4128% 3:0.2064% 4:0% 5:0.1032% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:21 PARQUET/SNAPPY:15
ExecOption: Codegen enabled: 0 out of 12
- AverageHdfsReadThreadConcurrency: 0.06191950464396285
- AverageScannerThreadConcurrency: 3.672858617131063
- BytesRead: 713004488
- BytesReadDataNodeCache: 0
- BytesReadLocal: 383150890
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 383150890
- DecompressionTime: 16043930129
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 36
- NumDisksAccessed: 8
- NumRowGroups: 5
- NumScannerThreadsStarted: 5
- PeakMemoryUsage: 383124072
- PerReadThreadRawHdfsThroughput: 21119664
- RemoteScanRanges: 14
- RowsRead: 52740168
- RowsReturned: 18343648
- RowsReturnedRate: 1967353
- ScanRangesComplete: 12
- ScannerThreadsInvoluntaryContextSwitches: 51936
- ScannerThreadsTotalWallClockTime: 1985157972200
- MaterializeTupleTime(*): 1926389464832
- ScannerThreadsSysTime: 22917429000
- ScannerThreadsUserTime: 200624536000
- ScannerThreadsVoluntaryContextSwitches: 730101
- TotalRawHdfsReadTime(*): 33760218251
- TotalReadThroughput: 1471605
- TotalTime: 9324024216
Instance 4242a648b2529ccb:b503ec709811c7dd (host=hdws80.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 17:1/128.00 MB 14:1/126.12 MB 13:1/57.54 MB 12:1/128.00 MB 1:2/256.00 MB 10:1/128.00 MB 0:4/508.27 MB
- AverageThreadTokens: 4.689618644067797
- InactiveTotalTime: 0
- PeakMemoryUsage: 424033600
- PerHostPeakMemUsage: 703116096
- PrepareTime: 589201
- RowsProduced: 15383570
- TotalCpuTime: 1941408347571
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 487940375310
- TotalStorageWaitTime: 6752231704
- TotalTime: 516220448363
DataStreamSender (dst_id=3)
- BytesSent: 551180431
- InactiveTotalTime: 0
- NetworkThroughput(*): 689982
- OverallThroughput: 1085350
- PeakMemoryUsage: 548864
- SerializeBatchTime: 10506955626
- ThriftTransmitTime(*): 798831942112
- TotalTime: 507836120793
- UncompressedRowBatchSize: 876983702
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 17:1/128.00 MB 14:1/126.12 MB 13:1/57.54 MB 12:1/128.00 MB 1:2/256.00 MB 10:1/128.00 MB 0:4/508.27 MB
Hdfs Read Thread Concurrency Bucket: 0:95.33% 1:3.287% 2:0.9544% 3:0.4242% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:9 PARQUET/SNAPPY:24
ExecOption: Codegen enabled: 0 out of 11
- AverageHdfsReadThreadConcurrency: 0.06468716861081654
- AverageScannerThreadConcurrency: 3.693531283138918
- BytesRead: 1131250437
- BytesReadDataNodeCache: 0
- BytesReadLocal: 437486853
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 437486853
- DecompressionTime: 22902119980
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 33
- NumDisksAccessed: 6
- NumRowGroups: 8
- NumScannerThreadsStarted: 8
- PeakMemoryUsage: 423492928
- PerReadThreadRawHdfsThroughput: 33945496
- RemoteScanRanges: 24
- RowsRead: 79674324
- RowsReturned: 15383570
- RowsReturnedRate: 1854857
- ScanRangesComplete: 11
- ScannerThreadsInvoluntaryContextSwitches: 65571
- ScannerThreadsTotalWallClockTime: 1916054853882
- MaterializeTupleTime(*): 1836741102434
- ScannerThreadsSysTime: 88465526000
- ScannerThreadsUserTime: 275249199000
- ScannerThreadsVoluntaryContextSwitches: 511189
- TotalRawHdfsReadTime(*): 33325493449
- TotalReadThroughput: 2399207
- TotalTime: 8293666824
Instance 4242a648b2529ccb:b503ec709811c7e1 (host=hdws83.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 19:1/128.00 MB 21:1/23.79 MB 20:1/128.00 MB 18:1/126.11 MB 22:1/126.10 MB 0:6/700.66 MB
- AverageThreadTokens: 4.617352614015573
- InactiveTotalTime: 0
- PeakMemoryUsage: 466041504
- PerHostPeakMemUsage: 745151136
- PrepareTime: 526841
- RowsProduced: 17063851
- TotalCpuTime: 1859517021918
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 491697836986
- TotalStorageWaitTime: 18781367070
- TotalTime: 512999254046
DataStreamSender (dst_id=3)
- BytesSent: 616473598
- InactiveTotalTime: 0
- NetworkThroughput(*): 793669
- OverallThroughput: 1208715
- PeakMemoryUsage: 540672
- SerializeBatchTime: 8910132486
- ThriftTransmitTime(*): 776738064325
- TotalTime: 510023534913
- UncompressedRowBatchSize: 972772847
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 19:1/128.00 MB 21:1/23.79 MB 20:1/128.00 MB 18:1/126.11 MB 22:1/126.10 MB 0:6/700.66 MB
Hdfs Read Thread Concurrency Bucket: 0:82.76% 1:16.46% 2:0.5562% 3:0.2225% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:12 PARQUET/SNAPPY:21
ExecOption: Codegen enabled: 0 out of 11
- AverageHdfsReadThreadConcurrency: 0.18242491657397109
- AverageScannerThreadConcurrency: 3.617352614015573
- BytesRead: 963793743
- BytesReadDataNodeCache: 0
- BytesReadLocal: 151313189
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 151313189
- DecompressionTime: 15427759050
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 33
- NumDisksAccessed: 6
- NumRowGroups: 7
- NumScannerThreadsStarted: 8
- PeakMemoryUsage: 465443488
- PerReadThreadRawHdfsThroughput: 9907499
- RemoteScanRanges: 23
- RowsRead: 68838002
- RowsReturned: 17063851
- RowsReturnedRate: 5867264
- ScanRangesComplete: 11
- ScannerThreadsInvoluntaryContextSwitches: 46871
- ScannerThreadsTotalWallClockTime: 1855446758801
- MaterializeTupleTime(*): 1799607053765
- ScannerThreadsSysTime: 22965433000
- ScannerThreadsUserTime: 236582783000
- ScannerThreadsVoluntaryContextSwitches: 508030
- TotalRawHdfsReadTime(*): 97279210951
- TotalReadThroughput: 2144131
- TotalTime: 2908314495
Instance 4242a648b2529ccb:b503ec709811c7db (host=hdws71.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 3:1/14.45 MB 0:5/527.76 MB 1:2/253.98 MB 19:1/122.17 MB
- AverageThreadTokens: 3.648471615720524
- InactiveTotalTime: 0
- PeakMemoryUsage: 353309952
- PerHostPeakMemUsage: 632540416
- PrepareTime: 615328
- RowsProduced: 15628858
- TotalCpuTime: 1327558974289
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 476224221849
- TotalStorageWaitTime: 5626193356
- TotalTime: 493591586013
DataStreamSender (dst_id=3)
- BytesSent: 550483394
- InactiveTotalTime: 0
- NetworkThroughput(*): 684550
- OverallThroughput: 1125589
- PeakMemoryUsage: 557056
- SerializeBatchTime: 8020566503
- ThriftTransmitTime(*): 804152748643
- TotalTime: 489062199569
- UncompressedRowBatchSize: 890967034
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 3:1/14.45 MB 0:5/527.76 MB 1:2/253.98 MB 19:1/122.17 MB
Hdfs Read Thread Concurrency Bucket: 0:97.71% 1:1.856% 2:0.2183% 3:0.1092% 4:0% 5:0% 6:0.1092% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:12 PARQUET/SNAPPY:15
ExecOption: Codegen enabled: 0 out of 9
- AverageHdfsReadThreadConcurrency: 0.03275109170305677
- AverageScannerThreadConcurrency: 2.648471615720524
- BytesRead: 638871446
- BytesReadDataNodeCache: 0
- BytesReadLocal: 277643414
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 277643414
- DecompressionTime: 7655215148
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 27
- NumDisksAccessed: 5
- NumRowGroups: 5
- NumScannerThreadsStarted: 5
- PeakMemoryUsage: 352744696
- PerReadThreadRawHdfsThroughput: 32089204
- RemoteScanRanges: 15
- RowsRead: 46513814
- RowsReturned: 15628858
- RowsReturnedRate: 3504300
- ScanRangesComplete: 9
- ScannerThreadsInvoluntaryContextSwitches: 27011
- ScannerThreadsTotalWallClockTime: 1312995045025
- MaterializeTupleTime(*): 1292654625287
- ScannerThreadsSysTime: 8472527000
- ScannerThreadsUserTime: 152233512000
- ScannerThreadsVoluntaryContextSwitches: 200011
- TotalRawHdfsReadTime(*): 19909232607
- TotalReadThroughput: 1394909
- TotalTime: 4459907778
Instance 4242a648b2529ccb:b503ec709811c7dc (host=hdws78.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 9:1/125.96 MB 8:1/128.00 MB 7:1/126.05 MB 6:1/126.19 MB 0:3/381.91 MB 3:1/126.13 MB 1:2/254.09 MB
- AverageThreadTokens: 3.8027681660899653
- InactiveTotalTime: 0
- PeakMemoryUsage: 374318472
- PerHostPeakMemUsage: 662126184
- PrepareTime: 386958
- RowsProduced: 15212945
- TotalCpuTime: 1323923347202
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 445551720487
- TotalStorageWaitTime: 11497679364
- TotalTime: 468747578927
DataStreamSender (dst_id=3)
- BytesSent: 532428729
- InactiveTotalTime: 0
- NetworkThroughput(*): 712260
- OverallThroughput: 1151315
- PeakMemoryUsage: 557056
- SerializeBatchTime: 9064311172
- ThriftTransmitTime(*): 747519820487
- TotalTime: 462452616610
- UncompressedRowBatchSize: 867256741
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 9:1/125.96 MB 8:1/128.00 MB 7:1/126.05 MB 6:1/126.19 MB 0:3/381.91 MB 3:1/126.13 MB 1:2/254.09 MB
Hdfs Read Thread Concurrency Bucket: 0:94% 1:5.421% 2:0% 3:0.4614% 4:0% 5:0% 6:0% 7:0% 8:0.1153% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:18 PARQUET/SNAPPY:12
ExecOption: Codegen enabled: 0 out of 10
- AverageHdfsReadThreadConcurrency: 0.07727797001153403
- AverageScannerThreadConcurrency: 2.8027681660899653
- BytesRead: 644025721
- BytesReadDataNodeCache: 0
- BytesReadLocal: 134627226
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 134627226
- DecompressionTime: 6132460914
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 30
- NumDisksAccessed: 6
- NumRowGroups: 4
- NumScannerThreadsStarted: 5
- PeakMemoryUsage: 373818760
- PerReadThreadRawHdfsThroughput: 20017168
- RemoteScanRanges: 16
- RowsRead: 47504076
- RowsReturned: 15212945
- RowsReturnedRate: 2441215
- ScanRangesComplete: 10
- ScannerThreadsInvoluntaryContextSwitches: 33106
- ScannerThreadsTotalWallClockTime: 1311722146569
- MaterializeTupleTime(*): 1281934600225
- ScannerThreadsSysTime: 8788547000
- ScannerThreadsUserTime: 157497842000
- ScannerThreadsVoluntaryContextSwitches: 175998
- TotalRawHdfsReadTime(*): 32173667229
- TotalReadThroughput: 1485634
- TotalTime: 6231708045
Instance 4242a648b2529ccb:b503ec709811c7e2 (host=hdws72.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 16:1/128.00 MB 14:1/128.00 MB 11:1/126.12 MB 10:1/128.00 MB 1:1/126.21 MB 0:4/506.28 MB 5:1/126.13 MB
- AverageThreadTokens: 3.375478927203065
- InactiveTotalTime: 0
- PeakMemoryUsage: 346956872
- PerHostPeakMemUsage: 626039368
- PrepareTime: 478700
- RowsProduced: 13534354
- TotalCpuTime: 1021910734528
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 397050772457
- TotalStorageWaitTime: 4883983654
- TotalTime: 420013622497
DataStreamSender (dst_id=3)
- BytesSent: 467820679
- InactiveTotalTime: 0
- NetworkThroughput(*): 683202
- OverallThroughput: 1135015
- PeakMemoryUsage: 557056
- SerializeBatchTime: 6830956171
- ThriftTransmitTime(*): 684747059000
- TotalTime: 412171290275
- UncompressedRowBatchSize: 771563942
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 16:1/128.00 MB 14:1/128.00 MB 11:1/126.12 MB 10:1/128.00 MB 1:1/126.21 MB 0:4/506.28 MB 5:1/126.13 MB
Hdfs Read Thread Concurrency Bucket: 0:98.72% 1:0.7663% 2:0.2554% 3:0% 4:0.1277% 5:0.1277% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:18 PARQUET/SNAPPY:12
ExecOption: Codegen enabled: 0 out of 10
- AverageHdfsReadThreadConcurrency: 0.024265644955300127
- AverageScannerThreadConcurrency: 2.375478927203065
- BytesRead: 605595361
- BytesReadDataNodeCache: 0
- BytesReadLocal: 402857670
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 402857670
- DecompressionTime: 9238311219
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 30
- NumDisksAccessed: 6
- NumRowGroups: 4
- NumScannerThreadsStarted: 4
- PeakMemoryUsage: 346530888
- PerReadThreadRawHdfsThroughput: 66775500
- RemoteScanRanges: 14
- RowsRead: 44705627
- RowsReturned: 13534354
- RowsReturnedRate: 1741474
- ScanRangesComplete: 10
- ScannerThreadsInvoluntaryContextSwitches: 21532
- ScannerThreadsTotalWallClockTime: 1001952581712
- MaterializeTupleTime(*): 976339201707
- ScannerThreadsSysTime: 8448527000
- ScannerThreadsUserTime: 142012874000
- ScannerThreadsVoluntaryContextSwitches: 208554
- TotalRawHdfsReadTime(*): 9069125058
- TotalReadThroughput: 1546859
- TotalTime: 7771778183
Instance 4242a648b2529ccb:b503ec709811c7de (host=hdws67.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 13:1/128.00 MB 1:4/508.25 MB 10:1/126.15 MB 22:1/126.02 MB 0:1/126.05 MB 8:1/128.00 MB 6:1/128.00 MB
- AverageThreadTokens: 4.152905198776758
- InactiveTotalTime: 0
- PeakMemoryUsage: 375192976
- PerHostPeakMemUsage: 654302608
- PrepareTime: 427674
- RowsProduced: 10299431
- TotalCpuTime: 1203856396120
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 358175545492
- TotalStorageWaitTime: 3124691988
- TotalTime: 375089111409
DataStreamSender (dst_id=3)
- BytesSent: 366526909
- InactiveTotalTime: 0
- NetworkThroughput(*): 631158
- OverallThroughput: 986328
- PeakMemoryUsage: 557056
- SerializeBatchTime: 5265862129
- ThriftTransmitTime(*): 580720671044
- TotalTime: 371607271868
- UncompressedRowBatchSize: 587148055
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 13:1/128.00 MB 1:4/508.25 MB 10:1/126.15 MB 22:1/126.02 MB 0:1/126.05 MB 8:1/128.00 MB 6:1/128.00 MB
Hdfs Read Thread Concurrency Bucket: 0:94.95% 1:4.135% 2:0.7657% 3:0% 4:0% 5:0.1531% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:15 PARQUET/SNAPPY:15
ExecOption: Codegen enabled: 0 out of 10
- AverageHdfsReadThreadConcurrency: 0.06431852986217458
- AverageScannerThreadConcurrency: 3.1577335375191424
- BytesRead: 786557714
- BytesReadDataNodeCache: 0
- BytesReadLocal: 447068957
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 447068957
- DecompressionTime: 12472228503
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 30
- NumDisksAccessed: 6
- NumRowGroups: 5
- NumScannerThreadsStarted: 4
- PeakMemoryUsage: 374717840
- PerReadThreadRawHdfsThroughput: 29363386
- RemoteScanRanges: 14
- RowsRead: 55487457
- RowsReturned: 10299431
- RowsReturnedRate: 3007209
- ScanRangesComplete: 10
- ScannerThreadsInvoluntaryContextSwitches: 36718
- ScannerThreadsTotalWallClockTime: 1189518431774
- MaterializeTupleTime(*): 1148882768012
- ScannerThreadsSysTime: 13436838000
- ScannerThreadsUserTime: 188043750000
- ScannerThreadsVoluntaryContextSwitches: 407243
- TotalRawHdfsReadTime(*): 26787022989
- TotalReadThroughput: 2405367
- TotalTime: 3424912837
Instance 4242a648b2529ccb:b503ec709811c7da (host=hdws69.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 8:1/126.06 MB 3:1/126.18 MB 2:1/126.06 MB 1:3/384.00 MB 4:1/58.04 MB 0:3/315.24 MB
- AverageThreadTokens: 5.099280575539568
- InactiveTotalTime: 0
- PeakMemoryUsage: 431473432
- PerHostPeakMemUsage: 710555928
- PrepareTime: 290305
- RowsProduced: 9866146
- TotalCpuTime: 1520561609635
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 354067800348
- TotalStorageWaitTime: 2714960880
- TotalTime: 366264385876
DataStreamSender (dst_id=3)
- BytesSent: 349711623
- InactiveTotalTime: 0
- NetworkThroughput(*): 614138
- OverallThroughput: 959532
- PeakMemoryUsage: 540672
- SerializeBatchTime: 4847410593
- ThriftTransmitTime(*): 569434453513
- TotalTime: 364460243870
- UncompressedRowBatchSize: 562447438
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 8:1/126.06 MB 3:1/126.18 MB 2:1/126.06 MB 1:3/384.00 MB 4:1/58.04 MB 0:3/315.24 MB
Hdfs Read Thread Concurrency Bucket: 0:98.13% 1:1.007% 2:0.5755% 3:0% 4:0% 5:0.1439% 6:0% 7:0% 8:0.1439% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:12 PARQUET/SNAPPY:18
ExecOption: Codegen enabled: 0 out of 10
- AverageHdfsReadThreadConcurrency: 0.04028776978417266
- AverageScannerThreadConcurrency: 4.099280575539568
- BytesRead: 817499012
- BytesReadDataNodeCache: 0
- BytesReadLocal: 171856655
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 171856655
- DecompressionTime: 12423008475
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 30
- NumDisksAccessed: 6
- NumRowGroups: 6
- NumScannerThreadsStarted: 7
- PeakMemoryUsage: 431104792
- PerReadThreadRawHdfsThroughput: 61351214
- RemoteScanRanges: 19
- RowsRead: 57310058
- RowsReturned: 9866146
- RowsReturnedRate: 5582324
- ScanRangesComplete: 10
- ScannerThreadsInvoluntaryContextSwitches: 32162
- ScannerThreadsTotalWallClockTime: 1508790112748
- MaterializeTupleTime(*): 1460447043291
- ScannerThreadsSysTime: 26557658000
- ScannerThreadsUserTime: 198732418000
- ScannerThreadsVoluntaryContextSwitches: 921887
- TotalRawHdfsReadTime(*): 13324903511
- TotalReadThroughput: 2352515
- TotalTime: 1767390125
Instance 4242a648b2529ccb:b503ec709811c7e0 (host=hdws75.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/128.00 MB 1:4/510.11 MB 16:1/128.00 MB 0:3/380.23 MB
- AverageThreadTokens: 4.197268588770865
- InactiveTotalTime: 0
- PeakMemoryUsage: 337879872
- PerHostPeakMemUsage: 635793208
- PrepareTime: 308088
- RowsProduced: 9000458
- TotalCpuTime: 1165129365418
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 340877794661
- TotalStorageWaitTime: 5877204585
- TotalTime: 358029981991
DataStreamSender (dst_id=3)
- BytesSent: 320524974
- InactiveTotalTime: 0
- NetworkThroughput(*): 609441
- OverallThroughput: 912844
- PeakMemoryUsage: 540672
- SerializeBatchTime: 6890776602
- ThriftTransmitTime(*): 525932551889
- TotalTime: 351127836231
- UncompressedRowBatchSize: 513096446
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 18:1/128.00 MB 1:4/510.11 MB 16:1/128.00 MB 0:3/380.23 MB
Hdfs Read Thread Concurrency Bucket: 0:86.15% 1:9.741% 2:3.957% 3:0.1522% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:9 PARQUET/SNAPPY:18
ExecOption: Codegen enabled: 0 out of 9
- AverageHdfsReadThreadConcurrency: 0.1811263318112633
- AverageScannerThreadConcurrency: 3.207001522070015
- BytesRead: 933992020
- BytesReadDataNodeCache: 0
- BytesReadLocal: 268537645
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 268537645
- DecompressionTime: 18960636650
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 27
- NumDisksAccessed: 4
- NumRowGroups: 6
- NumScannerThreadsStarted: 5
- PeakMemoryUsage: 337265464
- PerReadThreadRawHdfsThroughput: 14398113
- RemoteScanRanges: 22
- RowsRead: 64892755
- RowsReturned: 9000458
- RowsReturnedRate: 1320431
- ScanRangesComplete: 9
- ScannerThreadsInvoluntaryContextSwitches: 42739
- ScannerThreadsTotalWallClockTime: 1152217911180
- MaterializeTupleTime(*): 1110275711209
- ScannerThreadsSysTime: 16000998000
- ScannerThreadsUserTime: 208901053000
- ScannerThreadsVoluntaryContextSwitches: 416758
- TotalRawHdfsReadTime(*): 64869054011
- TotalReadThroughput: 2843202
- TotalTime: 6816299111
Instance 4242a648b2529ccb:b503ec709811c7e4 (host=hdws74.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 4:1/126.12 MB 2:1/128.00 MB 21:1/126.14 MB 20:1/128.00 MB 19:1/128.00 MB 17:1/125.96 MB 15:1/125.98 MB 14:1/126.17 MB 0:3/330.04 MB 1:3/359.74 MB 10:1/128.00 MB
- AverageThreadTokens: 3.808988764044944
- InactiveTotalTime: 0
- PeakMemoryUsage: 363505552
- PerHostPeakMemUsage: 642588048
- PrepareTime: 499116
- RowsProduced: 6672051
- TotalCpuTime: 860656152394
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 296490529594
- TotalStorageWaitTime: 5132791794
- TotalTime: 305797374680
DataStreamSender (dst_id=3)
- BytesSent: 233250909
- InactiveTotalTime: 0
- NetworkThroughput(*): 516367
- OverallThroughput: 769839
- PeakMemoryUsage: 557056
- SerializeBatchTime: 3642209863
- ThriftTransmitTime(*): 451715001820
- TotalTime: 302986286607
- UncompressedRowBatchSize: 380359067
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 4:1/126.12 MB 2:1/128.00 MB 21:1/126.14 MB 20:1/128.00 MB 19:1/128.00 MB 17:1/125.96 MB 15:1/125.98 MB 14:1/126.17 MB 0:3/330.04 MB 1:3/359.74 MB 10:1/128.00 MB
Hdfs Read Thread Concurrency Bucket: 0:96.07% 1:3.371% 2:0.1873% 3:0.1873% 4:0.1873% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:27 PARQUET/SNAPPY:18
ExecOption: Codegen enabled: 0 out of 15
- AverageHdfsReadThreadConcurrency: 0.05056179775280899
- AverageScannerThreadConcurrency: 2.808988764044944
- BytesRead: 897397896
- BytesReadDataNodeCache: 0
- BytesReadLocal: 542623765
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 542623765
- DecompressionTime: 14140904886
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 45
- NumDisksAccessed: 10
- NumRowGroups: 6
- NumScannerThreadsStarted: 4
- PeakMemoryUsage: 362751880
- PerReadThreadRawHdfsThroughput: 62389876
- RemoteScanRanges: 18
- RowsRead: 62340891
- RowsReturned: 6672051
- RowsReturnedRate: 2404746
- ScanRangesComplete: 15
- ScannerThreadsInvoluntaryContextSwitches: 40758
- ScannerThreadsTotalWallClockTime: 856018072885
- MaterializeTupleTime(*): 808973571124
- ScannerThreadsSysTime: 14776921000
- ScannerThreadsUserTime: 202576658000
- ScannerThreadsVoluntaryContextSwitches: 407924
- TotalRawHdfsReadTime(*): 14383710130
- TotalReadThroughput: 3361015
- TotalTime: 2774533461
Instance 4242a648b2529ccb:b503ec709811c7d9 (host=hdws84.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:2/256.00 MB 1:3/380.19 MB
- AverageThreadTokens: 3.724264705882353
- InactiveTotalTime: 0
- PeakMemoryUsage: 300204520
- PerHostPeakMemUsage: 589027256
- PrepareTime: 220795
- RowsProduced: 3604830
- TotalCpuTime: 418827405519
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 144449597147
- TotalStorageWaitTime: 2022887658
- TotalTime: 150627830518
DataStreamSender (dst_id=3)
- BytesSent: 125179761
- InactiveTotalTime: 0
- NetworkThroughput(*): 633693
- OverallThroughput: 856656
- PeakMemoryUsage: 532480
- SerializeBatchTime: 1713055337
- ThriftTransmitTime(*): 197539817768
- TotalTime: 146126018050
- UncompressedRowBatchSize: 205503494
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:2/256.00 MB 1:3/380.19 MB
Hdfs Read Thread Concurrency Bucket: 0:96.67% 1:2.593% 2:0% 3:0% 4:0.3704% 5:0% 6:0.3704% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:6 PARQUET/SNAPPY:9
ExecOption: Codegen enabled: 0 out of 5
- AverageHdfsReadThreadConcurrency: 0.06296296296296296
- AverageScannerThreadConcurrency: 2.7444444444444445
- BytesRead: 425651973
- BytesReadDataNodeCache: 0
- BytesReadLocal: 0
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 0
- DecompressionTime: 8838642100
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 15
- NumDisksAccessed: 1
- NumRowGroups: 3
- NumScannerThreadsStarted: 3
- PeakMemoryUsage: 299631072
- PerReadThreadRawHdfsThroughput: 45509506
- RemoteScanRanges: 14
- RowsRead: 29970789
- RowsReturned: 3604830
- RowsReturnedRate: 806073
- ScanRangesComplete: 5
- ScannerThreadsInvoluntaryContextSwitches: 12319
- ScannerThreadsTotalWallClockTime: 412971976694
- MaterializeTupleTime(*): 396015733548
- ScannerThreadsSysTime: 3260203000
- ScannerThreadsUserTime: 87573471000
- ScannerThreadsVoluntaryContextSwitches: 82416
- TotalRawHdfsReadTime(*): 9353034166
- TotalReadThroughput: 3152977
- TotalTime: 4472087099
Instance 4242a648b2529ccb:b503ec709811c7e6 (host=hdws70.houzz.com:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 11:1/126.17 MB 1:3/253.57 MB 0:4/500.00 MB 15:1/128.00 MB
- AverageThreadTokens: 4.570934256055363
- InactiveTotalTime: 0
- PeakMemoryUsage: 404973032
- PerHostPeakMemUsage: 697217072
- PrepareTime: 287409
- RowsProduced: 3173687
- TotalCpuTime: 557229380470
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 143120631071
- TotalStorageWaitTime: 1983596385
- TotalTime: 150528922596
DataStreamSender (dst_id=3)
- BytesSent: 110014744
- InactiveTotalTime: 0
- NetworkThroughput(*): 567243
- OverallThroughput: 752278
- PeakMemoryUsage: 540672
- SerializeBatchTime: 1417907925
- ThriftTransmitTime(*): 193946329696
- TotalTime: 146241972515
- UncompressedRowBatchSize: 180924983
HDFS_SCAN_NODE (id=0)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 11:1/126.17 MB 1:3/253.57 MB 0:4/500.00 MB 15:1/128.00 MB
Hdfs Read Thread Concurrency Bucket: 0:96.88% 1:1.736% 2:1.042% 3:0% 4:0% 5:0% 6:0.3472% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0%
File Formats: PARQUET/NONE:12 PARQUET/SNAPPY:15
ExecOption: Codegen enabled: 0 out of 9
- AverageHdfsReadThreadConcurrency: 0.059027777777777776
- AverageScannerThreadConcurrency: 3.5833333333333335
- BytesRead: 622564083
- BytesReadDataNodeCache: 0
- BytesReadLocal: 268640046
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 268640046
- DecompressionTime: 9726231175
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 27
- NumDisksAccessed: 5
- NumRowGroups: 5
- NumScannerThreadsStarted: 4
- PeakMemoryUsage: 404260320
- PerReadThreadRawHdfsThroughput: 61208756
- RemoteScanRanges: 18
- RowsRead: 43085342
- RowsReturned: 3173687
- RowsReturnedRate: 743660
- ScanRangesComplete: 9
- ScannerThreadsInvoluntaryContextSwitches: 20481
- ScannerThreadsTotalWallClockTime: 548613200112
- MaterializeTupleTime(*): 521585405724
- ScannerThreadsSysTime: 16133006000
- ScannerThreadsUserTime: 132632287000
- ScannerThreadsVoluntaryContextSwitches: 699536
- TotalRawHdfsReadTime(*): 10171160378
- TotalReadThroughput: 4323361
- TotalTime: 4267655991
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment