Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Query (id=2b49899693ba3112:4b7446bd79225bb1)
Summary
Session ID: a741df1a506dd43b:84c23b91de4180a0
Session Type: HIVESERVER2
HiveServer2 Protocol Version: V6
Start Time: 2018-05-11 09:24:23.146384000
End Time:
Query Type: QUERY
Query State: CREATED
Query Status: OK
Impala Version: impalad version 2.6.0-cdh5.8.4 RELEASE (build 207450616f75adbe082a4c2e1145a2384da83fa6)
User: talend
Connected User: hue/blabla01.xxxxx@xxxxx
Delegated User: talend
Network Address: ::ffff:10.192.98.6:56908
Default Db: bd_tableau
Sql Statement: select * from bd_tableau.ref_oc_rb_6
where date_pr='20151130'
Coordinator: blabla15.xxxxx:22000
Query Options (non default): QUERY_TIMEOUT_S=600
Plan:
----------------
Estimated Per-Host Requirements: Memory=320.00MB VCores=1
01:EXCHANGE [UNPARTITIONED]
| hosts=2 per-host-mem=unavailable
| tuple-ids=0 row-size=525B cardinality=3237
|
00:SCAN HDFS [bd_tableau.ref_oc_rb_6, RANDOM]
partitions=1/30 files=2 size=127.75KB
table stats: 3237 rows total
column stats: all
hosts=2 per-host-mem=320.00MB
tuple-ids=0 row-size=525B cardinality=3237
----------------
Estimated Per-Host Mem: 335544320
Estimated Per-Host VCores: 1
Request Pool: root.talend
Admission result: Admitted immediately
Planner Timeline
Analysis finished: 1233149
Equivalence classes computed: 1390866
Single node plan created: 3901266
Runtime filters computed: 3924530
Distributed plan created: 4666000
Lineage info computed: 5422245
Planning finished: 6676371
Query Timeline
Start execution: 136652
Planning finished: 8345173
Submit for admission: 9290994
Completed admission: 9436580
Ready to start 2 remote fragments: 10915913
All 2 remote fragments started: 12153245
Child queries finished: 12651168
Rows available: 32803455
First row fetched: 121430645
ImpalaServer
- ClientFetchWaitTimer: 88692793
- InactiveTotalTime: 0
- RowMaterializationTimer: 359503
- TotalTime: 0
Execution Profile 2b49899693ba3112:4b7446bd79225bb1
Number of filters: 0
Filter routing table:
ID Src. Node Tgt. Node(s) Targets Target type Partition filter Pending (Expected) First arrived Completed
-------------------------------------------------------------------------------------------------------------------
Fragment start latencies: Count: 2, 25th %-ile: 0, 50th %-ile: 0, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, 99.9th %-ile: 1ms
- FiltersReceived: 0
- FinalizationTimer: 0
- InactiveTotalTime: 0
- TotalTime: 23050328
Coordinator Fragment F01
- AverageThreadTokens: 0.0
- BloomFilterBytes: 0
- InactiveTotalTime: 0
- PeakMemoryUsage: 1790168
- PerHostPeakMemUsage: 0
- PrepareTime: 87176
- RowsProduced: 1024
- TotalCpuTime: 0
- TotalNetworkReceiveTime: 20098957
- TotalNetworkSendTime: 0
- TotalStorageWaitTime: 0
- TotalTime: 20883835
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 77309411328
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
EXCHANGE_NODE (id=1)
- BytesReceived: 628202
- ConvertRowBatchTime: 12275
- DeserializeRowBatchTimer: 1887200
- FirstBatchArrivalWaitTime: 20094463
- InactiveTotalTime: 20095552
- PeakMemoryUsage: 0
- RowsReturned: 1024
- RowsReturnedRate: 50912
- SendersBlockedTimer: 0
- SendersBlockedTotalTimer(*): 0
- TotalTime: 20112901
Averaged Fragment F00
split sizes: min: 62.21 KB, max: 65.53 KB, avg: 63.87 KB, stddev: 1.66 KB
- AverageThreadTokens: 0.0
- BloomFilterBytes: 0
- InactiveTotalTime: 0
- PeakMemoryUsage: 1963404
- PerHostPeakMemUsage: 1963404
- PrepareTime: 65029
- RowsProduced: 1618
- TotalCpuTime: 27072477
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 1763799
- TotalStorageWaitTime: 19018641
- TotalTime: 25985564
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 77309411328
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
DataStreamSender (dst_id=1)
- BytesSent: 314101
- InactiveTotalTime: 0
- NetworkThroughput(*): 796887898
- OverallThroughput: 99724418
- PeakMemoryUsage: 376
- RowsReturned: 1618
- SerializeBatchTime: 2724596
- TotalTime: 3203409
- TransmitDataRPCTime: 465577
- UncompressedRowBatchSize: 884515
HDFS_SCAN_NODE (id=0)
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.0
- BytesRead: 127458
- BytesReadDataNodeCache: 0
- BytesReadLocal: 127458
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 127458
- DecompressionTime: 256096
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 20
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 1954836
- PerReadThreadRawHdfsThroughput: 697984330
- RemoteScanRanges: 0
- RowsRead: 1618
- RowsReturned: 1618
- RowsReturnedRate: 79850
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 0
- ScannerThreadsTotalWallClockTime: 21501950
- MaterializeTupleTime(*): 979563
- ScannerThreadsSysTime: 737500
- ScannerThreadsUserTime: 1471000
- ScannerThreadsVoluntaryContextSwitches: 20
- TotalRawHdfsReadTime(*): 183523
- TotalReadThroughput: 0
- TotalTime: 20902029
Fragment F00
Instance 2b49899693ba3112:4b7446bd79225bb4 (host=blabla24.xxxxx:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/65.53 KB
- AverageThreadTokens: 0.0
- BloomFilterBytes: 0
- InactiveTotalTime: 0
- PeakMemoryUsage: 1998896
- PerHostPeakMemUsage: 1998896
- PrepareTime: 64260
- RowsProduced: 1657
- TotalCpuTime: 22854843
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 1538205
- TotalStorageWaitTime: 16036895
- TotalTime: 22083319
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 77309411328
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
DataStreamSender (dst_id=1)
- BytesSent: 321123
- InactiveTotalTime: 0
- NetworkThroughput(*): 1116627199
- OverallThroughput: 113780886
- PeakMemoryUsage: 376
- RowsReturned: 1657
- SerializeBatchTime: 2520198
- TotalTime: 2822293
- TransmitDataRPCTime: 287583
- UncompressedRowBatchSize: 897444
HDFS_SCAN_NODE (id=0)
ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 1
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 1:1/65.53 KB
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0%
File Formats: PARQUET/SNAPPY:20
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.0
- BytesRead: 130865
- BytesReadDataNodeCache: 0
- BytesReadLocal: 130865
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 130865
- DecompressionTime: 165620
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 20
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 1990328
- PerReadThreadRawHdfsThroughput: 757399497
- RemoteScanRanges: 0
- RowsRead: 1657
- RowsReturned: 1657
- RowsReturnedRate: 94697
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 0
- ScannerThreadsTotalWallClockTime: 18064172
- MaterializeTupleTime(*): 818353
- ScannerThreadsSysTime: 657000
- ScannerThreadsUserTime: 1305000
- ScannerThreadsVoluntaryContextSwitches: 14
- TotalRawHdfsReadTime(*): 172782
- TotalReadThroughput: 0
- TotalTime: 17497785
Instance 2b49899693ba3112:4b7446bd79225bb3 (host=blabla21.xxxxx:22000)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 6:1/62.21 KB
- AverageThreadTokens: 0.0
- BloomFilterBytes: 0
- InactiveTotalTime: 0
- PeakMemoryUsage: 1927912
- PerHostPeakMemUsage: 1927912
- PrepareTime: 65798
- RowsProduced: 1580
- TotalCpuTime: 31290112
- TotalNetworkReceiveTime: 0
- TotalNetworkSendTime: 1989393
- TotalStorageWaitTime: 22000388
- TotalTime: 29887809
BlockMgr
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- InactiveTotalTime: 0
- MaxBlockSize: 8388608
- MemoryLimit: 77309411328
- PeakMemoryUsage: 0
- TotalBufferWaitTime: 0
- TotalEncryptionTime: 0
- TotalIntegrityCheckTime: 0
- TotalReadBlockTime: 0
- TotalTime: 0
DataStreamSender (dst_id=1)
- BytesSent: 307079
- InactiveTotalTime: 0
- NetworkThroughput(*): 477148597
- OverallThroughput: 85667951
- PeakMemoryUsage: 376
- RowsReturned: 1580
- SerializeBatchTime: 2928995
- TotalTime: 3584526
- TransmitDataRPCTime: 643571
- UncompressedRowBatchSize: 871587
HDFS_SCAN_NODE (id=0)
ExecOption: Expr Evaluation Codegen Disabled, Codegen enabled: 0 out of 1
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 6:1/62.21 KB
Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0% 18:0% 19:0% 20:0% 21:0% 22:0% 23:0% 24:0% 25:0% 26:0% 27:0% 28:0%
File Formats: PARQUET/SNAPPY:20
- AverageHdfsReadThreadConcurrency: 0.0
- AverageScannerThreadConcurrency: 0.0
- BytesRead: 124051
- BytesReadDataNodeCache: 0
- BytesReadLocal: 124051
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 124051
- DecompressionTime: 346573
- InactiveTotalTime: 0
- MaxCompressedTextFileLength: 0
- NumColumns: 20
- NumDisksAccessed: 1
- NumRowGroups: 1
- NumScannerThreadsStarted: 1
- PeakMemoryUsage: 1919344
- PerReadThreadRawHdfsThroughput: 638569163
- RemoteScanRanges: 0
- RowsRead: 1580
- RowsReturned: 1580
- RowsReturnedRate: 65003
- ScanRangesComplete: 1
- ScannerThreadsInvoluntaryContextSwitches: 0
- ScannerThreadsTotalWallClockTime: 24939728
- MaterializeTupleTime(*): 1140773
- ScannerThreadsSysTime: 818000
- ScannerThreadsUserTime: 1637000
- ScannerThreadsVoluntaryContextSwitches: 26
- TotalRawHdfsReadTime(*): 194264
- TotalReadThroughput: 0
- TotalTime: 24306274
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.