Created
August 4, 2020 18:34
-
-
Save mingwei-li/b0d444aa2d5778c3eb369f5c31f7bcda to your computer and use it in GitHub Desktop.
hyperspace - explain
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
============================================================= | |
Plan with indexes: | |
============================================================= | |
Project [name#11] | |
+- Filter (isnotnull(id#10) && (id#10 = 1)) | |
<----+- FileScan parquet [id#10,name#11] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/Users/mingwli/Dev/lib/hadoop-2.9.2/bin/spark-warehouse/indexes/index/v__=0], PartitionFilters: [], PushedFilters: [IsNotNull(id), EqualTo(id,1)], ReadSchema: struct<id:int,name:string>----> | |
============================================================= | |
Plan without indexes: | |
============================================================= | |
Project [name#11] | |
+- Filter (isnotnull(id#10) && (id#10 = 1)) | |
<----+- FileScan csv [id#10,name#11] Batched: false, Format: CSV, Location: InMemoryFileIndex[hdfs://localhost:9000/mingwli/customers.csv], PartitionFilters: [], PushedFilters: [IsNotNull(id), EqualTo(id,1)], ReadSchema: struct<id:int,name:string>----> | |
============================================================= | |
Indexes used: | |
============================================================= | |
index:file:/Users/mingwli/Dev/lib/hadoop-2.9.2/bin/spark-warehouse/indexes/index/v__=0 | |
============================================================= | |
Physical operator stats: | |
============================================================= | |
+-----------------+-------------------+------------------+----------+ | |
|Physical Operator|Hyperspace Disabled|Hyperspace Enabled|Difference| | |
+-----------------+-------------------+------------------+----------+ | |
| *Scan csv| 1| 0| -1| | |
| *Scan parquet| 0| 1| 1| | |
| Filter| 1| 1| 0| | |
| Project| 1| 1| 0| | |
|WholeStageCodegen| 1| 1| 0| | |
+-----------------+-------------------+------------------+----------+ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment