Skip to content

Instantly share code, notes, and snippets.

@1ambda

1ambda/df-create-table.py

Created December 26, 2021 23:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 1ambda/5425d2d92024924dae8c563b3555cc28 to your computer and use it in GitHub Desktop.
Save 1ambda/5425d2d92024924dae8c563b3555cc28 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
df-create-table.py
# DESCRIBE DATABASE udon_db;
+----------+----------+-------------------------------+-------------+-------------+-------------+
| db_name | comment | location | owner_name | owner_type | parameters |
+----------+----------+-------------------------------+-------------+-------------+-------------+
| udon_db | Udon DB | s3://udon-data-lake/db | hadoop | USER | |
+----------+----------+-------------------------------+-------------+-------------+-------------+
# USE udon_db;
# SHOW TABLES;
+---------------------+
| tab_name |
+---------------------+
| udon_topping_sales |
+---------------------+
# DESCRIBE FORMATTED udon_topping_sales;
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| col_name | data_type | comment |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| # col_name | data_type | comment |
| | NULL | NULL |
| udon_topping_id | bigint | |
| udon_topping_name | string | |
| sales_price | double | |
| shipping_date | date | |
| created_at | timestamp | |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | udon_db | NULL |
| Owner: | hadoop | NULL |
| CreateTime: | Sat Sep 06 09:00:00 KST 2021 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | s3://udon-data-lake/db/udon_topping_sales | NULL |
| Table Type: | EXTERNAL_TABLE | NULL |
| Table Parameters: | NULL | NULL |
| | EXTERNAL | TRUE |
| | comment | Udon Topping Sales |
| | spark.sql.create.version | 3.1.2 |
| | spark.sql.sources.provider | PARQUET |
| | spark.sql.sources.schema.numParts | 1 |
| | spark.sql.sources.schema.part.0 | {\"type\":\"struct\",\"fields\":[{\"name\":\"udon_topping_id\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"udon_topping_name\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"sales_price\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"shipping_date\",\"type\":\"date\",\"nullable\":true,\"metadata\":{}},{\"name\":\"created_at\",\"type\":\"timestamp\",\"nullable\":true,\"metadata\":{}}]} |
| | transient_lastDdlTime | 1636125473 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe | NULL |
| InputFormat: | org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| Storage Desc Params: | NULL | NULL |
| | path | s3://udon-data-lake/db/udon_topping_sales |
| | serialization.format | 1 |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment