Skip to content

Instantly share code, notes, and snippets.

View josete89's full-sized avatar

Jose Luis josete89

35 followers · 32 following
  • adidas-group
  • Zaragoza
View GitHub Profile
@josete89
josete89 / delta-hive-describe-formatted.md
Last active May 4, 2022 16:42

DESCRIBE FORMATTED `delta-table`;

Query result

# col_name            	data_type           	comment             	 	 
col                 	array<string>       	from deserializer   

# Detailed Table Information
Database: test
@josete89
josete89 / describe-normal-hive.md
Created May 3, 2022 11:32

DESCRIBE non-delta-table

col_name data_type comment
id int
productName string
quantity int
@josete89
josete89 / delta-hive-table.md
Last active May 3, 2022 11:34

DESCRIBE delta-table

Query result:

col_name data_type comment
col array|from deserializer
@josete89
josete89 / data-product-visibility.json
Last active May 3, 2022 11:19
{
"tables": [{
"name": "sales_predictions",
"visibility": "internal",
"columns": [{ "name": "article_number","pci": true }]
},
{
"name": "sales_trends",
"visibility": "public",
"columns": [{ "name": "date","pci": true }]
@josete89
josete89 / data-product-outputs.json
Last active May 3, 2022 11:14
{
"outputs":{
"sales_trends":{
"file-format":"parquet",
"port-type": "blob",
"location": "s3://my-data-product/sales_trends",
"partition-columns": [ "foobar" ],
"retention-time": {"unit": "day","time": 2}
},
"sales_predictions":{
@josete89
josete89 / data-product-inputs.json
Last active May 3, 2022 11:10
{
"inputs": {
"DataProductA":{
"tables": ["table1","table2"]
}
}
}
@josete89
josete89 / data-product-consumers.json
Last active May 3, 2022 11:08
{
"consumers": [{
"account":"99987654321",
"contact": "foo.bar@adidas.com",
"type": "lakeformation",
"enterprise-system-landscape-information-tracker": "foobar",
"team": "foobar",
"consumer-role-arn": "arn:aws:iam::99987654321:role/ConsumerRole"
}]
}
@josete89
josete89 / data-product-producer-info.json
Created May 3, 2022 11:05
{
"account-type": "aws",
"producer-role-arn": "arn:aws:iam::12345678999:role/ProducerRole",
"producer-account-id": "12345678999",
"bucket-name": "trading-producer-data"
}
@josete89
josete89 / data-product.json
Last active May 3, 2022 11:03
{
"data-product-name": "my_data_product",
"data-product-owner": "foo.bar@adidas.com",
"data-product-mantainer": "foo.bar@adidas.com",
"enterprise-system-landscape-information-tracker": "foobar",
"edc-data-objects": [ "foobar" ],"source-system": "SAP-AFS",
"usage-patterns": ["foobar"],
"version": "0.0.1"
}
@josete89
josete89 / keybase.md
Created September 16, 2021 06:57

Keybase proof

I hereby claim:

  • I am josete89 on github.
  • I am alcaljos (https://keybase.io/alcaljos) on keybase.
  • I have a public key ASC05l0GF0Ny1y_ccI0Z7IZ4ItSpatIIxWnXnDpN9iVK5Qo

To claim this, I am signing this object: