Skip to content

Instantly share code, notes, and snippets.

@velotiotech
Created May 22, 2024 10:37
Show Gist options
  • Save velotiotech/11f8849c594008b9d7171690b3abf8a5 to your computer and use it in GitHub Desktop.
Save velotiotech/11f8849c594008b9d7171690b3abf8a5 to your computer and use it in GitHub Desktop.
{
"conf": {
"spark.sql.catalog.glue_catalog1": "org.apache.iceberg.spark.SparkCatalog",
"spark.sql.catalog.glue_catalog1.warehouse":
"s3://YOUR-BUCKET-NAME/iceberg/glue_catalog1/tables/",
"spark.sql.catalog.glue_catalog1.catalog-impl": "org.apache.iceberg.aws.glue.GlueCatalog",
"spark.sql.catalog.glue_catalog1.io-impl": "org.apache.iceberg.aws.s3.S3FileIO",
"spark.sql.catalog.glue_catalog1.lock-impl": "org.apache.iceberg.aws.glue.DynamoLockManager",
"spark.sql.catalog.glue_catalog1.lock.table": "myGlueLockTable",
"spark.sql.extensions": "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment