Skip to content

Instantly share code, notes, and snippets.

@puneetloya
Created May 17, 2023 00:42
Show Gist options
  • Save puneetloya/5a15456b7c84fe0d76723545fe937287 to your computer and use it in GitHub Desktop.
Save puneetloya/5a15456b7c84fe0d76723545fe937287 to your computer and use it in GitHub Desktop.
logs-databricks-tf-failure
2023-05-17T00:38:36.500Z [DEBUG] Starting graph walk: walkPlan
2023-05-17T00:38:36.506Z [INFO] ReferenceTransformer: reference not found: "var.env"
2023-05-17T00:38:36.506Z [INFO] ReferenceTransformer: reference not found: "var.org_name"
2023-05-17T00:38:36.506Z [DEBUG] ReferenceTransformer: "data.terraform_remote_state.workspace" references: []
2023-05-17T00:38:36.508Z [DEBUG] Initializing remote state backend: s3
2023-05-17T00:38:36.513Z [DEBUG] Initializing remote state backend: s3
2023-05-17T00:38:36.516Z [INFO] AWS Auth provider used: "EnvProvider"
2023-05-17T00:38:36.517Z [DEBUG] Trying to get account information via sts:GetCallerIdentity
2023-05-17T00:38:36.518Z [DEBUG] [aws-sdk-go] DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.2.9 aws-sdk-go/1.42.35 (go1.18.1; linux; amd64)
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=AKIAYTJ22SJRFGKD72PY/20230517/us-east-1/sts/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=89d37d2c4a89f694adbc92341cfc29f45ba9b8df6248e970697dcdb40ce98335
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20230517T003836Z
Accept-Encoding: gzip
Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2023-05-17T00:38:36.900Z [DEBUG] [aws-sdk-go] DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 403
Content-Type: text/xml
Date: Wed, 17 May 2023 00:38:36 GMT
X-Amzn-Requestid: f043a437-7b11-46f2-aea5-1189c98575f4
-----------------------------------------------------
2023-05-17T00:38:36.901Z [DEBUG] [aws-sdk-go] <GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<GetCallerIdentityResult>
<Arn>arn:aws:iam::591218250338:user/puneet</Arn>
<UserId>AIDAYTJ22SJRDT4C4T6LI</UserId>
<Account>591218250338</Account>
</GetCallerIdentityResult>
<ResponseMetadata>
<RequestId>f043a437-7b11-46f2-aea5-1189c98575f4</RequestId>
</ResponseMetadata>
</GetCallerIdentityResponse>
2023-05-17T00:38:36.914Z [DEBUG] [aws-sdk-go] DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /?max-keys=1000&prefix=env%3A%2F HTTP/1.1
Host: prod-mojo-korrel8-tf-state.s3.us-east-2.amazonaws.com
User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.2.9 aws-sdk-go/1.42.35 (go1.18.1; linux; amd64)
Authorization: AWS4-HMAC-SHA256 Credential=AKIAYTJ22SJRFGKD72PY/20230517/us-east-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=f4933e1f866778acd2363d2670c9f2b1ba14e1d424d303821c744da15c23f2ea
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230517T003836Z
Accept-Encoding: gzip
-----------------------------------------------------
2023-05-17T00:38:37.317Z [DEBUG] [aws-sdk-go] DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Wed, 17 May 2023 00:38:38 GMT
Server: AmazonS3
X-Amz-Bucket-Region: us-east-2
X-Amz-Id-2: zai7oZu2nn03A+JjzV6oQ4DUvxCRjptI6vYwOphfL8aOkriNc0p5t9mMrXJsgkkCO/kS8HJxDPU=
X-Amz-Request-Id: KC57WZBM7WDAYR2B
-----------------------------------------------------
2023-05-17T00:38:37.318Z [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>prod-mojo-korrel8-tf-state</Name><Prefix>env:/</Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated></ListBucketResult>
2023-05-17T00:38:37.328Z [DEBUG] [aws-sdk-go] DEBUG: Request s3/GetObject Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /databricks-workspace/terraform.tfstate HTTP/1.1
Host: prod-mojo-korrel8-tf-state.s3.us-east-2.amazonaws.com
User-Agent: APN/1.0 HashiCorp/1.0 Terraform/1.2.9 aws-sdk-go/1.42.35 (go1.18.1; linux; amd64)
Authorization: AWS4-HMAC-SHA256 Credential=AKIAYTJ22SJRFGKD72PY/20230517/us-east-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=d48caf11d3664639e0d250e111cf7382547925e68ab926b72cdde159a0af7c1c
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230517T003837Z
Accept-Encoding: gzip
-----------------------------------------------------
2023-05-17T00:38:37.816Z [DEBUG] [aws-sdk-go] DEBUG: Response s3/GetObject Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 27325
Accept-Ranges: bytes
Content-Type: application/json
Date: Wed, 17 May 2023 00:38:38 GMT
Etag: "4a0270b5ab9aad51a89e5e6183180aab"
Last-Modified: Tue, 16 May 2023 04:52:45 GMT
Server: AmazonS3
X-Amz-Id-2: +pdVBgJaIHuIMsUS43Nojn+qPzACxu9IV0/g/vn4LNhkOF05YgHeYzoMFUcJhxGCHJttHLduyBc=
X-Amz-Request-Id: KC51CKX1E0NZVMXM
X-Amz-Server-Side-Encryption: AES256
X-Amz-Version-Id: VJ6.cPK4_LBagjsZDrAxDLUcQovuzKge
-----------------------------------------------------
2023-05-17T00:38:37.816Z [DEBUG] [aws-sdk-go]
2023-05-17T00:38:37.908Z [DEBUG] created provider logger: level=debug
2023-05-17T00:38:37.908Z [INFO] provider: configuring client automatic mTLS
2023-05-17T00:38:38.221Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 args=[.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0]
2023-05-17T00:38:38.233Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 pid=434
2023-05-17T00:38:38.233Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0
2023-05-17T00:38:38.459Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Databricks Terraform Provider
2023-05-17T00:38:38.459Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:38.459Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Version 1.15.0
2023-05-17T00:38:38.459Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:38.459Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: https://registry.terraform.io/providers/databricks/databricks/latest/docs
2023-05-17T00:38:38.459Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:38.466Z [INFO] provider.terraform-provider-databricks_v1.15.0: configuring server automatic mTLS: timestamp=2023-05-17T00:38:38.463Z
2023-05-17T00:38:38.896Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: plugin address: address=/tmp/plugin613865868 network=unix timestamp=2023-05-17T00:38:38.896Z
2023-05-17T00:38:38.896Z [DEBUG] provider: using plugin: version=5
2023-05-17T00:38:39.360Z [WARN] ValidateProviderConfig from "provider[\"registry.terraform.io/databricks/databricks\"]" changed the config value, but that value is unused
2023-05-17T00:38:39.372Z [INFO] provider.terraform-provider-databricks_v1.15.0: Explicit and implicit attributes: host, token: timestamp=2023-05-17T00:38:39.371Z
2023-05-17T00:38:39.376Z [INFO] ReferenceTransformer: reference not found: "var.bucket"
2023-05-17T00:38:39.376Z [INFO] ReferenceTransformer: reference not found: "var.databricks_workspace_name"
2023-05-17T00:38:39.376Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.databricks_metastore.this" references: []
2023-05-17T00:38:40.119Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: GET /api/2.1/unity-catalog/metastores/bdf99cfb-5e7d-47e9-ba0c-631f149c8fc9
< HTTP/2.0 200 OK
< {
< "cloud": "aws",
< "created_at": 1684212848959,
< "created_by": "puneet@company.ai",
< "delta_sharing_scope": "INTERNAL",
< "full_name": "prod-mojo",
< "global_metastore_id": "aws:us-east-2:bdf99cfb-5e7d-47e9-ba0c-631f149c8fc9",
< "metastore_id": "bdf99cfb-5e7d-47e9-ba0c-631f149c8fc9",
< "name": "prod-mojo",
< "owner": "puneet@company.ai",
< "privilege_model_version": "1.0",
< "region": "us-east-2",
< "securable_kind": "METASTORE_STANDARD",
< "securable_type": "METASTORE",
< "storage_root": "s3://prod-mojo-mybucket/metastore/bdf99cfb-5e7d-47e9-ba0c-631f149c8f... (2 more bytes)",
< "updated_at": 1684212848959,
< "updated_by": "puneet@company.ai"
< }: timestamp=2023-05-17T00:38:40.117Z
2023-05-17T00:38:40.161Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Suppressing diff for delta_sharing_scope: platform="INTERNAL" config="": timestamp=2023-05-17T00:38:40.161Z
2023-05-17T00:38:40.162Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Ignoring configuration drift from s3://prod-mojo-mybucket/metastore/bdf99cfb-5e7d-47e9-ba0c-631f149c8fc9 to s3://prod-mojo-mybucket/metastore: timestamp=2023-05-17T00:38:40.161Z
2023-05-17T00:38:40.167Z [WARN] Provider "registry.terraform.io/databricks/databricks" produced an invalid plan for module.databricks_metastore.databricks_metastore.this, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .delta_sharing_recipient_token_lifetime_in_seconds: planned value cty.NumberIntVal(0) for a non-computed attribute
- .delta_sharing_scope: planned value cty.StringVal("INTERNAL") for a non-computed attribute
2023-05-17T00:38:40.172Z [INFO] ReferenceTransformer: reference not found: "var.workspace_id"
2023-05-17T00:38:40.172Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.databricks_metastore_assignment.this" references: []
2023-05-17T00:38:40.300Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: GET /api/2.1/unity-catalog/metastore_summary
< HTTP/2.0 200 OK
< {
< "cloud": "aws",
< "created_at": 1684212848959,
< "created_by": "puneet@company.ai",
< "delta_sharing_scope": "INTERNAL",
< "global_metastore_id": "aws:us-east-2:bdf99cfb-5e7d-47e9-ba0c-631f149c8fc9",
< "metastore_id": "bdf99cfb-5e7d-47e9-ba0c-631f149c8fc9",
< "name": "prod-mojo",
< "owner": "puneet@company.ai",
< "privilege_model_version": "1.0",
< "region": "us-east-2",
< "storage_root": "s3://prod-mojo-mybucket/metastore/bdf99cfb-5e7d-47e9-ba0c-631f149c8f... (2 more bytes)",
< "updated_at": 1684212848959,
< "updated_by": "puneet@company.ai"
< }: timestamp=2023-05-17T00:38:40.299Z
2023-05-17T00:38:40.318Z [WARN] Provider "registry.terraform.io/databricks/databricks" produced an invalid plan for module.databricks_metastore.databricks_metastore_assignment.this, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .default_catalog_name: planned value cty.StringVal("hive_metastore") for a non-computed attribute
2023-05-17T00:38:40.326Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2023-05-17T00:38:40.344Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 pid=434
2023-05-17T00:38:40.344Z [DEBUG] provider: plugin exited
2023-05-17T00:38:40.347Z [DEBUG] Building and walking plan graph for DestroyMode
2023-05-17T00:38:40.351Z [DEBUG] adding implicit provider configuration provider["terraform.io/builtin/terraform"], implied first by data.terraform_remote_state.aws_databricks_common
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "data.aws_region.current" (*terraform.NodePlanDestroyableResourceInstance) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore.this" (*terraform.NodePlanDestroyableResourceInstance) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.workspace" (*terraform.NodePlanDestroyableResourceInstance) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "aws_s3_bucket.dis_s3" (*terraform.NodePlanDestroyableResourceInstance) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.aws_databricks_common" (*terraform.NodePlanDestroyableResourceInstance) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore_assignment.this" (*terraform.NodePlanDestroyableResourceInstance) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.vpc" (*terraform.NodePlanDestroyableResourceInstance) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:40.352Z [DEBUG] ProviderTransformer: "aws_route53_zone.primary" (*terraform.NodePlanDestroyableResourceInstance) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:40.357Z [DEBUG] Removing "data.terraform_remote_state.vpc", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "aws_route53_zone.primary", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "data.aws_region.current", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.task_execution_role (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "output.ecs_sd_namespace_id (destroy)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "output.ingestion_bucket (destroy)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "output.fargate_cluster_name (destroy)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.sd_namespace_id (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.autoloader.module.cluster.output.cluster_id (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.autoloader.module.cluster.output.inst_pool_cluster_id (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "provider[\"registry.terraform.io/hashicorp/aws\"]", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "provider[\"terraform.io/builtin/terraform\"]", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.hosted_zone (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.cluster_id (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.cluster_arn (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.cluster_name (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "output.ecs_log_group (destroy)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "output.fargate_image_pull_secret_dockerhub (destroy)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.image_pull_secret_dockerhub (expand)", filtered by targeting.
2023-05-17T00:38:40.357Z [DEBUG] Removing "module.fargate-cluster.output.image_pull_secret_gitlab (expand)", filtered by targeting.
2023-05-17T00:38:40.358Z [DEBUG] Removing "output.fargate_cluster_id (destroy)", filtered by targeting.
2023-05-17T00:38:40.358Z [DEBUG] Removing "output.fargate_task_execution_role (destroy)", filtered by targeting.
2023-05-17T00:38:40.358Z [DEBUG] Removing "output.fargate_image_pull_secret_gitlab (destroy)", filtered by targeting.
2023-05-17T00:38:40.358Z [DEBUG] Removing "data.terraform_remote_state.workspace", filtered by targeting.
2023-05-17T00:38:40.358Z [DEBUG] Removing "aws_s3_bucket.dis_s3", filtered by targeting.
2023-05-17T00:38:40.358Z [DEBUG] Removing "data.terraform_remote_state.aws_databricks_common", filtered by targeting.
2023-05-17T00:38:40.358Z [DEBUG] Removing "module.fargate-cluster.output.cloudwatch_group_name (expand)", filtered by targeting.
2023-05-17T00:38:40.359Z [DEBUG] Starting graph walk: walkPlanDestroy
2023-05-17T00:38:40.360Z [DEBUG] created provider logger: level=debug
2023-05-17T00:38:40.360Z [INFO] provider: configuring client automatic mTLS
2023-05-17T00:38:40.673Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 args=[.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0]
2023-05-17T00:38:40.689Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 pid=446
2023-05-17T00:38:40.689Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0
2023-05-17T00:38:40.922Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Databricks Terraform Provider
2023-05-17T00:38:40.922Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:40.922Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Version 1.15.0
2023-05-17T00:38:40.922Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:40.922Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: https://registry.terraform.io/providers/databricks/databricks/latest/docs
2023-05-17T00:38:40.922Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:40.930Z [INFO] provider.terraform-provider-databricks_v1.15.0: configuring server automatic mTLS: timestamp=2023-05-17T00:38:40.926Z
2023-05-17T00:38:41.359Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: plugin address: address=/tmp/plugin817779505 network=unix timestamp=2023-05-17T00:38:41.359Z
2023-05-17T00:38:41.359Z [DEBUG] provider: using plugin: version=5
2023-05-17T00:38:41.803Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2023-05-17T00:38:41.816Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 pid=446
2023-05-17T00:38:41.816Z [DEBUG] provider: plugin exited
2023-05-17T00:38:41.823Z [DEBUG] building apply graph to check for errors
2023-05-17T00:38:41.861Z [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/time"], implied first by module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand)
2023-05-17T00:38:41.862Z [DEBUG] adding implicit provider configuration provider["terraform.io/builtin/terraform"], implied first by data.terraform_remote_state.workspace (expand)
2023-05-17T00:38:41.866Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_sql_endpoint.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.866Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_ecs_cluster_capacity_providers.ecs_cluster_capacity_providers (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.867Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_policy.bucket-policy (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.867Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.app (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.867Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.867Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.867Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.metrics (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.868Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_role.role_for_s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.868Z [DEBUG] ProviderTransformer: "data.aws_region.current (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.868Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.868Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_policy.pass_role_for_s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.868Z [DEBUG] ProviderTransformer: "module.fargate-cluster.data.aws_iam_policy_document.secrets_policy (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.869Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.aws_databricks_common (expand)" (*terraform.nodeExpandApplyableResource) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:41.869Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_library.app (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.869Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.databricks_spark_version.spark_version (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.869Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.secret_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.869Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.workspace_account (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.869Z [DEBUG] ProviderTransformer: "aws_route53_zone.primary (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.869Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.vpc (expand)" (*terraform.nodeExpandApplyableResource) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:41.870Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.logs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.870Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.assume_role_for_ec2 (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.870Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.config (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.870Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_policy.secrets_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.870Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_instance_pool.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.870Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.871Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.app_properties (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.871Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_instance_profile.shared (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.871Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.871Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore.this (destroy)" (*terraform.NodeDestroyResourceInstance) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.871Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_job.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.871Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_role.task_execution (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.871Z [DEBUG] ProviderTransformer: "module.fargate-cluster.data.aws_iam_policy_document.assume_role_policy (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.872Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/time"]
2023-05-17T00:38:41.872Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.872Z [DEBUG] ProviderTransformer: "aws_s3_bucket.dis_s3 (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.872Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore_assignment.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.872Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore_assignment.this (destroy)" (*terraform.NodeDestroyResourceInstance) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.872Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.pass_role_for_s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.872Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.873Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.dockerhub (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.873Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.workspace (expand)" (*terraform.nodeExpandApplyableResource) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:41.873Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_cluster.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.873Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.databricks_node_type.node (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.873Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.docker (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.873Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_policy.logs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.873Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.874Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:41.874Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.874Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_instance_profile.shared (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:41.886Z [ERROR] AttachSchemaTransformer: No provider config schema available for provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:41.899Z [DEBUG] ReferenceTransformer: "local.fargate_cluster_name (expand)" references: [local.env (expand) data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.899Z [DEBUG] ReferenceTransformer: "module.databricks_metastore (expand)" references: []
2023-05-17T00:38:41.899Z [DEBUG] ReferenceTransformer: "module.fargate-cluster (close)" references: []
2023-05-17T00:38:41.899Z [DEBUG] ReferenceTransformer: "data.aws_region.current (expand)" references: []
2023-05-17T00:38:41.899Z [DEBUG] ReferenceTransformer: "module.autoloader.var.cluster_name (expand)" references: [local.databricks_cluster_name (expand)]
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.output.cluster_id (expand)" references: [module.autoloader.module.cluster.databricks_cluster.this (expand)]
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cluster_arn (expand)" references: [module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)]
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "output.ingestion_bucket" references: [aws_s3_bucket.dis_s3 (expand)]
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)" references: [module.autoloader.module.cluster.var.buckets_to_create (expand)]
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.min_nodes (expand)" references: [module.autoloader.var.min_nodes (expand)]
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_instance_profile.shared (expand)" references: []
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "var.source_buckets" references: []
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.buckets_existing (expand)" references: [module.autoloader.var.source_buckets (expand)]
2023-05-17T00:38:41.900Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.log_retention_days (expand)" references: []
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.fargate_spot_weight (expand)" references: [var.fargate_cluster]
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster (close)" references: []
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_instance_pool.this (expand)" references: [module.autoloader.module.cluster.var.create_instance_pool (expand)]
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand)" references: [module.autoloader.module.cluster.var.create_instance_pool (expand)]
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "data.terraform_remote_state.vpc (expand)" references: []
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_policy.logs (expand)" references: []
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.memory (expand)" references: [module.autoloader.var.memory (expand)]
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "local.databricks_ws_token (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.all_arns (expand)" references: [module.autoloader.module.cluster.local.bucket_arns (expand) module.autoloader.module.cluster.local.existing_bucket_arns (expand)]
2023-05-17T00:38:41.901Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.s3_access (expand)" references: []
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.logs (expand)" references: []
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "aws_route53_zone.primary (expand)" references: []
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.data.aws_iam_policy_document.assume_role_policy (expand)" references: []
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.autoloader.var.cross_account_role_name (expand)" references: [local.cross_account_role_name (expand)]
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.var.bucket (expand)" references: [local.root_bucket (expand)]
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "local.env (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "local.org_name (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.all_arns_wc (expand)" references: [module.autoloader.module.cluster.local.bucket_arns_wc (expand) module.autoloader.module.cluster.local.existing_bucket_arns_wc (expand)]
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_policy.pass_role_for_s3_access (expand)" references: []
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_instance_profile.shared (expand)" references: []
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cloudwatch_group_name (expand)" references: [module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)]
2023-05-17T00:38:41.902Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.cluster_name (expand)" references: [module.autoloader.var.cluster_name (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "module.autoloader.var.sql_enable_serverless (expand)" references: []
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "local.connect_vars (expand)" references: [local.default_kafka_connect_vars (expand) var.kafka_connect]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "local.svc_subnets (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "module.autoloader (expand)" references: []
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "data.terraform_remote_state.aws_databricks_common (expand)" references: []
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.data.aws_iam_policy_document.secrets_policy (expand)" references: []
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.buckets_to_create (expand)" references: [module.autoloader.var.output_bucket (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "module.autoloader.var.output_bucket (expand)" references: [local.output_bucket (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "local.connect_cluster_name (expand)" references: [local.env (expand) data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "local.output_bucket (expand)" references: [local.databricks_cluster_name (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "local.checkpoint_bucket (expand)" references: [local.databricks_cluster_name (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "local.root_bucket (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_library.app (expand)" references: []
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "var.fargate_cluster" references: []
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "output.fargate_cluster_name" references: [local.fargate_cluster_name (expand)]
2023-05-17T00:38:41.903Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.availability (expand)" references: [module.autoloader.var.availability (expand)]
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "local.cidr_block (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "local.workspace_tags (expand)" references: [local.databricks_ws_name (expand)]
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.existing_bucket_arns (expand)" references: [module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)]
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.image_pull_secret_dockerhub (expand)" references: [module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)]
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/databricks/databricks\"]" references: [local.databricks_ws_token (expand) local.databricks_ws_url (expand)]
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "var.kafka" references: []
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.spot_bid_price_percent (expand)" references: []
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/time\"]" references: []
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)" references: []
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.gpu_count (expand)" references: []
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.pass_role_for_s3_access (expand)" references: []
2023-05-17T00:38:41.904Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "output.ecs_log_group" references: [module.fargate-cluster.output.cloudwatch_group_name (expand)]
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "local.databricks_ws_name (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cluster_id (expand)" references: [module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)]
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "var.client_sasl_iam_enabled" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "module.autoloader.var.spark_version (expand)" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "module.autoloader.var.max_retries (expand)" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "local.databricks_cluster_name (expand)" references: [local.env (expand) local.org_name (expand)]
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand)" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.assume_role_for_ec2 (expand)" references: []
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "local.databricks_ws_id (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:41.905Z [DEBUG] ReferenceTransformer: "output.ecs_sd_namespace_id" references: [module.fargate-cluster.output.sd_namespace_id (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.cluster_name (expand)" references: [local.fargate_cluster_name (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "local.databricks_ws_url (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)" references: [module.autoloader.module.cluster.var.buckets_existing (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.create_instance_pool (expand)" references: []
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.databricks_pool_name (expand)" references: []
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.var.cores (expand)" references: []
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.var.bucket_region (expand)" references: [data.aws_region.current (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.var.max_nodes (expand)" references: []
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.bucket_arns (expand)" references: [module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.sd_namespace_id (expand)" references: [module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "aws_s3_bucket.dis_s3 (expand)" references: []
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.autotermination_minutes (expand)" references: []
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "output.fargate_task_execution_role" references: [module.fargate-cluster.output.task_execution_role (expand)]
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.databricks_metastore (close)" references: []
2023-05-17T00:38:41.906Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.var.databricks_workspace_name (expand)" references: [local.databricks_ws_name (expand)]
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.dockerhub_repo (expand)" references: []
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "local.default_kafka_connect_vars (expand)" references: []
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.tags (expand)" references: [module.autoloader.module.cluster.var.tags (expand) module.autoloader.module.cluster.var.org_name (expand) module.autoloader.module.cluster.var.cluster_name (expand)]
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cluster_name (expand)" references: [module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)]
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)" references: []
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.var.workspace_id (expand)" references: [local.databricks_ws_id (expand)]
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.autoloader.var.source_buckets (expand)" references: [var.source_buckets aws_s3_bucket.dis_s3 (expand)]
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "local.cloudwatch_log_group (expand)" references: [local.kafka_cluster_name (expand)]
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "local.kafka_vars (expand)" references: [local.default_kafka_vars (expand) var.kafka]
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.config (expand)" references: []
2023-05-17T00:38:41.907Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_cluster.this (expand)" references: [module.autoloader.module.cluster.var.create_instance_pool (expand)]
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "local.kafka_subnet_ids (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.databricks_spark_version.spark_version (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "var.env" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.bucket_arns_wc (expand)" references: [module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)]
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.metrics (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.cross_account_role_name (expand)" references: [module.autoloader.var.cross_account_role_name (expand)]
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.local_disk (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.app_properties (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.secret_access (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.var.availability (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "local.lb_subnets (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.image_pull_secret_gitlab (expand)" references: [module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)]
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "provider[\"terraform.io/builtin/terraform\"]" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_role.role_for_s3_access (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "var.org_name" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.tags (expand)" references: [module.autoloader.var.spark_version (expand)]
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "local.default_kafka_vars (expand)" references: []
2023-05-17T00:38:41.908Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.databricks_metastore.this (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_policy.secrets_access (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.dockerhub (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_ecs_cluster_capacity_providers.ecs_cluster_capacity_providers (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.maven_coordinates (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.vpc_id (expand)" references: [local.vpc_id (expand)]
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "output.fargate_image_pull_secret_dockerhub" references: [module.fargate-cluster.output.image_pull_secret_dockerhub (expand)]
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_sql_endpoint.this (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.databricks_metastore_assignment.this (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "var.kafka_connect" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.cores (expand)" references: [module.autoloader.var.cores (expand)]
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.var.input_s3path_prefix (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.var.databricks_job_name (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.var.org_name (expand)" references: [local.org_name (expand)]
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.docker_repo_username (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_policy.bucket-policy (expand)" references: []
2023-05-17T00:38:41.909Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.databricks_node_type.node (expand)" references: []
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "output.fargate_cluster_id" references: [module.fargate-cluster.output.cluster_id (expand)]
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.autoloader.var.memory (expand)" references: []
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.docker_repo_password (expand)" references: []
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.tags (expand)" references: [local.common_tags (expand)]
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "local.kafka_cluster_name (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster (expand)" references: []
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.spark_conf (expand)" references: []
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.autoloader.var.tags (expand)" references: [local.common_tags (expand) local.workspace_tags (expand)]
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "local.vpc_id (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "local.common_tags (expand)" references: [local.org_name (expand) local.env (expand)]
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/aws\"]" references: []
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.autoloader (close)" references: []
2023-05-17T00:38:41.910Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.spark_version (expand)" references: [module.autoloader.var.spark_version (expand)]
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.fargate_weight (expand)" references: [var.fargate_cluster]
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.app (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.max_nodes (expand)" references: [module.autoloader.var.max_nodes (expand)]
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.autoloader.var.min_nodes (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "local.cross_account_role_name (expand)" references: [data.terraform_remote_state.aws_databricks_common (expand)]
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.task_execution_role (expand)" references: [module.fargate-cluster.aws_iam_role.task_execution (expand) module.fargate-cluster.aws_iam_role.task_execution (expand)]
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "data.terraform_remote_state.workspace (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_job.this (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.fargate-cluster (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.autoloader.var.databricks_ws_name (expand)" references: [local.databricks_ws_name (expand)]
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.autoloader.var.sql_cluster_size (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.gitlab_docker_repo (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "output.fargate_image_pull_secret_gitlab" references: [module.fargate-cluster.output.image_pull_secret_gitlab (expand)]
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_role.task_execution (expand)" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "var.webhook" references: []
2023-05-17T00:38:41.911Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.output.inst_pool_cluster_id (expand)" references: [module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand)]
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.hosted_zone (expand)" references: [module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)]
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.docker (expand)" references: []
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.env (expand)" references: [var.env]
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.containerInsights (expand)" references: []
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.all_buckets (expand)" references: [module.autoloader.module.cluster.var.buckets_to_create (expand) module.autoloader.module.cluster.var.buckets_existing (expand)]
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.workspace_account (expand)" references: []
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.org_name (expand)" references: [module.autoloader.var.org_name (expand)]
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.fargate_base (expand)" references: [var.fargate_cluster]
2023-05-17T00:38:41.912Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.existing_bucket_arns_wc (expand)" references: [module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)]
2023-05-17T00:38:41.939Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.output.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:41.939Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.var.workspace_id (expand) is no longer needed, removing
2023-05-17T00:38:41.939Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.var.databricks_workspace_name (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.dockerhub_repo (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.tags (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.config (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: local.cloudwatch_log_group (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: local.kafka_vars (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.databricks_spark_version.spark_version (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: local.kafka_subnet_ids (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.metrics (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.cross_account_role_name (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.app_properties (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_role_policy_attachment.secret_access (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.local_disk (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_role.role_for_s3_access (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: local.lb_subnets (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.databricks_metastore.this (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_policy.secrets_access (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.tags (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: local.default_kafka_vars (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_sql_endpoint.this (expand) is no longer needed, removing
2023-05-17T00:38:41.940Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.databricks_metastore_assignment.this (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_secretsmanager_secret_version.dockerhub (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_ecs_cluster_capacity_providers.ecs_cluster_capacity_providers (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.maven_coordinates (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.vpc_id (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.docker_repo_username (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_policy.bucket-policy (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.databricks_node_type.node (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.cores (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.var.input_s3path_prefix (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.var.databricks_job_name (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.docker_repo_password (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.spark_conf (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.var.tags (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.tags (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: local.kafka_cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.spark_version (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.fargate_weight (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: local.vpc_id (expand) is no longer needed, removing
2023-05-17T00:38:41.941Z [DEBUG] pruneUnusedNodes: local.common_tags (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_job.this (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.app (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.max_nodes (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.var.databricks_ws_name (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.var.sql_cluster_size (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.gitlab_docker_repo (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_secretsmanager_secret_version.docker (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.env (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.output.inst_pool_cluster_id (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.output.hosted_zone (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_role_policy_attachment.workspace_account (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.org_name (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.containerInsights (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.all_buckets (expand) is no longer needed, removing
2023-05-17T00:38:41.942Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.fargate_base (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.min_nodes (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.output.cluster_id (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.output.cluster_arn (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.fargate_spot_weight (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_instance_pool.this (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_instance_profile.shared (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.log_retention_days (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_policy.logs (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_role_policy_attachment.s3_access (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_role_policy_attachment.logs (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.memory (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.var.max_nodes (expand) is no longer needed, removing
2023-05-17T00:38:41.943Z [DEBUG] pruneUnusedNodes: module.autoloader.var.bucket_region (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.var.cores (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.databricks_pool_name (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_cluster.this (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.autotermination_minutes (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.create_instance_pool (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: local.databricks_ws_id (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.var.spark_version (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.aws_iam_policy_document.assume_role_for_ec2 (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.var.max_retries (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.var.org_name (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.gpu_count (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.aws_iam_policy_document.pass_role_for_s3_access (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: provider["registry.terraform.io/hashicorp/time"] is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: module.autoloader.var.memory (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: local.workspace_tags (expand) is no longer needed, removing
2023-05-17T00:38:41.944Z [DEBUG] pruneUnusedNodes: local.cidr_block (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.availability (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.spot_bid_price_percent (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: local.connect_cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.var.min_nodes (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_library.app (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: local.checkpoint_bucket (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: local.svc_subnets (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: local.connect_vars (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.var.sql_enable_serverless (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.all_arns (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: local.default_kafka_connect_vars (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: aws_route53_zone.primary (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.bucket_arns (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.all_arns_wc (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_policy.pass_role_for_s3_access (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.var.availability (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_instance_profile.shared (expand) is no longer needed, removing
2023-05-17T00:38:41.945Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.data.aws_iam_policy_document.assume_role_policy (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: local.databricks_ws_name (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.var.cross_account_role_name (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.var.bucket (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.existing_bucket_arns (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.databricks_metastore (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: data.aws_region.current (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.data.aws_iam_policy_document.secrets_policy (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.bucket_arns_wc (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: local.root_bucket (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: local.cross_account_role_name (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.existing_bucket_arns_wc (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.buckets_to_create (expand) is no longer needed, removing
2023-05-17T00:38:41.946Z [DEBUG] pruneUnusedNodes: module.autoloader.var.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: module.autoloader.var.output_bucket (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: data.terraform_remote_state.aws_databricks_common (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: local.output_bucket (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.buckets_existing (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: local.databricks_cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: local.org_name (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: module.autoloader.var.source_buckets (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: aws_s3_bucket.dis_s3 (expand) is no longer needed, removing
2023-05-17T00:38:41.947Z [DEBUG] pruneUnusedNodes: module.autoloader (expand) is no longer needed, removing
2023-05-17T00:38:41.949Z [DEBUG] Removing "module.fargate-cluster.output.task_execution_role (expand)", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "module.fargate-cluster (expand)", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "module.fargate-cluster.aws_iam_role.task_execution (expand)", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "var.webhook", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "output.fargate_image_pull_secret_gitlab", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "local.fargate_cluster_name (expand)", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "module.fargate-cluster (close)", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "output.ingestion_bucket", filtered by targeting.
2023-05-17T00:38:41.949Z [DEBUG] Removing "module.autoloader.module.cluster (close)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "var.source_buckets", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "data.terraform_remote_state.vpc (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "local.env (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.output.cloudwatch_group_name (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "var.fargate_cluster", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "output.fargate_cluster_name", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "var.kafka", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.output.image_pull_secret_dockerhub (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.output.cluster_id (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "output.ecs_log_group", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "var.client_sasl_iam_enabled", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "output.ecs_sd_namespace_id", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.output.sd_namespace_id (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "output.fargate_task_execution_role", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.databricks_metastore (close)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "var.env", filtered by targeting.
2023-05-17T00:38:41.950Z [DEBUG] Removing "var.org_name", filtered by targeting.
2023-05-17T00:38:41.951Z [DEBUG] Removing "module.fargate-cluster.output.image_pull_secret_gitlab (expand)", filtered by targeting.
2023-05-17T00:38:41.951Z [DEBUG] Removing "output.fargate_image_pull_secret_dockerhub", filtered by targeting.
2023-05-17T00:38:41.951Z [DEBUG] Removing "var.kafka_connect", filtered by targeting.
2023-05-17T00:38:41.951Z [DEBUG] Removing "output.fargate_cluster_id", filtered by targeting.
2023-05-17T00:38:41.951Z [DEBUG] Removing "provider[\"registry.terraform.io/hashicorp/aws\"]", filtered by targeting.
2023-05-17T00:38:41.951Z [DEBUG] Removing "module.autoloader (close)", filtered by targeting.
2023-05-17T00:38:41.974Z [DEBUG] command: asking for input: "\nDo you really want to destroy all resources?"
yes
2023-05-17T00:38:49.274Z [INFO] backend/local: apply calling Apply
2023-05-17T00:38:49.275Z [DEBUG] Building and walking apply graph for DestroyMode plan
2023-05-17T00:38:49.312Z [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/time"], implied first by module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand)
2023-05-17T00:38:49.312Z [DEBUG] adding implicit provider configuration provider["terraform.io/builtin/terraform"], implied first by data.terraform_remote_state.aws_databricks_common (expand)
2023-05-17T00:38:49.318Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.318Z [DEBUG] ProviderTransformer: "module.fargate-cluster.data.aws_iam_policy_document.secrets_policy (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.318Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_policy.pass_role_for_s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.318Z [DEBUG] ProviderTransformer: "data.aws_region.current (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.318Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.secret_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.318Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_sql_endpoint.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.318Z [DEBUG] ProviderTransformer: "aws_route53_zone.primary (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.319Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.workspace (expand)" (*terraform.nodeExpandApplyableResource) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:49.319Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.app (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.319Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_role.role_for_s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.320Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_policy.bucket-policy (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.320Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.databricks_node_type.node (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.320Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.config (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.320Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore.this (destroy)" (*terraform.NodeDestroyResourceInstance) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.321Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.metrics (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.321Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.aws_databricks_common (expand)" (*terraform.nodeExpandApplyableResource) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:49.321Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.321Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.321Z [DEBUG] ProviderTransformer: "aws_s3_bucket.dis_s3 (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.321Z [DEBUG] ProviderTransformer: "module.fargate-cluster.data.aws_iam_policy_document.assume_role_policy (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.322Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.pass_role_for_s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.322Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_policy.secrets_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.322Z [DEBUG] ProviderTransformer: "data.terraform_remote_state.vpc (expand)" (*terraform.nodeExpandApplyableResource) needs provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:49.322Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore_assignment.this (destroy)" (*terraform.NodeDestroyResourceInstance) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.322Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_job.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.322Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.323Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.323Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/time"]
2023-05-17T00:38:49.323Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.databricks_spark_version.spark_version (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.323Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_instance_profile.shared (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.324Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.324Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_dbfs_file.app_properties (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.324Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.s3_access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.324Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.324Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.dockerhub (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.324Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_cluster.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.325Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.logs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.325Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.325Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_role.task_execution (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.325Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_iam_policy.logs (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.325Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.databricks_instance_pool.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.325Z [DEBUG] ProviderTransformer: "module.autoloader.databricks_library.app (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.325Z [DEBUG] ProviderTransformer: "module.databricks_metastore.databricks_metastore_assignment.this (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/databricks/databricks"]
2023-05-17T00:38:49.326Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.assume_role_for_ec2 (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.326Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_ecs_cluster_capacity_providers.ecs_cluster_capacity_providers (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.326Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.docker (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.326Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_instance_profile.shared (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.326Z [DEBUG] ProviderTransformer: "module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.326Z [DEBUG] ProviderTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.workspace_account (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-05-17T00:38:49.339Z [ERROR] AttachSchemaTransformer: No provider config schema available for provider["terraform.io/builtin/terraform"]
2023-05-17T00:38:49.352Z [DEBUG] ReferenceTransformer: "module.autoloader.var.memory (expand)" references: []
2023-05-17T00:38:49.352Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)" references: []
2023-05-17T00:38:49.352Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)" references: []
2023-05-17T00:38:49.352Z [DEBUG] ReferenceTransformer: "var.client_sasl_iam_enabled" references: []
2023-05-17T00:38:49.352Z [DEBUG] ReferenceTransformer: "var.kafka" references: []
2023-05-17T00:38:49.352Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.org_name (expand)" references: [module.autoloader.var.org_name (expand)]
2023-05-17T00:38:49.352Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.availability (expand)" references: [module.autoloader.var.availability (expand)]
2023-05-17T00:38:49.353Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.task_execution_role (expand)" references: [module.fargate-cluster.aws_iam_role.task_execution (expand) module.fargate-cluster.aws_iam_role.task_execution (expand)]
2023-05-17T00:38:49.353Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/databricks/databricks\"]" references: [local.databricks_ws_url (expand) local.databricks_ws_token (expand)]
2023-05-17T00:38:49.353Z [DEBUG] ReferenceTransformer: "aws_route53_zone.primary (expand)" references: []
2023-05-17T00:38:49.353Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_instance_pool.this (expand)" references: [module.autoloader.module.cluster.var.create_instance_pool (expand)]
2023-05-17T00:38:49.353Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.spark_version (expand)" references: [module.autoloader.var.spark_version (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.spark_conf (expand)" references: []
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.cluster_name (expand)" references: [module.autoloader.var.cluster_name (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.var.availability (expand)" references: []
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "local.cloudwatch_log_group (expand)" references: [local.kafka_cluster_name (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.tags (expand)" references: [module.autoloader.module.cluster.var.tags (expand) module.autoloader.module.cluster.var.org_name (expand) module.autoloader.module.cluster.var.cluster_name (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "var.kafka_connect" references: []
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.max_nodes (expand)" references: [module.autoloader.var.max_nodes (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.buckets_existing (expand)" references: [module.autoloader.var.source_buckets (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.var.databricks_ws_name (expand)" references: [local.databricks_ws_name (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader.var.cross_account_role_name (expand)" references: [local.cross_account_role_name (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.docker_repo_username (expand)" references: []
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "output.fargate_image_pull_secret_dockerhub" references: [module.fargate-cluster.output.image_pull_secret_dockerhub (expand)]
2023-05-17T00:38:49.354Z [DEBUG] ReferenceTransformer: "module.autoloader (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.app_properties (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.databricks_metastore.this (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.databricks_pool_name (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.autoloader.var.source_buckets (expand)" references: [var.source_buckets aws_s3_bucket.dis_s3 (expand)]
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "local.common_tags (expand)" references: [local.org_name (expand) local.env (expand)]
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cluster_arn (expand)" references: [module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)]
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.var.workspace_id (expand)" references: [local.databricks_ws_id (expand)]
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.docker_repo_password (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "local.checkpoint_bucket (expand)" references: [local.databricks_cluster_name (expand)]
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.fargate-cluster (close)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_role.task_execution (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.config (expand)" references: []
2023-05-17T00:38:49.355Z [DEBUG] ReferenceTransformer: "local.databricks_ws_name (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.image_pull_secret_dockerhub (expand)" references: [module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.databricks_metastore (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.var.output_bucket (expand)" references: [local.output_bucket (expand)]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.var.databricks_job_name (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.data.aws_iam_policy_document.assume_role_policy (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.workspace_account (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_role_policy_attachment.s3_access (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.assume_role_for_ec2 (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.spot_bid_price_percent (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.tags (expand)" references: [module.autoloader.var.spark_version (expand)]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.vpc_id (expand)" references: [local.vpc_id (expand)]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "local.env (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "output.fargate_cluster_name" references: [local.fargate_cluster_name (expand)]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "aws_s3_bucket.dis_s3 (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_library.app (expand)" references: []
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.env (expand)" references: [var.env]
2023-05-17T00:38:49.356Z [DEBUG] ReferenceTransformer: "local.databricks_ws_token (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/time\"]" references: []
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand)" references: [module.autoloader.module.cluster.var.create_instance_pool (expand)]
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.aws_iam_policy_document.pass_role_for_s3_access (expand)" references: []
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.databricks_metastore_assignment.this (expand)" references: []
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "var.org_name" references: []
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.cores (expand)" references: [module.autoloader.var.cores (expand)]
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "local.cidr_block (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "local.fargate_cluster_name (expand)" references: [local.env (expand) data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "local.kafka_vars (expand)" references: [local.default_kafka_vars (expand) var.kafka]
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "var.webhook" references: []
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.min_nodes (expand)" references: [module.autoloader.var.min_nodes (expand)]
2023-05-17T00:38:49.357Z [DEBUG] ReferenceTransformer: "output.ecs_sd_namespace_id" references: [module.fargate-cluster.output.sd_namespace_id (expand)]
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.autoloader (close)" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.create_instance_pool (expand)" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.gitlab_docker_repo (expand)" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.dockerhub_repo (expand)" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "local.output_bucket (expand)" references: [local.databricks_cluster_name (expand)]
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_policy.secrets_access (expand)" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.bucket_arns (expand)" references: [module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)]
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand)" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)" references: [module.autoloader.module.cluster.var.buckets_existing (expand)]
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "var.env" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "local.default_kafka_vars (expand)" references: []
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "local.cross_account_role_name (expand)" references: [data.terraform_remote_state.aws_databricks_common (expand)]
2023-05-17T00:38:49.358Z [DEBUG] ReferenceTransformer: "output.fargate_cluster_id" references: [module.fargate-cluster.output.cluster_id (expand)]
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.maven_coordinates (expand)" references: []
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.autoloader.var.org_name (expand)" references: [local.org_name (expand)]
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.autoloader.var.max_retries (expand)" references: []
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "local.lb_subnets (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "local.connect_vars (expand)" references: [local.default_kafka_connect_vars (expand) var.kafka_connect]
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.databricks_metastore (close)" references: []
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_policy.logs (expand)" references: []
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "var.fargate_cluster" references: []
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.autoloader.var.sql_cluster_size (expand)" references: []
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.sd_namespace_id (expand)" references: [module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)]
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.output.inst_pool_cluster_id (expand)" references: [module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand)]
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_ecs_cluster_capacity_providers.ecs_cluster_capacity_providers (expand)" references: []
2023-05-17T00:38:49.359Z [DEBUG] ReferenceTransformer: "local.kafka_subnet_ids (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "local.default_kafka_connect_vars (expand)" references: []
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.existing_bucket_arns (expand)" references: [module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/aws\"]" references: []
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.data.aws_iam_policy_document.secrets_policy (expand)" references: []
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.metrics (expand)" references: []
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.databricks_spark_version.spark_version (expand)" references: []
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.memory (expand)" references: [module.autoloader.var.memory (expand)]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "local.vpc_id (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "local.kafka_cluster_name (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "data.terraform_remote_state.aws_databricks_common (expand)" references: []
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.containerInsights (expand)" references: []
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.fargate_spot_weight (expand)" references: [var.fargate_cluster]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "output.fargate_task_execution_role" references: [module.fargate-cluster.output.task_execution_role (expand)]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "module.autoloader.var.bucket_region (expand)" references: [data.aws_region.current (expand)]
2023-05-17T00:38:49.360Z [DEBUG] ReferenceTransformer: "output.fargate_image_pull_secret_gitlab" references: [module.fargate-cluster.output.image_pull_secret_gitlab (expand)]
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "output.ecs_log_group" references: [module.fargate-cluster.output.cloudwatch_group_name (expand)]
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.all_buckets (expand)" references: [module.autoloader.module.cluster.var.buckets_to_create (expand) module.autoloader.module.cluster.var.buckets_existing (expand)]
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.image_pull_secret_gitlab (expand)" references: [module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)]
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "data.terraform_remote_state.vpc (expand)" references: []
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.dockerhub (expand)" references: []
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_cluster.this (expand)" references: [module.autoloader.module.cluster.var.create_instance_pool (expand)]
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.var.sql_enable_serverless (expand)" references: []
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.var.tags (expand)" references: [local.common_tags (expand) local.workspace_tags (expand)]
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "local.root_bucket (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster (expand)" references: []
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_policy.pass_role_for_s3_access (expand)" references: []
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.gpu_count (expand)" references: []
2023-05-17T00:38:49.361Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.cross_account_role_name (expand)" references: [module.autoloader.var.cross_account_role_name (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.fargate_weight (expand)" references: [var.fargate_cluster]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "local.workspace_tags (expand)" references: [local.databricks_ws_name (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "output.ingestion_bucket" references: [aws_s3_bucket.dis_s3 (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_secretsmanager_secret_version.docker (expand)" references: []
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.var.databricks_workspace_name (expand)" references: [local.databricks_ws_name (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster (close)" references: []
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.fargate-cluster (expand)" references: []
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "data.aws_region.current (expand)" references: []
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.autoloader.var.input_s3path_prefix (expand)" references: []
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "local.org_name (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "local.databricks_ws_id (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "local.databricks_ws_url (expand)" references: [data.terraform_remote_state.workspace (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cluster_name (expand)" references: [module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)]
2023-05-17T00:38:49.362Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.all_arns (expand)" references: [module.autoloader.module.cluster.local.bucket_arns (expand) module.autoloader.module.cluster.local.existing_bucket_arns (expand)]
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "data.terraform_remote_state.workspace (expand)" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_dbfs_file.app (expand)" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_role.role_for_s3_access (expand)" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_policy.bucket-policy (expand)" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.var.max_nodes (expand)" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.all_arns_wc (expand)" references: [module.autoloader.module.cluster.local.bucket_arns_wc (expand) module.autoloader.module.cluster.local.existing_bucket_arns_wc (expand)]
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "local.connect_cluster_name (expand)" references: [local.env (expand) data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.data.databricks_node_type.node (expand)" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "var.source_buckets" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.autotermination_minutes (expand)" references: []
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.buckets_to_create (expand)" references: [module.autoloader.var.output_bucket (expand)]
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.var.cluster_name (expand)" references: [local.databricks_cluster_name (expand)]
2023-05-17T00:38:49.363Z [DEBUG] ReferenceTransformer: "module.autoloader.var.min_nodes (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.databricks_metastore.var.bucket (expand)" references: [local.root_bucket (expand)]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.var.cores (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.var.spark_version (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "local.databricks_cluster_name (expand)" references: [local.env (expand) local.org_name (expand)]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.hosted_zone (expand)" references: [module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.logs (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.databricks_instance_profile.shared (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_iam_instance_profile.shared (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.var.local_disk (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.cluster_name (expand)" references: [local.fargate_cluster_name (expand)]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "local.svc_subnets (expand)" references: [data.terraform_remote_state.vpc (expand)]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_job.this (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.log_retention_days (expand)" references: []
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.fargate_base (expand)" references: [var.fargate_cluster]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.existing_bucket_arns_wc (expand)" references: [module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand)]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cluster_id (expand)" references: [module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)]
2023-05-17T00:38:49.364Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.output.cluster_id (expand)" references: [module.autoloader.module.cluster.databricks_cluster.this (expand)]
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.output.cloudwatch_group_name (expand)" references: [module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)]
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "provider[\"terraform.io/builtin/terraform\"]" references: []
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_iam_role_policy_attachment.secret_access (expand)" references: []
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)" references: []
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "module.autoloader.databricks_sql_endpoint.this (expand)" references: []
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)" references: [module.autoloader.module.cluster.var.buckets_to_create (expand)]
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "module.fargate-cluster.var.tags (expand)" references: [local.common_tags (expand)]
2023-05-17T00:38:49.365Z [DEBUG] ReferenceTransformer: "module.autoloader.module.cluster.local.bucket_arns_wc (expand)" references: [module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand)]
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.databricks_metastore.this (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.databricks_pool_name (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.output.cluster_arn (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.app_properties (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.var.workspace_id (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.docker_repo_password (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: local.checkpoint_bucket (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.config (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_role_policy_attachment.workspace_account (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_role_policy_attachment.s3_access (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.aws_iam_policy_document.assume_role_for_ec2 (expand) is no longer needed, removing
2023-05-17T00:38:49.392Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.spot_bid_price_percent (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.var.databricks_job_name (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.data.aws_iam_policy_document.assume_role_policy (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.vpc_id (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_library.app (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.env (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.aws_iam_policy_document.pass_role_for_s3_access (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.databricks_metastore_assignment.this (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.cores (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: local.cidr_block (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: local.kafka_vars (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.min_nodes (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.gitlab_docker_repo (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.dockerhub_repo (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_policy.secrets_access (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: local.default_kafka_vars (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.time_sleep.inst_prof_wait (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.var.max_retries (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: local.lb_subnets (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.maven_coordinates (expand) is no longer needed, removing
2023-05-17T00:38:49.393Z [DEBUG] pruneUnusedNodes: local.connect_vars (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.autoloader.var.sql_cluster_size (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.output.inst_pool_cluster_id (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_policy.logs (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: local.kafka_subnet_ids (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: local.default_kafka_connect_vars (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_ecs_cluster_capacity_providers.ecs_cluster_capacity_providers (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.metrics (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.databricks_spark_version.spark_version (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.memory (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: local.vpc_id (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.data.aws_iam_policy_document.secrets_policy (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.containerInsights (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.fargate_spot_weight (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.autoloader.var.bucket_region (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_secretsmanager_secret_version.dockerhub (expand) is no longer needed, removing
2023-05-17T00:38:49.394Z [DEBUG] pruneUnusedNodes: module.autoloader.var.sql_enable_serverless (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.var.tags (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.all_buckets (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.gpu_count (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.cross_account_role_name (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.fargate_weight (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: local.workspace_tags (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_policy.pass_role_for_s3_access (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.var.databricks_workspace_name (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_secretsmanager_secret_version.docker (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.var.input_s3path_prefix (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: local.databricks_ws_id (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.output.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: data.aws_region.current (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_dbfs_file.app (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_role.role_for_s3_access (expand) is no longer needed, removing
2023-05-17T00:38:49.395Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.tags (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: local.cloudwatch_log_group (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.docker_repo_username (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.var.cross_account_role_name (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.var.databricks_ws_name (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: local.databricks_ws_name (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.max_nodes (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: aws_route53_zone.primary (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.tags (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.spark_conf (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.spark_version (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_instance_pool.this (expand) is no longer needed, removing
2023-05-17T00:38:49.396Z [DEBUG] pruneUnusedNodes: provider["registry.terraform.io/hashicorp/time"] is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.var.memory (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.availability (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.org_name (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_cluster.inst_pool_cluster (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_role_policy_attachment.secret_access (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: local.cross_account_role_name (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.var.org_name (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.tags (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_sql_endpoint.this (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.databricks_job.this (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.output.cluster_id (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.fargate_base (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.log_retention_days (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.aws_iam_role_policy_attachment.logs (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.output.hosted_zone (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: local.svc_subnets (expand) is no longer needed, removing
2023-05-17T00:38:49.397Z [DEBUG] pruneUnusedNodes: module.fargate-cluster.var.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: local.kafka_cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.local_disk (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: local.common_tags (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_policy.bucket-policy (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.var.max_nodes (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.all_arns_wc (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.all_arns (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.var.availability (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.autotermination_minutes (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.var.cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: module.autoloader.var.min_nodes (expand) is no longer needed, removing
2023-05-17T00:38:49.398Z [DEBUG] pruneUnusedNodes: local.connect_cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.bucket_arns (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.bucket_arns_wc (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.databricks_node_type.node (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_s3_bucket.storage_buckets (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.databricks_metastore.var.bucket (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.aws_iam_instance_profile.shared (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.existing_bucket_arns (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_instance_profile.shared (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.databricks_metastore (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: data.terraform_remote_state.aws_databricks_common (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.buckets_to_create (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.var.cores (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.local.existing_bucket_arns_wc (expand) is no longer needed, removing
2023-05-17T00:38:49.399Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.data.aws_s3_bucket.existing (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.buckets_existing (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader.var.spark_version (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader.var.output_bucket (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: local.root_bucket (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.databricks_cluster.this (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: local.output_bucket (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster.var.create_instance_pool (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader.var.source_buckets (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: aws_s3_bucket.dis_s3 (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: local.databricks_cluster_name (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader.module.cluster (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: module.autoloader (expand) is no longer needed, removing
2023-05-17T00:38:49.400Z [DEBUG] pruneUnusedNodes: local.org_name (expand) is no longer needed, removing
2023-05-17T00:38:49.402Z [DEBUG] Removing "module.fargate-cluster.aws_service_discovery_private_dns_namespace.ecs (expand)", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "module.fargate-cluster.aws_secretsmanager_secret.dockerhub (expand)", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "var.client_sasl_iam_enabled", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "var.kafka", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "module.fargate-cluster.output.task_execution_role (expand)", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "var.kafka_connect", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "output.fargate_image_pull_secret_dockerhub", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "module.fargate-cluster.aws_ecs_cluster.ecs_cluster (expand)", filtered by targeting.
2023-05-17T00:38:49.402Z [DEBUG] Removing "module.fargate-cluster (close)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "module.fargate-cluster.aws_iam_role.task_execution (expand)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "module.fargate-cluster.aws_secretsmanager_secret.gitlab (expand)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "module.fargate-cluster.output.image_pull_secret_dockerhub (expand)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "local.env (expand)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "output.fargate_cluster_name", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "var.org_name", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "local.fargate_cluster_name (expand)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "var.webhook", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "output.ecs_sd_namespace_id", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "module.autoloader (close)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "var.env", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "output.fargate_cluster_id", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "module.databricks_metastore (close)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "var.fargate_cluster", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "module.fargate-cluster.output.sd_namespace_id (expand)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "provider[\"registry.terraform.io/hashicorp/aws\"]", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "output.fargate_task_execution_role", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "output.fargate_image_pull_secret_gitlab", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "output.ecs_log_group", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "module.fargate-cluster.output.image_pull_secret_gitlab (expand)", filtered by targeting.
2023-05-17T00:38:49.403Z [DEBUG] Removing "data.terraform_remote_state.vpc (expand)", filtered by targeting.
2023-05-17T00:38:49.404Z [DEBUG] Removing "output.ingestion_bucket", filtered by targeting.
2023-05-17T00:38:49.404Z [DEBUG] Removing "module.autoloader.module.cluster (close)", filtered by targeting.
2023-05-17T00:38:49.404Z [DEBUG] Removing "module.fargate-cluster (expand)", filtered by targeting.
2023-05-17T00:38:49.404Z [DEBUG] Removing "var.source_buckets", filtered by targeting.
2023-05-17T00:38:49.404Z [DEBUG] Removing "module.fargate-cluster.output.cluster_id (expand)", filtered by targeting.
2023-05-17T00:38:49.404Z [DEBUG] Removing "module.fargate-cluster.aws_cloudwatch_log_group.ecs (expand)", filtered by targeting.
2023-05-17T00:38:49.404Z [DEBUG] Removing "module.fargate-cluster.output.cloudwatch_group_name (expand)", filtered by targeting.
2023-05-17T00:38:49.406Z [DEBUG] Starting graph walk: walkDestroy
2023-05-17T00:38:49.412Z [DEBUG] created provider logger: level=debug
2023-05-17T00:38:49.412Z [INFO] provider: configuring client automatic mTLS
2023-05-17T00:38:49.724Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 args=[.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0]
2023-05-17T00:38:49.739Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 pid=457
2023-05-17T00:38:49.739Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0
2023-05-17T00:38:49.970Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Databricks Terraform Provider
2023-05-17T00:38:49.970Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:49.970Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: Version 1.15.0
2023-05-17T00:38:49.971Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:49.971Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: https://registry.terraform.io/providers/databricks/databricks/latest/docs
2023-05-17T00:38:49.971Z [DEBUG] provider.terraform-provider-databricks_v1.15.0:
2023-05-17T00:38:49.978Z [INFO] provider.terraform-provider-databricks_v1.15.0: configuring server automatic mTLS: timestamp=2023-05-17T00:38:49.975Z
2023-05-17T00:38:50.406Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: plugin address: address=/tmp/plugin1978377567 network=unix timestamp=2023-05-17T00:38:50.406Z
2023-05-17T00:38:50.406Z [DEBUG] provider: using plugin: version=5
2023-05-17T00:38:50.868Z [WARN] ValidateProviderConfig from "provider[\"registry.terraform.io/databricks/databricks\"]" changed the config value, but that value is unused
2023-05-17T00:38:50.875Z [INFO] provider.terraform-provider-databricks_v1.15.0: Explicit and implicit attributes: host, token: timestamp=2023-05-17T00:38:50.875Z
2023-05-17T00:38:50.890Z [INFO] Starting apply for module.databricks_metastore.databricks_metastore_assignment.this
2023-05-17T00:38:50.890Z [DEBUG] module.databricks_metastore.databricks_metastore_assignment.this: applying the planned Delete change
2023-05-17T00:38:51.476Z [DEBUG] provider.terraform-provider-databricks_v1.15.0: DELETE /api/2.1/unity-catalog/workspaces/3416610697718891/metastore?metastore_id=bdf99cfb-5e7d-47e9-ba0c-631f149c8fc9
< HTTP/2.0 400 Bad Request
< {
< "details": [
< {
< "@type": "type.googleapis.com/google.rpc.RequestInfo",
< "request_id": "adb256db-ae26-4b58-92d6-791eced5924e",
< "serving_data": ""
< }
< ],
< "error_code": "INVALID_PARAMETER_VALUE",
< "message": "Missing required field: metastore_id"
< }: timestamp=2023-05-17T00:38:51.475Z
2023-05-17T00:38:51.479Z [ERROR] provider.terraform-provider-databricks_v1.15.0: Response contains error diagnostic: @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR tf_rpc=ApplyResourceChange @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_summary="cannot delete metastore assignment: Missing required field: metastore_id" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=99edf2fe-2fa3-9226-822f-653b92565098 tf_resource_type=databricks_metastore_assignment timestamp=2023-05-17T00:38:51.478Z
2023-05-17T00:38:51.487Z [ERROR] vertex "module.databricks_metastore.databricks_metastore_assignment.this (destroy)" error: cannot delete metastore assignment: Missing required field: metastore_id
│ Error: cannot delete metastore assignment: Missing required field: metastore_id
2023-05-17T00:38:51.509Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2023-05-17T00:38:51.525Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.15.0/linux_amd64/terraform-provider-databricks_v1.15.0 pid=457
2023-05-17T00:38:51.525Z [DEBUG] provider: plugin exited
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment