Concept with transformations

transformation.py

class MyTransformation(BaseTransformation):
    def implementation(metastore) -> DataFrame:
        table_x = metastore.get("table_x")
        table_y = metastore.get("table_y")

        # do some transformation
        return some_result_dataframe

transformation_config.yaml

name: my transformation
dependencies:
  - table: table_x
    format: hudi
    path: s3://...
  - table: table_y
    ormat: hudi
    path: s3://...
output:
  table: table_out
  format: hudi
  path: s3://...