Ravi Dayabhai & Conrad Warren 🧱 2024-06-21
Consider the following array of 25 squares:
You are filling the array with rectangles by repeating the following two steps:
from functools import reduce | |
from transforms.api import transform_df, Input, Output | |
@transform_df( | |
Output("/data_project/manifests/trp_synced_paths_manifest"), | |
incremental_manifest=Output("/data_project/manifests/trp_incremental_manifest"), | |
synced_paths=Input("/data_project/manifests/trp_synced_paths_manifest"), | |
bucket_data=Input("/data_project/input/s3_bucket_data_source") # Data source for your bucket content | |
) | |
def incremental_sync_and_update(ctx, synced_paths, bucket_data): |