gistfile1.txt

O nosso data flow é:
   (S3) -> (Lambda/Python) -> Glue (Transferencia e Metadados) -> (S3) -> Glue -> Redshift (DW) -> QuickSight (Visualização)
     -> Athena (Queries nos dados crus) ou QuickSight (Visualização)
- AWS Glue para o processo de extração dos dados "crus". Estou exporando o DBT. 
- A partir de dados armazendos no S3 (DB, arquivos) - voce faz a primeira filtragem e gera um novo Dataset-


--------------
- https://www.databricks.com/product/data-intelligence-platform
- https://www.databricks.com/product/data-intelligence-platform
- https://www.alura.com.br/artigos/airflow-entendendo-dags#:~:text=O%20Airflow%20%C3%A9%20uma%20plataforma,e%20monitorar%20fluxos%20de%20trabalho.
- https://aws.amazon.com/pt/glue/?nc1=h_ls
- https://medium.com/microsoftazure/an-introduction-to-streaming-etl-on-azure-databricks-using-structured-streaming-databricks-16b369d77e34
- https://www.analytics8.com/blog/dbt-overview-what-is-dbt-and-what-can-it-do-for-my-data-pipeline/#:~:text=dbt%20(data%20build%20tool)%20makes,entire%20transformation%20process%20with%20code.
- https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/build-an-etl-service-pipeline-to-load-data-incrementally-from-amazon-s3-to-amazon-redshift-using-aws-glue.html
- https://k21academy.com/microsoft-azure/dp-100/devops-for-data-science/