Skip to content

Instantly share code, notes, and snippets.

@esenthil2018
Created March 23, 2023 20:18
Show Gist options
  • Save esenthil2018/72825e1786827a3fd499702239156460 to your computer and use it in GitHub Desktop.
Save esenthil2018/72825e1786827a3fd499702239156460 to your computer and use it in GitHub Desktop.
Tool Scalability Deployment Language Data Processing ML Libraries Execution Environment Ease of Use Use Cases
Apache Airflow High On-prem/cloud Python Limited Any Local, Kubernetes Moderate Data pipelines, workflow management, task scheduling
MLflow Medium On-prem/cloud Python Limited Any Local, Kubernetes Easy Experiment tracking, model management, model deployment
Kubeflow High Cloud Python Limited Any Kubernetes Moderate End-to-end ML pipelines, distributed training, model serving
Prefect High On-prem/cloud Python Limited Any Local, Kubernetes Easy Data pipelines, workflow management, task scheduling
TFX High Cloud Python TensorFlow TensorFlow Kubernetes Moderate End-to-end ML pipelines, TensorFlow-specific tasks
Dask High On-prem/cloud Python High Any Local, Kubernetes, HPC Moderate Large-scale data processing, parallel and distributed computing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment