optimized_configs.md

Performance Optimized Configs

Load Optimized

1. Make sure Propeller's pod has at least 8Gi of memory and 4CPUs
2. Increase Propeller's throughput:

   propeller:
     workers: 400
   

3. Increase MaxParalellism in admin's config:

   flyteadmin:
     maxParallelism: 20
   

Faster (and smaller) wf's Optimized

propeller:
  # Frequency of re-evaluating downstream tasks
  downstream-eval-duration: 10s
  
  # Frequency of re-evaluating workflows
  workflow-reeval-duration: 5s
  
  # Configuration to control the Kubernetes client
  kube-client-config:
    burst: 100
    qps: 400
    timeout: 30s
    
  # Workflow workqueue configuration, affects the way the work is consumed from the queue.
  queue:
    batch-size: -1
    batching-interval: 1s
    queue:
      base-delay: 3s
      capacity: 10000
      max-delay: 10s
      rate: 1000
      type: maxof
    sub-queue:
      base-delay: 0s
      capacity: 1000
      max-delay: 0s
      rate: 100
      type: bucket
    type: batch