Skip to content

Instantly share code, notes, and snippets.

View AlexsanderHamir's full-sized avatar

Alexsander Hamir AlexsanderHamir

7 followers · 1 following
View GitHub Profile
@AlexsanderHamir
AlexsanderHamir / results.sh
Created December 3, 2025 13:48
Prometheus - After Setting PROMETHEUS_MULTIPROC_DIR (Queried During Load Test)
╭─    ~/Documents/litellm-proxy-memory-issues    litellm_PROM…nconsistency *9 !15 ?12 ──────────────────────────────────── ✔  base   05:43:53  ─╮
╰─ wget -qO- http://localhost:4000/metrics \ ─╯
| grep 'litellm_deployment_total_requests_total{' \
| awk -F'[{} ]' '{
print "Metric: " $1;
print "Value: " $NF;
print "Labels:";
n = split($2, a, ",");
for (i = 1; i <= n; i++) print " " a[i];
print "";
@AlexsanderHamir
AlexsanderHamir / results.sh
Created December 3, 2025 13:45
Prometheus - Before Setting PROMETHEUS_MULTIPROC_DIR (Queried During Load Test)
╭─    ~/Documents/litellm-proxy-memory-issues    litellm_PROM…nconsistency *9 !15 ?12 ──────────────────────────────────── ✔  base   05:43:07  ─╮
╰─ wget -qO- http://localhost:4000/metrics \ ─╯
| grep 'litellm_deployment_total_requests_total{' \
| awk -F'[{} ]' '{
print "Metric: " $1;
print "Value: " $NF;
print "Labels:";
n = split($2, a, ",");
for (i = 1; i <= n; i++) print " " a[i];
print "";
@AlexsanderHamir
AlexsanderHamir / results.sh
Created December 2, 2025 19:04
Prometheus - Metrics After Setting PROMETHEUS_MULTIPROC_DIR
╭─    ~/Documents/litellm-proxy-memory-issues    litellm_PROM…nconsistency *9 !15 ?12 ───────────────────────────── ✔  5s   base   10:57:06  ─╮
╰─ wget -qO- http://localhost:4000/metrics \ ─╯
| grep 'litellm_deployment_total_requests_total{' \
| awk -F'[{} ]' '{
print "Metric: " $1;
print "Value: " $NF;
print "Labels:";
n = split($2, a, ",");
for (i = 1; i <= n; i++) print " " a[i];
print "";
@AlexsanderHamir
AlexsanderHamir / results.sh
Created December 2, 2025 19:02
Prometheus - Metrics Before Setting PROMETHEUS_MULTIPROC_DIR
╭─    ~/Documents/litellm-proxy-memory-issues    litellm_PROM…nconsistency *9 !15 ?12 ──────────────────────────── ✔  16s   base   10:44:04  ─╮
╰─ wget -qO- http://localhost:4000/metrics \ ─╯
| grep 'litellm_deployment_total_requests_total{' \
| awk -F'[{} ]' '{
print "Metric: " $1;
print "Value: " $NF;
print "Labels:";
n = split($2, a, ",");
for (i = 1; i <= n; i++) print " " a[i];
print "";
@AlexsanderHamir
AlexsanderHamir / results.sh
Created December 2, 2025 18:59
Prometheus - Metrics During Load Test
pyenv shell 3.12.10
╭─    ~/Documents/litellm-proxy-memory-issues  loading ───────────────────────────────────────────────────────────────────── ✔  base   10:19:26  ─╮
╰─ pyenv shell 3.12.10 ─╯
╭─    ~/Documents/litellm-proxy-memory-issues    litellm_PROM…nconsistency *9 ──────────────────────────────────────────── ✔  base   10:19:28  ─╮
╰─ wget -qO- http://localhost:4000/metrics \ ─╯ ─╯
| grep 'litellm_deployment_total_requests_total{' \
| awk -F'[{} ]' '{
print "Metric: " $1;
print "Value: " $NF;
@AlexsanderHamir
AlexsanderHamir / results.md
Created December 2, 2025 15:35
Prometheus Results 
## Full Metrics Capture

Below is the full raw output captured from repeated executions of the same command.

### Command

```bash
wget -qO- http://localhost:4000/metrics \
 | grep 'litellm_deployment_total_requests_total{' \
@AlexsanderHamir
AlexsanderHamir / readme.md
Last active December 2, 2025 15:44
Prometheus - Steps to Repro

Check LiteLLM Version

litellm --version                                                                                          ─╯

LiteLLM: Current Version = 1.80.7

1. Create prometheus-config.yaml




  


  




    


          
    

      

        
        

          

              @AlexsanderHamir
          

          

            
                AlexsanderHamir
                / command.sh
            
            

              Last active
              December 2, 2025 15:37
            

              
                Prometheus - Test Server Initialization
              
          

        

      

        

  

    
    


        



  


  

        

          
          rm -rf "$PROMETHEUS_MULTIPROC_DIR"
        

        

          
          mkdir -p "$PROMETHEUS_MULTIPROC_DIR"
        

        

          
          litellm  --config prometheus-config.yaml --num_workers 2 
        

  





    


  




    


          
    

      

        
        

          

              @AlexsanderHamir
          

          

            
                AlexsanderHamir
                / config.yaml
            
            

              Last active
              December 2, 2025 15:37
            

              
                Prometheus - LiteLLM Config
              
          

        

      

        

  

    
    


        



  


  

        

          
          model_list:
        

        

          
            - model_name: db-openai-endpoint
        

        

          
              litellm_params:
        

        

          
                model: openai/*
        

        

          
                api_base: https://exampleopenaiendpoint-production-0ee2.up.railway.app/
        

        

          
          

        

        

          
          litellm_settings:
        

        

          
            success_callback: ["prometheus"]
        

        

          
            failure_callback: ["prometheus"]
        

  





    


  




    


          
    

      

        
        

          

              @AlexsanderHamir
          

          

            
                AlexsanderHamir
                / no_cache_hits.py
            
            

              Last active
              November 18, 2025 17:39
            

              
                LiteLLM Locust File (/audio/transcriptions)
              
          

        

      

        

  

    
    


        



  


  

        

          
          from pathlib import Path
        

        

          
          

        

        

          
          from locust import HttpUser, between, task
        

        

          
          

        

        

          
          AUDIO_FILE_BYTES = (Path(__file__).resolve().parent / "speech_small.mp3").read_bytes()
        

        

          
          

        

        

          
          

        

        

          
          class MyUser(HttpUser):
        

        

          
              """
        

        

          
              Minimal Locust user for repeatedly hitting `/v1/audio/transcriptions`.