Skip to content

Instantly share code, notes, and snippets.

316usman /
Created March 27, 2024 18:20 — forked from aleksandr-smechov/
vLLM FastAPI server with very basic auth
# Adapted from
import argparse
import asyncio
import json
import os
import time
from http import HTTPStatus
from typing import AsyncGenerator, Dict, List, Optional, Tuple, Union