Skip to content

Instantly share code, notes, and snippets.

@john-adeojo

john-adeojo/vLLM_endpoint.py

Created January 30, 2024 13:00
Show Gist options
  • Save john-adeojo/444bac9bae05ca5d2adc1328c2a06b15 to your computer and use it in GitHub Desktop.
Save john-adeojo/444bac9bae05ca5d2adc1328c2a06b15 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
vLLM_endpoint.py
# Assign up model path and endpoint
model_path = "JAdeojo/casperhansen-mixtral-instruct-awq-clone-dec23"
vllm_endpoint = "https://7963anhsa13sbx-8000.proxy.runpod.net/v1"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment