Skip to content

Instantly share code, notes, and snippets.

@ivanfioravanti
Created April 27, 2024 17:33
Show Gist options
  • Save ivanfioravanti/513cf934bb5e89390c14b2ba66f37d96 to your computer and use it in GitHub Desktop.
Save ivanfioravanti/513cf934bb5e89390c14b2ba66f37d96 to your computer and use it in GitHub Desktop.
LibreChat section for MLX
# For more information, see the Configuration Guide:
# https://docs.librechat.ai/install/configuration/custom_config.html
# Configuration version (required)
version: 1.0.6
# Cache settings: Set to true to enable caching
cache: true
# Custom nterface configuration
interface:
# Privacy policy settings
privacyPolicy:
externalUrl: 'https://librechat.ai/privacy-policy'
openNewTab: true
# Terms of service
termsOfService:
externalUrl: 'https://librechat.ai/tos'
openNewTab: true
endpoints:
custom:
- name: "MLX"
apiKey: "MLX"
baseURL: "http://host.docker.internal:8080/v1/"
models:
default: [
"mlx-community/Meta-Llama-3-8B-Instruct-8bit",
]
fetch: false
titleConvo: true
titleModel: "mlx-community/Meta-Llama-3-8B-Instruct-8bit"
summarize: false
summaryModel: "mlx-community/Meta-Llama-3-8B-Instruct-8bit"
forcePrompt: false
modelDisplayLabel: "MLX"
iconURL: "https://ml-explore.github.io/mlx/build/html/_static/mlx_logo.png"
addParams:
"max_tokens": 2000
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment