Skip to content

Instantly share code, notes, and snippets.

@ljredux
Last active May 14, 2023 11:46
Llama Loader - A bat file which cycles through .bin (model) files in the same folder as llama.cpp (https://github.com/ggerganov/llama.cpp) executables and asks the user which one to load. An asterisk denotes the previous model loaded.
@echo off
:start
title llama.cpp
setlocal EnableDelayedExpansion
echo Llama Loader v0.3 by ljredux
echo An asterisk signifies the model which was previously loaded.
echo.
if exist selected_model.txt (
for /f "tokens=*" %%a in (selected_model.txt) do set "previous_model=%%a"
)
set /a count=0
for %%f in (*.bin) do (
set /a count+=1
set "file[!count!]=%%f"
if "%%f"=="%previous_model%" (
echo *!count!.%%f
) else (
echo !count!.%%f
)
)
echo.
set /p selection="Enter the number of the model you wish to load: "
set "selected_model=!file[%selection%]!"
echo Selected model: %selected_model%
echo %selected_model%>selected_model.txt
title llama.cpp - %selected_model%
REM Customize llama.cpp options
main -i --interactive-first -r "### Human:" --temp 0 -c 2048 -n -1 --repeat_penalty 1.2 --instruct --keep -1 -m %selected_model%
REM Reload if user selects N after unavoidable "Terminate batch job (Y/N)?" prompt.
echo.
goto start
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment