sb-static
sb-static
/ sgr_assistant.py
Last active
September 3, 2025 01:55
SGR Demo - assistant based on small LLM (Qwen3-4B) using llama.cpp and Schema-Guided Reasoning (SGR)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| This Python code demonstrates Schema-Guided Reasoning (SGR) with llama.cpp and small model - Qwen3-4B Q8_0. It: | |
| - implements a business agent capable of planning and reasoning | |
| - implements tool calling using only SGR and simple dispatch | |
| - uses with a simple (inexpensive) non-reasoning model for that | |
| This demo is modified from https://abdullin.com/schema-guided-reasoning/demo to support local llm | |
| Test model: Qwen3-4B-Instruct-2507-Q8_0 (https://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF/resolve/main/Qwen3-4B-Instruct-2507-Q8_0.gguf) |