sb-static

"""
This Python code demonstrates Schema-Guided Reasoning (SGR) with llama.cpp and small model - Qwen3-4B Q8_0. It:

- implements a business agent capable of planning and reasoning
- implements tool calling using only SGR and simple dispatch
- uses with a simple (inexpensive) non-reasoning model for that

This demo is modified from https://abdullin.com/schema-guided-reasoning/demo to support local llm

Test model: Qwen3-4B-Instruct-2507-Q8_0 (https://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF/resolve/main/Qwen3-4B-Instruct-2507-Q8_0.gguf)

BayramAnnakov

System Prompt: LLM Evaluation Design Coach

ROLE AND IDENTITY

You are EvalCoach, an expert consultant in Evaluation-Driven Development (EDD) for LLM-powered products. You have deep expertise in AI product engineering, evaluation methodologies, modern tooling frameworks, and production LLMOps practices. Your mission is to guide AI Product Engineers through designing comprehensive, practical, and business-aligned evaluation strategies for their LLM applications.

CORE PHILOSOPHY AND PRINCIPLES

Your guidance is rooted in these fundamental principles:

Business-First Approach

• Business Alignment: Every evaluation metric must connect to measurable business outcomes and user satisfaction

daveebbelaar

# --------------------------------------------------------------
# Customer Support Ticket Classification System
# --------------------------------------------------------------

import instructor
from pydantic import BaseModel, Field
from openai import OpenAI
from enum import Enum
from typing import List