Running qwen3-coder locally with VS Code.md

Run with Claude Code

When running with claude code, certain things don't work (e.g. the web_search tool, or running agents in parallel). Inference is slower than with Claude, and results are worse.

Install llama.cpp and set up qwen3-coder per these instructions: https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally#run-qwen3-coder-30b-a3b-instruct

As of 2025/08/17, you need to download the fixed ninja chat template per these instructions: https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally#tool-calling-fixes

Install claude-code-router per these instructions: https://github.com/musistudio/claude-code-router?tab=readme-ov-file#-getting-started

Create a ~/.claude-code-router/config.json that contains:

{
  "LOG": true,
  "LOG_LEVEL": "trace",
  "CLAUDE_PATH": "",
  "HOST": "127.0.0.1",
  "PORT": 3456,
  "APIKEY": "",
  "API_TIMEOUT_MS": "600000",
  "transformers": [],
  "Providers": [
    {
      "name": "llama.cpp",
      "api_base_url": "http://localhost:8080/v1/chat/completions",
      "api_key": "sk-xxx",
      "models": [
        "qwen3-coder-30b-a3b-instruct"
      ]
    }
  ],
  "StatusLine": {
    "enabled": false,
    "currentStyle": "default",
    "default": {
      "modules": []
    },
    "powerline": {
      "modules": []
    }
  },
  "Router": {
    "default": "llama.cpp,qwen3-coder-30b-a3b-instruct",
    "background": "llama.cpp,qwen3-coder-30b-a3b-instruct",
    "think": "llama.cpp,qwen3-coder-30b-a3b-instruct",
    "longContext": "llama.cpp,qwen3-coder-30b-a3b-instruct",
    "longContextThreshold": 60000,
    "webSearch": "llama.cpp,qwen3-coder-30b-a3b-instruct"
  },
  "stream": false
}

Then:

1. Run llama-server:

llama-server \
    -hf unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_XL \
    --jinja -ngl 99 --threads -1 --ctx-size 262144 \
    --temp 0.7 --min-p 0.0 --top-p 0.80 --top-k 20 --repeat-penalty 1.05 --chat-template-file ~/Downloads/chat_template.jinja --alias qwen3-coder-30b-a3b-instruct -fa

1. Run claude code router:

ccr code

Run without Claude Code

Running without claude code is very un-agentic: it feels a lot less powerful.

Common instructions:

1. Install LM Studio: https://lmstudio.ai/ Instructions: https://formulae.brew.sh/cask/lm-studio
   • Note: as of 2025/08/16, Ollama currently doesn't support chat templates for qwen tool calls. For now, the recommended workaround is to use LM Studio instead.
2. Download qwen/qwen3-coder-30b in LM studio: https://lmstudio.ai/models/qwen/qwen3-coder-30b
3. Change LM Studio's model settings to the https://docs.unsloth.ai/basics/qwen3-coder-how-to-run-locally#recommended-settings,
   • If on an M4 mac or another machine with lots of memory, increase the context window size and optionally enable KV cache quantization
4. Load the model in LM Studio (default port is 1234)

Option 1: Cline VS Code extension

1. Install the Cline VS code extension
2. Open Cline Settings and change the API Configuration details:
   • API Provider: OpenAI Compatible
   • Base URL: http://localhost:1234/v1
   • OpenAI Compatible API Key: lm-studio (the contents here don't matter)
   • Model ID: qwen/qwen3-coder-30b
   • Model configuration: change the context windows size (e.g. 1000000 on a large M4 mac), set temperature to 0.7 per qwen3-coder's recommended settings

Option 2: qwen-code

NOTE: I couldn't get the model's output to be well-rendered using this approach: it would output raw markdown without line breaks, which was very hard to read in a terminal.

NOTE: I haven't tested tool use, but they might have issues as of 2025/08/16: QwenLM/qwen-code#176

Instructions as of 2025/08/16:

1. Create a .env file in your project root with the following content:

OPENAI_API_KEY=lm-studio
OPENAI_BASE_URL=http://localhost:1234/v1
OPENAI_MODEL=qwen/qwen3-coder-30b

2. Install qwen-code, Qwen's version of the claude / gemini CLI tool: https://github.com/QwenLM/qwen-code?tab=readme-ov-file#install-from-npm
3. Run qwen: it should auto-detect the .env file you created above.