Unsloth Desktop Runs Claude Code on Your Own GPU. Two Defaults Break It First.

Unsloth's new desktop app serves an Anthropic-compatible `/v1/messages` endpoint, so Claude Code can drive a local GGUF. The two settings that decide whether it actually works are buried in warning boxes.

The most interesting document Unsloth published this week is not the launch post. It is a yellow warning box halfway down their Claude Code guide that says local models will run about 90% slower unless you turn off the attribution header Claude Code prepends to every request.

That box is the real story of local agent inference in 2026. The model is not the hard part anymore. Quantized 30-billion-parameter mixture-of-experts models that fit in 24 GB of unified memory can hold a coding loop together well enough to be useful. The hard part is that agent CLIs were built against hosted inference, and every assumption baked into them (huge system prompts, per-request headers, server-side tools) turns into a performance cliff the moment the endpoint is a laptop.

Unsloth Desktop, whose changelog entry lands on August 10 for macOS, Windows, and Linux, is the first setup I have used where connecting Claude Code to a local model takes one command instead of an afternoon. It is also the first one where I hit both cliffs inside twenty minutes.

What actually shipped

Unsloth is best known as a training library. The desktop app wraps that plus llama.cpp into a local UI for running and fine-tuning open models, and it covers GGUF and MLX, text, vision, audio, embedding, and diffusion models. The repository carries two licenses: Apache 2.0 for the core package, AGPL-3.0 for the Studio interface. Worth knowing before it goes anywhere near a product.

The feature that changes how you work is smaller than the training story. Unsloth serves two API dialects on one port:

POST /v1/messages, Anthropic Messages API, for Claude Code, OpenClaw, and the Anthropic SDK
POST /v1/chat/completions and /v1/responses, OpenAI-compatible, for opencode, Cursor, Continue, Cline, and the OpenAI SDK

Both handle streaming, tool calling in their native schema, and vision inputs. So the same loaded model can back Claude Code and Codex at once, and you can point a frontier orchestrator at the cloud while the noisy high-volume subagent calls stay on your hardware.

You install with a single line and it comes up on port 8888:

curl -fsSL https://unsloth.ai/install.sh | sh

Then, from your project folder, with a model loaded:

unsloth start claude

That launches Claude Code against the local model without touching your existing Claude Code configuration. Unsloth sets the base URL, key, model, and context length for that launch only. It is the cleanest version of this I have seen, and it is why the two defaults underneath matter so much.

Default one: the header that costs you 90% of your speed

Claude Code prepends an attribution line to the start of its system prompt: x-anthropic-billing-header: cc_version=...; cch=...;. The value changes on every request.

Sit with what that does to a local model. Your entire system prompt prefix, which is the largest fixed block of tokens in the conversation and the thing KV caching exists to reuse, now begins with a string that is different every turn. The cache misses on the first token. Every turn. You recompute the whole prefix, forever.

Against hosted Anthropic models you would never notice, because the prefix cost is amortized inside somebody else's cluster. Against a GGUF on your own GPU it is the difference between a usable agent and one you abandon.

Unsloth's fix is a launch flag, so there is no file to edit:

claude --settings '{"env":{"CLAUDE_CODE_ATTRIBUTION_HEADER":"0","CLAUDE_CODE_ENABLE_TELEMETRY":"0"}}' \
  --model unsloth/gemma-4-26B-A4B-it-GGUF

To make it stick, set CLAUDE_CODE_ATTRIBUTION_HEADER to "0" inside the env block of ~/.claude/settings.json. Recent Claude Code builds also honor the shell variable, and older ones ignore it, which is why the docs push the settings form.

While you are there, two more flags shrink the prompt further:

claude --model unsloth/gemma-4-26B-A4B-it-GGUF --bare --exclude-dynamic-system-prompt-sections

--bare skips auto-discovery of hooks, skills, plugins, MCP servers, and CLAUDE.md, keeping Bash and file read and edit. --exclude-dynamic-system-prompt-sections moves per-machine sections out of the prompt prefix so more of it stays cacheable. You give up your whole plugin setup for speed, which is a real trade, and on a local model it is usually the right one.

Default two: the server eats your agent's tool calls

The second one is stranger and took me longer to diagnose, because it throws no error at all and looks like a model problem.

Unsloth Studio runs its own server-side tools. Python execution, bash, web search, all handled inside the inference server and streamed back as tool_result events. For chat, that is a genuinely good feature. For a coding agent it is a disaster, because Unsloth intercepts the tool calls that Claude Code intended to run itself. The docs describe the symptom exactly: Claude Code answers, but never edits files.

You are watching an agent describe the change it is making while making no change.

The fix is one flag, and the docs are explicit that you want it any time an external coding agent is driving:

unsloth run \
  --model unsloth/gemma-4-26B-A4B-it-GGUF \
  --disable-tools \
  --reasoning off \
  -p 8888

--disable-tools switches to passthrough so Claude Code's own Write, Edit, and Bash tools do the work. --reasoning off is optional and usually helps for agentic coding, where thinking tokens buy less than they cost.

There is a security default worth knowing in the same area. Unsloth ties the server-side tool policy to the bind address: tools are on by default on 127.0.0.1, and off by default on 0.0.0.0 or any non-loopback address, with a yes-or-no prompt if you force them on. The docs give the reason in one sentence, and it is the correct one: a leaked API key on a network-exposed server means arbitrary code execution on the host. That policy resolves at the process level and individual requests cannot override it. Good design, and a real reason to think twice before adding -H 0.0.0.0 so you can hit the model from your phone.

Put this into practice

The lowest-friction path, in order, on a machine with 24 GB or more of VRAM or unified memory:

Install with curl -fsSL https://unsloth.ai/install.sh | sh, or grab the desktop build from Unsloth's download page. Open http://127.0.0.1:8888 and set a password.
Load a model. Gemma 4 26B-A4B at UD-Q4_K_XL is the documented starting point. Qwen3.5-35B-A3B is the other one worth trying for coding.
Send one message in the Unsloth chat UI to confirm the model actually loaded.
Restart the server with --disable-tools before you connect any agent.
Set CLAUDE_CODE_ATTRIBUTION_HEADER to "0" in ~/.claude/settings.json.
From a project folder, run unsloth start claude. Check /model inside Claude Code to confirm it is pointed at your local model and not silently falling back to a cloud one.

If you want to wire it by hand instead, the manual path is four variables, and skipping the second one is how you end up staring at a 401 Unauthorized: ANTHROPIC_BASE_URL set to http://localhost:8888, ANTHROPIC_AUTH_TOKEN set to your sk-unsloth-… key from Settings → API, an empty ANTHROPIC_API_KEY so Claude Code stops asking for a cloud key, and ANTHROPIC_MODEL set to the exact id returned by GET /v1/models. When it refuses to connect entirely, unset ANTHROPIC_BASE_URL is the first thing to try.

Honest limitations

This is a beta, and it reads like one in places. The Linux install step in the desktop docs still links out to jan.ai's documentation, which tells you something about how recently the app was assembled.

The headline accuracy claim is stated two different ways in two Unsloth documents. The desktop page advertises "50% more accurate, self-healing tool calls." The API page says self-healing tool calling "helps reduce broken or malformed tool calls by 50%." Those are different claims, and neither comes with a named model, a task set, or a baseline. Treat it as a company figure until someone publishes a reproduction.

It is also unclear whether the self-healing repair still applies once you pass --disable-tools. That flag is documented as controlling server-side tools, meaning web search, Python, and bash, and Unsloth does not say anywhere whether the healing layer also covers a client agent's own tool calls. Ask before you count on it.

The privacy story is partial. Unsloth says the app runs fully offline with no telemetry, and local inference is local. Web search and deep research reach the internet by definition, and if you connect cloud models through the same interface, those calls leave your machine like any other API call.

And a 26B model at four bits is not Opus. It holds a focused refactor together. It does not hold a 200,000-token codebase investigation together, and the moment you need that you are back on the cloud model paying cloud prices. The realistic shape here is a frontier orchestrator with a local worker tier underneath, not a replacement.

Try it on something boring

Point it at a repository you do not care about and give it a task with a clear finish line. Rename a symbol across twelve files. Write tests for one module. Something where you will notice immediately if the agent is talking instead of editing.

That first run will tell you more about whether local agent inference is ready for your work than any benchmark will. And now the setup costs you one command instead of an afternoon, which is the actual thing that changed this week.

Sources: unslothai/unsloth on GitHub, Introducing Unsloth Desktop, How to Run Local LLMs with Claude Code, How to use Unsloth as an API endpoint.