> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Use Coding Agents with Local Models via Tokios

> Point Claude Code, OpenCode, Aider, Cline, Continue, and more at your local models through one Tokios endpoint — switch models, no agent reconfiguring.

Every major coding agent communicates with a model backend over an OpenAI- or Anthropic-compatible API. Point that API at Tokios, and your agent talks to a local model running on your own hardware — from any machine, not just the one running the inference server. Switch models by changing 1 field; the agent configuration stays the same.

## How coding agents connect to backends

Coding agents send chat completions (and often tool calls) over HTTP. They read an endpoint URL and an API key from environment variables or a config file. Tokios exposes both an OpenAI-compatible and an Anthropic-compatible surface at the same URL, so most agents work with a single configuration change.

| Agent | API format | Key env var | Base URL |
| - | - | - | - |
| Claude Code | Anthropic | `ANTHROPIC_API_KEY` | `ANTHROPIC_BASE_URL` |
| OpenCode | OpenAI | `options.apiKey` | `options.baseURL` |
| Aider | OpenAI | `OPENAI_API_KEY` | `OPENAI_API_BASE` |
| Cline | OpenAI | API key in UI | Custom endpoint |
| Continue | OpenAI | API key in config | `apiBase` |
| Roo Code | OpenAI | API key in UI | Custom endpoint |

## Prerequisites

* A running Tokios connector paired with your local model — see [Connector install](https://docs.tokios.com/setup/connector-install.md)
* A registered deployment name — see [Register a model](https://docs.tokios.com/setup/register-model.md)
* A Tokios API key — see [API keys](https://docs.tokios.com/setup/api-keys.md)

## Claude Code

Claude Code uses the Anthropic Messages API. Set 2 environment variables and pass your deployment name with `--model`:

> [tab] macOS & Linux
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> export ANTHROPIC_BASE_URL=https://api.tokios.com
> export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
> claude --model qwen3-coder
> ```

> [tab] Windows CMD
>
> ```cmd theme={"theme":{"light":"github-light","dark":"css-variables"}}
> set ANTHROPIC_BASE_URL=https://api.tokios.com
> set ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
> claude --model qwen3-coder
> ```

> [tab] Windows PowerShell
>
> ```powershell theme={"theme":{"light":"github-light","dark":"css-variables"}}
> $env:ANTHROPIC_BASE_URL = "https://api.tokios.com"
> $env:ANTHROPIC_API_KEY = "sk-tok-YOUR_KEY"
> claude --model qwen3-coder
> ```

Claude Code's built-in model picker only lists `claude-*` ids. Always pass `--model` explicitly with your Tokios deployment name. See the full [Claude Code guide](https://docs.tokios.com/guides/claude-code.md) for persistence and troubleshooting.

## OpenCode

OpenCode uses the AI SDK with an OpenAI-compatible provider. Add Tokios to `opencode.json`:

```json opencode.json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "tokios": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Tokios",
      "options": {
        "baseURL": "https://api.tokios.com/v1",
        "apiKey": "sk-tok-YOUR_KEY"
      },
      "models": {
        "qwen3-coder": {
          "name": "qwen3-coder"
        }
      }
    }
  }
}
```

> [!WARNING]
>
> `options.baseURL` must include `/v1`. OpenCode resolves request paths relative to this URL, so omitting `/v1` causes 404 errors.

See the full [OpenCode guide](https://docs.tokios.com/guides/opencode.md) for detailed setup and troubleshooting.

## Aider

Aider reads OpenAI-compatible configuration from environment variables:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
export OPENAI_API_BASE=https://api.tokios.com/v1
export OPENAI_API_KEY=sk-tok-YOUR_KEY
aider --model openai/qwen3-coder
```

The `openai/` prefix tells Aider to use the OpenAI chat completions format. Aider supports tool calling with most models that implement the OpenAI function-calling schema.

## Cline

Cline connects to custom OpenAI-compatible endpoints through its VS Code settings:

1. Open Cline settings in VS Code.
2. Under **API Configuration**, select **OpenAI Compatible**.
3. Set **Base URL** to `https://api.tokios.com/v1`.
4. Paste your `sk-tok-YOUR_KEY` in the **API Key** field.
5. Enter your deployment name (e.g. `qwen3-coder`) as the **Model ID**.

Cline uses tool calling extensively. Pick a model with strong tool-call support — see [Model recommendations](#model-recommendations-for-coding) below.

## Continue

Add Tokios as a custom provider in `~/.continue/config.json`:

```json ~/.continue/config.json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "models": [
    {
      "title": "Tokios - Qwen3 Coder",
      "provider": "openai",
      "model": "qwen3-coder",
      "apiBase": "https://api.tokios.com/v1",
      "apiKey": "sk-tok-YOUR_KEY"
    }
  ]
}
```

Continue supports both chat completions and tool calling through this provider configuration.

## Roo Code

Roo Code uses the same OpenAI-compatible endpoint pattern:

1. Open Roo Code settings.
2. Select **OpenAI Compatible** as the API provider.
3. Set **Base URL** to `https://api.tokios.com/v1`.
4. Enter your API key and deployment name.

## Switch models without reconfiguring

Because every agent reads the model name from a single field — the `model` parameter in the API request — you can register multiple deployments on Tokios and swap between them without touching any agent config:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
# Use Qwen3-Coder for code tasks
claude --model qwen3-coder

# Use GLM-4.5-Air for general reasoning
claude --model glm-4-chat

# Use Gemma for lightweight tasks
claude --model gemma-tunnel
```

The endpoint and API key stay the same. Tokios routes each request to the correct local backend.

## Model recommendations for coding

Not all local models handle coding tasks equally well. Tool calling — the ability to invoke functions, read files, and run commands — is critical for agent-based workflows.

| Model | Total params | Best for | Tool calling | Notes |
| - | - | - | - | - |
| Muse Glimmer 30B | 29.6B | Agentic coding, tool use | Good | Trained for tool invocation against precise schemas with failure recovery; 4-bit build fits a single 24 GB GPU |
| Qwen3-Coder-30B | 30B | Code generation, refactoring | Good | Confirmed working with Cline and Aider |
| Qwen3-32B | 32B | General + coding | Good | Strong all-rounder; 43 tok/s on DGX Spark |
| GLM-4.5-Air | 106B MoE | Complex reasoning | Good | 12B active params; community-favorite for Spark |
| Qwen3.5-35B-A3B | 35B MoE | Fast coding | Good | 3B active; 120 tok/s with Atlas on Spark |
| Gemma-4-26B-A4B | 26B MoE | Lightweight tasks | Moderate | 4B active; fast but less capable on complex edits |

> [!WARNING]
>
> vLLM on AMD GPUs has known issues with tool calling. If you run on Strix Halo or other AMD hardware, use llama.cpp with the Vulkan backend for cleaner tool-call support.

## Tips for coding with local models

**Use a model that supports tool calling.** Agents like Claude Code, OpenCode, and Cline rely on structured function calls. If your model doesn't support the OpenAI function-calling schema, the agent falls back to text-based tool invocation, which is slower and less reliable.

**Set context length appropriately.** Coding agents send large prompts with file contents and conversation history. Set `--max-model-len` on your inference server to at least 8192 tokens. Models with 32K+ context handle multi-file tasks better.

**Prefer MoE models for throughput.** Mixture-of-experts models like Qwen3.5-35B-A3B activate only a fraction of their parameters per token, delivering higher throughput than dense models of similar total size. For iterative coding tasks where latency matters, MoE is often the better choice.

**Test tool calls before relying on them.** Send a quick test with a tool-calling payload to confirm your model and backend handle function calls correctly:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-coder",
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "tools": [{"type": "function", "function": {"name": "calculator", "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}}}}]
  }'
```

If the response includes a `tool_calls` array, tool calling works.

## One endpoint for all your agents

Tokios gives every coding agent the same URL and API key. Register your models once, generate 1 API key, and configure any number of agents to point at `https://api.tokios.com`. Switch models, add new backends, or move models to different hardware — your agent configurations don't change.

> [card] [Model routing](https://docs.tokios.com/guides/model-routing.md)
>
> Learn how to register multiple models and route requests by deployment name.

> [card] [Choose a local model](https://docs.tokios.com/guides/choose-a-local-model.md)
>
> Compare backends and quantization options before committing a model to agentic workflows.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.