> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Coding agents that work with local models through Tokios

> A compatibility matrix of 15 coding agents — IDE-based and terminal — that connect to local models through the Tokios OpenAI- and Anthropic-compatible API.

15 coding agents connect to local models through Tokios. All use standard OpenAI or Anthropic API format — you change 2 things: the **base URL** and your **API key**. Once your model is registered as a Tokios deployment, every agent below reaches it through the same endpoint.

> [!TIP]
>
> **Prerequisites:** Before configuring any agent below, you need a running local model, a paired Tokios connector, a registered deployment, and a `sk-tok-…` API key. See the [Quickstart](https://docs.tokios.com/quickstart.md) if you haven't set those up yet.

## Agent compatibility matrix

| Agent | Type | API format | Tool calling | Setup guide |
| - | - | - | - | - |
| Claude Code | Terminal | Anthropic | Yes | [/guides/claude-code](https://docs.tokios.com/guides/claude-code.md) |
| OpenAI Codex | Terminal | OpenAI (Responses) | Yes | [/guides/openai-codex](https://docs.tokios.com/guides/openai-codex.md) |
| Aider | Terminal | OpenAI | Yes | [/guides/aider](https://docs.tokios.com/guides/aider.md) |
| Cline | IDE | OpenAI | Yes | [/guides/cline](https://docs.tokios.com/guides/cline.md) |
| Continue | IDE | OpenAI | Yes | [/guides/continue](https://docs.tokios.com/guides/continue.md) |
| Roo Code | IDE | OpenAI | Yes | [/guides/roo-code](https://docs.tokios.com/guides/roo-code.md) |
| OpenCode | Terminal | OpenAI | Yes | [/guides/opencode](https://docs.tokios.com/guides/opencode.md) |
| Goose | IDE | OpenAI | Yes | [/guides/goose](https://docs.tokios.com/guides/goose.md) |
| Pi | Terminal | OpenAI · Anthropic | Yes | [/guides/pi](https://docs.tokios.com/guides/pi.md) |
| Cursor | IDE | OpenAI | Yes | [/guides/cursor](https://docs.tokios.com/guides/cursor.md) |
| Zed | IDE | OpenAI · Anthropic | Unknown | [/guides/zed](https://docs.tokios.com/guides/zed.md) |
| JetBrains AI Assistant | IDE | OpenAI | Unknown | [/guides/jetbrains-ai-assistant](https://docs.tokios.com/guides/jetbrains-ai-assistant.md) |
| VS Code Copilot | IDE | OpenAI | Unknown | [/guides/vscode-copilot](https://docs.tokios.com/guides/vscode-copilot.md) |
| omp (oh-my-pi) | Terminal | OpenAI · Anthropic | Unknown | [/guides/omp](https://docs.tokios.com/guides/omp.md) |
| Hermes Agent | Terminal | OpenAI | Yes | [/guides/hermes-agent](https://docs.tokios.com/guides/hermes-agent.md) |

> [!NOTE]
>
> "Tool calling" means the agent can send `tools` in the request and parse `tool_calls` from the response. Whether it works end-to-end depends on both the agent and the local model — Qwen3-Coder-30B, Llama-3-70B, and Mistral all handle tool calls well on vLLM.

## IDE-based agents

These agents live inside your editor and send completions requests as you code. They all use the OpenAI Chat Completions API format, except where noted.

### Cline and Roo Code

Cline and its fork Roo Code both use an "OpenAI Compatible" provider. Set 3 fields in the extension settings:

1. **API Provider:** `OpenAI Compatible`
2. **Base URL:** `https://api.tokios.com/v1`
3. **API Key:** your `sk-tok-…` key
4. **Model ID:** your deployment name (for example, `qwen3-coder-tunnel`)

Both are configured entirely through the settings panel — there is no config file to edit. See the [Cline](https://docs.tokios.com/guides/cline.md) and [Roo Code](https://docs.tokios.com/guides/roo-code.md) guides for full setup.

### Continue

Continue reads `~/.continue/config.yaml`. Add a model entry:

```yaml theme={"theme":{"light":"github-light","dark":"css-variables"}}
models:
  - name: Tokios
    provider: openai
    model: qwen3-coder-tunnel
    apiBase: https://api.tokios.com/v1
    apiKey: sk-tok-YOUR_KEY
```

See the [Continue](https://docs.tokios.com/guides/continue.md) guide. Note that Continue supports both Chat Completions and Responses — use Chat Completions for local models.

### Cursor

Cursor's custom model override accepts a base URL and API key. Support varies by Cursor version — verify that custom base URL routing works in your build before relying on it:

1. Open **Cursor Settings** → **Models**.
2. Add a custom model with base URL `https://api.tokios.com/v1` and your `sk-tok-…` key.

See the [Cursor](https://docs.tokios.com/guides/cursor.md) guide for version-specific notes.

### Zed

Zed supports custom OpenAI-compatible providers in its `settings.json`:

```json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "language_models": {
    "openai_compatible": {
      "Tokios": {
        "api_url": "https://api.tokios.com/v1",
        "available_models": [
          { "name": "qwen3-coder-tunnel", "max_tokens": 128000 }
        ]
      }
    }
  }
}
```

The API key does **not** go in `settings.json` — enter it in Zed's LLM provider UI, or export `TOKIOS_API_KEY` (Zed derives the variable name from the provider id). See the [Zed](https://docs.tokios.com/guides/zed.md) guide.

### JetBrains AI Assistant

JetBrains AI Assistant supports BYOK (Bring Your Own Key) through its plugin settings. Enter the base URL and API key in the custom provider fields:

* **Base URL:** `https://api.tokios.com/v1`
* **API Key:** your `sk-tok-…` key

See the [JetBrains AI Assistant](https://docs.tokios.com/guides/jetbrains-ai-assistant.md) guide.

### VS Code Copilot

GitHub Copilot's BYOK support is emerging. When your VS Code version supports custom providers, point it at:

* **Base URL:** `https://api.tokios.com/v1`
* **API Key:** your `sk-tok-…` key

See the [VS Code Copilot](https://docs.tokios.com/guides/vscode-copilot.md) guide for the latest compatibility.

## Terminal agents

These run in your shell and use environment variables or config files to connect.

### Claude Code

Claude Code uses the **Anthropic** API format — not OpenAI. Set 2 environment variables:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3-coder-tunnel
```

Note: the base URL has no `/v1` suffix for Anthropic format.

See the [Claude Code](https://docs.tokios.com/guides/claude-code.md) guide for persisting these across sessions.

### OpenAI Codex

OpenAI Codex uses the **Responses API** (`wire_api=responses`). Set the base URL in `~/.codex/config.toml`:

```toml theme={"theme":{"light":"github-light","dark":"css-variables"}}
[model_providers.tokios]
name = "Tokios"
base_url = "https://api.tokios.com/v1"
env_key = "OPENAI_API_KEY"
wire_api = "responses"

[profiles.tokios]
model = "qwen3-coder-tunnel"
model_provider = "tokios"
disable_response_storage = true
```

`env_key` names the variable holding your key rather than the key itself, and `disable_response_storage` is required because Tokios serves the Responses surface statelessly. Then run:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
OPENAI_API_KEY=sk-tok-YOUR_KEY codex --profile tokios
```

See the [OpenAI Codex](https://docs.tokios.com/guides/openai-codex.md) guide.

### Aider

Aider reads the base URL and key from the environment, and needs the `openai/` model prefix:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
export OPENAI_API_BASE=https://api.tokios.com/v1
export OPENAI_API_KEY=sk-tok-YOUR_KEY
aider --model openai/qwen3-coder-tunnel
```

See the [Aider](https://docs.tokios.com/guides/aider.md) guide.

### OpenCode

OpenCode uses the AI SDK (Vercel) under the hood. Add a custom provider in `opencode.json`:

```json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "providers": {
    "tokios": {
      "type": "openai",
      "baseURL": "https://api.tokios.com/v1",
      "apiKey": "sk-tok-YOUR_KEY"
    }
  },
  "models": {
    "qwen3-coder": {
      "provider": "tokios",
      "model": "qwen3-coder-tunnel"
    }
  }
}
```

See the [OpenCode](https://docs.tokios.com/guides/opencode.md) guide.

### Goose

Goose's built-in `openai` provider accepts a custom host. Goose persists these to `~/.config/goose/config.yaml`:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
export GOOSE_PROVIDER=openai
export GOOSE_MODEL=qwen3-coder-tunnel
export OPENAI_HOST=https://api.tokios.com
export OPENAI_API_KEY=sk-tok-YOUR_KEY
```

For a named provider instead, drop a JSON file in `~/.config/goose/custom_providers/`.

See the [Goose](https://docs.tokios.com/guides/goose.md) guide.

### Pi and omp

Pi uses a custom provider with `api: openai-completions`. omp (oh-my-pi) inherits the same pattern — `providers` is a mapping keyed by provider name, and the keys are camelCase. omp reads `~/.omp/agent/models.yml`; Pi reads `~/.pi/agent/models.json`:

```yaml theme={"theme":{"light":"github-light","dark":"css-variables"}}
providers:
  tokios:
    baseUrl: https://api.tokios.com/v1
    api: openai-completions
    apiKey: sk-tok-YOUR_KEY
    models:
      - id: qwen3-coder-tunnel
        input: [text]
```

See the [Pi](https://docs.tokios.com/guides/pi.md) and [omp](https://docs.tokios.com/guides/omp.md) guides.

### Hermes Agent

Hermes Agent connects to any OpenAI-compatible `/v1/chat/completions` endpoint. It reads `~/.hermes/config.yaml`, with the provider keys nested under `model:`:

```yaml theme={"theme":{"light":"github-light","dark":"css-variables"}}
model:
  default: qwen3-coder-tunnel
  provider: custom
  base_url: https://api.tokios.com/v1
  api_key: sk-tok-YOUR_KEY
```

See the [Hermes Agent](https://docs.tokios.com/guides/hermes-agent.md) guide.

## Universal setup pattern

Every agent above follows the same 3-step pattern:

1. **Set the base URL** to `https://api.tokios.com/v1` (OpenAI) or `https://api.tokios.com` (Anthropic).
2. **Set the API key** to your `sk-tok-…` key.
3. **Set the model** to your registered deployment name.

If your agent isn't listed here but supports a custom OpenAI-compatible base URL, it almost certainly works. Try the universal env vars:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
export OPENAI_BASE_URL=https://api.tokios.com/v1
export OPENAI_API_KEY=sk-tok-YOUR_KEY
```

## Tool-calling compatibility by engine

Tool calling requires both agent support and backend support. Here's which local engines handle it:

| Engine | Tool calling | Notes |
| - | - | - |
| vLLM | Yes | Full OpenAI tool-call format; best compatibility |
| Ollama | Yes (v0.3+) | Via `/v1/chat/completions` with `tools` field |
| llama.cpp | Partial | `--chat-template` with tool format; varies by model |
| LM Studio | Yes | OpenAI-compatible tool-call format |
| SGLang | Yes | OpenAI-compatible; matches vLLM behavior |

> [!WARNING]
>
> Tool calling depends on the **model**, not just the engine. A model must be fine-tuned or prompted to emit structured `tool_calls` in its response. Qwen3-Coder-30B, Llama-3-70B-Instruct, and Mistral-7B-Instruct all handle tool calls reliably. [Muse Glimmer 30B](https://docs.tokios.com/models/muse-glimmer.md) is trained specifically for this: tool invocation against precise schemas, with failure recovery across multi-step workflows.

## Best local models for coding agents

| Model | Parameters | Best for | Engine recommendation |
| - | - | - | - |
| Muse Glimmer 30B | 29.6B dense | Agentic coding, tool calling, single 24 GB GPU | Ollama, llama.cpp |
| DeepSeek-V4-Flash-0731 | 284B target / 13B active MoE | Frontier coding agents, tool use, long repositories | vLLM or SGLang (multi-device) |
| Qwen3-Coder-30B | 30B dense | General coding, tool calling | llama.cpp (Vulkan), vLLM |
| Qwen3-32B | 32B dense | Reasoning-heavy code tasks | vLLM, Ollama |
| DeepSeek-R1 | 671B MoE | Complex reasoning, architecture | vLLM (requires 2+ GPUs) |
| Mistral-7B-Instruct | 7B dense | Fast completions, low-resource | Ollama, llama.cpp |
| Llama-3-70B-Instruct | 70B dense | Tool calling, multi-file edits | vLLM (multi-GPU) |

> [!TIP]
>
> For DGX Spark (128GB unified memory), Qwen3-Coder-30B at Q4\_K\_M is the sweet spot — it fits easily, generates fast, and handles tool calls well. For Strix Halo, use llama.cpp with Vulkan for the best token generation speed.

> [!TIP]
>
> `DeepSeek-V4-Flash-0731` is the current DeepSeek choice for coding agents. Its official model card reports major gains over the preview on Terminal Bench 2.1, DeepSWE, and Toolathlon-Verified. It still needs a multi-device host. Review the [local hardware guide](https://docs.tokios.com/blog/deepseek-v4-flash-local-hardware.md), then validate tool calls with the exact runtime, `reasoning_effort`, context length, and sampling settings you plan to use.

> [!TIP]
>
> [Muse Glimmer 30B](https://docs.tokios.com/models/muse-glimmer.md) is the option here that runs fully resident on a single 24 GB GPU, and Meta trained it for agentic tool use — but its benchmarks are vendor-reported with no independent reproduction yet. Use Meta's own sampling settings (`temperature 1.0`, `top_p 0.95`, `top_k 64`); reusing values tuned for another local model is a common cause of degraded output.

## One endpoint serves them all

You don't need to reconfigure your model when you switch agents. Register your deployment once in Tokios, create a `sk-tok-…` key, and point any agent at the same endpoint. Switch from Aider to Claude Code by changing 2 environment variables — your model, your connector, and your deployment stay the same.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Set up a connector and register your first deployment.

> [card] [Compatible clients](https://docs.tokios.com/guides/compatible-clients.md)
>
> The full matrix of 39 compatible clients, not just coding agents.

> [card] [Choose a local model](https://docs.tokios.com/guides/choose-a-local-model.md)
>
> Pick the right model for your hardware and use case.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create, scope, and rotate keys for teammates and agents.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.