> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# All clients compatible with Tokios — complete matrix

> 39 clients confirmed working with Tokios: coding agents, chat apps, SDKs, observability tools, and gateways. All use standard OpenAI or Anthropic API format.

Tokios exposes standard OpenAI and Anthropic API surfaces, so any client that lets you set a custom base URL works. You change 2 things: the **base URL** and your **API key** (`sk-tok-…`). The `"model"` value is the public deployment name you registered (for example, `gemma-tunnel`).

## Quick verdict: 39 clients confirmed

| Category | Count | API format |
| - | - | - |
| Coding agents and IDEs | 15 | OpenAI or Anthropic |
| Chat applications | 10 | OpenAI |
| SDKs and frameworks | 8 | OpenAI or Anthropic |
| Observability and gateway tools | 3 | OpenAI |
| Evaluation tools | 1 | OpenAI |
| RAG platforms | 2 | OpenAI |

All 39 clients connect by pointing at `https://api.tokios.com/v1` (OpenAI) or `https://api.tokios.com` (Anthropic).

## Which base URL?

| Client speaks… | Base URL | Examples |
| - | - | - |
| OpenAI style | `https://api.tokios.com/v1` | Most clients below — SDKs, chat apps, coding agents |
| Anthropic style | `https://api.tokios.com` (no `/v1`) | Claude Code, Anthropic Python SDK |

## Coding agents and IDEs (15 clients)

These connect from inside your editor or terminal for code generation, editing, and tool-calling workflows.

| Client | API format | Tool calling | Streaming | Setup guide |
| - | - | - | - | - |
| Claude Code | Anthropic | Yes | Yes | [/guides/claude-code](https://docs.tokios.com/guides/claude-code.md) |
| OpenAI Codex | OpenAI (Responses) | Yes | Yes | [/guides/openai-codex](https://docs.tokios.com/guides/openai-codex.md) |
| Aider | OpenAI | Yes | Yes | [/guides/aider](https://docs.tokios.com/guides/aider.md) |
| Cline | OpenAI | Yes | Yes | [/guides/cline](https://docs.tokios.com/guides/cline.md) |
| Continue | OpenAI | Yes | Yes | [/guides/continue](https://docs.tokios.com/guides/continue.md) |
| Roo Code | OpenAI | Yes | Yes | [/guides/roo-code](https://docs.tokios.com/guides/roo-code.md) |
| OpenCode | OpenAI | Yes | Yes | [/guides/opencode](https://docs.tokios.com/guides/opencode.md) |
| Goose | OpenAI | Yes | Yes | [/guides/goose](https://docs.tokios.com/guides/goose.md) |
| Pi | OpenAI · Anthropic | Yes | Yes | [/guides/pi](https://docs.tokios.com/guides/pi.md) |
| Cursor | OpenAI | Yes | Yes | [/guides/cursor](https://docs.tokios.com/guides/cursor.md) |
| Zed | OpenAI · Anthropic | Unknown | Yes | [/guides/zed](https://docs.tokios.com/guides/zed.md) |
| JetBrains AI Assistant | OpenAI | Unknown | Yes | [/guides/jetbrains-ai-assistant](https://docs.tokios.com/guides/jetbrains-ai-assistant.md) |
| VS Code Copilot | OpenAI | Unknown | Yes | [/guides/vscode-copilot](https://docs.tokios.com/guides/vscode-copilot.md) |
| omp (oh-my-pi) | OpenAI · Anthropic | Unknown | Yes | [/guides/omp](https://docs.tokios.com/guides/omp.md) |
| Hermes Agent | OpenAI | Yes | Yes | [/guides/hermes-agent](https://docs.tokios.com/guides/hermes-agent.md) |

> [!NOTE]
>
> Claude Code uses `ANTHROPIC_BASE_URL` (no `/v1`). Zed, Pi, and omp can speak either surface — they default to OpenAI, and switching them to Anthropic also means dropping the `/v1`. Everything else above uses `OPENAI_BASE_URL` or equivalent with the `/v1` suffix. See the [Coding agent compatibility](https://docs.tokios.com/guides/coding-agent-compatibility.md) page for per-agent setup details.

## Chat applications (10 clients)

General-purpose chat UIs for conversational interaction with your local models.

| Client | OpenAI compat | Streaming | Tool calling | Vision | Setup guide |
| - | - | - | - | - | - |
| Open WebUI | Yes | Yes | Depends on model | Yes | [/guides/open-webui](https://docs.tokios.com/guides/open-webui.md) |
| LibreChat | Yes | Yes | Depends on model | Yes | [/guides/librechat](https://docs.tokios.com/guides/librechat.md) |
| Jan | Yes | Yes | Unknown | Unknown | [/guides/jan](https://docs.tokios.com/guides/jan.md) |
| Chatbox | Yes | Yes | Unknown | Unknown | [/guides/chatbox](https://docs.tokios.com/guides/chatbox.md) |
| Cherry Studio | Yes | Yes | Depends on model | Yes | [/guides/cherry-studio](https://docs.tokios.com/guides/cherry-studio.md) |
| LobeChat | Yes | Yes | Depends on model | Yes | [/guides/lobechat](https://docs.tokios.com/guides/lobechat.md) |
| SillyTavern | Yes | Yes | Unknown | Yes | [/guides/sillytavern](https://docs.tokios.com/guides/sillytavern.md) |
| Big-AGI | Yes | Yes | Unknown | Unknown | [/guides/big-agi](https://docs.tokios.com/guides/big-agi.md) |
| Msty | Yes | Yes | Unknown | Unknown | [/guides/msty](https://docs.tokios.com/guides/msty.md) |
| AnythingLLM | Yes | Yes | Depends on model | Yes | — |

> [!TIP]
>
> "Depends on model" means the chat app sends the `tools` field in the request, but the local model must support tool calling for it to work. Qwen3-Coder-30B and Mistral-7B-Instruct handle tool calls reliably on most engines.

### Per-client setup for chat apps

| Client | Base URL setting | API key setting |
| - | - | - |
| Open WebUI | **Admin Panel** → **Connections** → OpenAI API URL | API key field |
| LibreChat | `librechat.yaml` → `customEndpoints` | `librechat.yaml` → `apiKey` |
| Jan | **Settings** → **Remote Models** → custom endpoint | API key field |
| Chatbox | **Settings** → **Model Provider** → API Host | API key field |
| Cherry Studio | **Settings** → **Custom Provider** → Base URL | API key field |
| LobeChat | **Settings** → **Language Model** → OpenAI → API Proxy | API key field |
| SillyTavern | **API Connections** → **Text Completion** → API URL | API key field |
| Big-AGI | **Sources** → **Add OpenAI source** → endpoint URL | API key field |
| Msty | **Settings** → **AI Models** → custom endpoint URL | API key field |
| AnythingLLM | **Generic OpenAI** provider → base URL | API key field |

## SDKs and frameworks (8 clients)

Programmatic libraries for building applications on top of your Tokios endpoint.

| Client | API format | Responses API | Streaming | Tool calling | Vision |
| - | - | - | - | - | - |
| OpenAI Python SDK | OpenAI | Yes | Yes | Yes | Yes |
| OpenAI Node SDK | OpenAI | Yes | Yes | Yes | Yes |
| Anthropic Python SDK | Anthropic | No | Yes | Yes | Yes |
| LangChain (Python) | OpenAI | No | Yes | Yes | Yes |
| LlamaIndex | OpenAI | No | Yes | Yes | Unknown |
| Vercel AI SDK | OpenAI | No | Yes | Yes | Unknown |
| Pydantic AI | OpenAI | No | Yes | Yes | Unknown |
| LiteLLM | OpenAI | No | Yes | Yes | Yes |

### Code examples

```python OpenAI Python SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="gemma-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)
for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")
```

```javascript OpenAI Node SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.tokios.com/v1',
  apiKey: 'sk-tok-YOUR_KEY',
});

const response = await client.chat.completions.create({
  model: 'gemma-tunnel',
  messages: [{ role: 'user', content: 'Hello!' }],
  stream: true,
});
for await (const chunk of response) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
```

```python LangChain (Python) theme={"theme":{"light":"github-light","dark":"css-variables"}}
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gemma-tunnel",
    openai_api_base="https://api.tokios.com/v1",
    openai_api_key="sk-tok-YOUR_KEY",
)
print(llm.invoke("Summarize this code: print('hello')").content)
```

```python Anthropic Python SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
import anthropic

client = anthropic.Anthropic(
    base_url="https://api.tokios.com",
    api_key="sk-tok-YOUR_KEY",
)

message = client.messages.create(
    model="gemma-tunnel",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)
print(message.content[0].text)
```

```javascript Vercel AI SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';

const tokios = createOpenAICompatible({
  baseURL: 'https://api.tokios.com/v1',
  apiKey: 'sk-tok-YOUR_KEY',
  name: 'tokios',
});

const { text } = await generateText({
  model: tokios('gemma-tunnel'),
  prompt: 'Write a hello world in Rust.',
});
console.log(text);
```

> [!TIP]
>
> Clients that read `OPENAI_BASE_URL` or `ANTHROPIC_BASE_URL` pick Tokios up automatically once you export those variables in your shell:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> export OPENAI_BASE_URL=https://api.tokios.com/v1
> export OPENAI_API_KEY=sk-tok-YOUR_KEY
> ```

## Observability and gateway tools (4 clients)

Tools for tracing, proxying, or evaluating your Tokios endpoint.

| Client | Type | OpenAI compat | Streaming | Setup guide |
| - | - | - | - | - |
| Langfuse | Observability | Yes | Yes | [/guides/langfuse](https://docs.tokios.com/guides/langfuse.md) |
| LiteLLM | Gateway | Yes | Yes | — |
| Promptfoo | Evaluation | Yes | Yes | — |
| AionUi | Agent platform | Yes | Yes | [/guides/aionui](https://docs.tokios.com/guides/aionui.md) |

### Langfuse

Langfuse traces LLM calls by proxying requests through its SDK. Configure it as an OpenAI provider with your Tokios base URL:

* **LLM Connection base URL:** `https://api.tokios.com/v1`
* **API Key:** your `sk-tok-…` key

See the [Langfuse](https://docs.tokios.com/guides/langfuse.md) guide.

### LiteLLM

LiteLLM can proxy Tokios as an upstream provider. Add it to your `litellm_config.yaml`:

```yaml theme={"theme":{"light":"github-light","dark":"css-variables"}}
model_list:
  - model_name: gemma-tunnel
    litellm_params:
      model: openai/gemma-tunnel
      api_base: https://api.tokios.com/v1
      api_key: sk-tok-YOUR_KEY
```

### Promptfoo

Promptfoo evaluates LLM outputs. Add Tokios as an OpenAI-compatible provider in your `promptfooconfig.yaml`:

```yaml theme={"theme":{"light":"github-light","dark":"css-variables"}}
providers:
  - id: openai:gemma-tunnel
    config:
      apiBaseUrl: https://api.tokios.com/v1
      apiKeyEnvar: OPENAI_API_KEY
```

## RAG and agent platforms (2 clients)

| Client | OpenAI compat | Streaming | Tool calling | Vision |
| - | - | - | - | - |
| AnythingLLM | Yes | Yes | Depends on model | Yes |
| OpenClaw | Yes | Yes | Unknown | Unknown |

AnythingLLM wraps any OpenAI-compatible endpoint as a "Generic OpenAI" provider and adds document retrieval on top. OpenClaw connects as an OpenAI-compatible gateway client.

## Feature support matrix

A consolidated view of feature support across all 39 clients.

| Feature | Clients supporting it | Notes |
| - | - | - |
| SSE streaming | All 39 | Standard for real-time token output |
| Tool calling | 20 confirmed, 10 unknown | Depends on both client and model |
| Responses API | 3 (OpenAI Codex, OpenAI Python SDK, OpenAI Node SDK) | Newer OpenAI format |
| Vision (multimodal input) | 13 confirmed | Requires vision-capable model deployment |
| Custom base URL | All 39 | Required for Tokios connection |
| Anthropic Messages API | 6 (Claude Code, Anthropic SDK, LibreChat, Zed, Pi, omp) | Use base URL without `/v1` |

## The universal setup pattern

Every client above follows the same 3 steps:

1. **Set the base URL** to `https://api.tokios.com/v1` (OpenAI) or `https://api.tokios.com` (Anthropic).
2. **Set the API key** to your `sk-tok-…` key.
3. **Set the model** to your registered deployment name.

If a client isn't listed here but supports a custom OpenAI-compatible base URL, it almost certainly works. Test it with:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "your-deployment-name", "messages": [{"role": "user", "content": "Hello!"}]}'
```

## One API key works with all compatible clients

Create a single `sk-tok-…` key in the [Tokios dashboard](https://tokios.com/console/keys), and it works across every client on this page. Scope it to specific deployments with model-name patterns, or use `*` for full access. Rotate keys anytime — revoking cuts access immediately.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create, scope, and rotate sk-tok-… keys for teammates and agents.

> [card] [Coding agents](https://docs.tokios.com/guides/coding-agent-compatibility.md)
>
> Detailed setup for 15 coding agents that connect through Tokios.

> [card] [Remote access comparison](https://docs.tokios.com/guides/remote-access-comparison.md)
>
> How Tokios compares to Tailscale, Cloudflare Tunnel, and SSH tunnels.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.