> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Turn a Strix Halo or Ryzen AI Max box into a model endpoint

> Give the model on your AMD Strix Halo or Ryzen AI Max+ 395 box an OpenAI- and Anthropic-compatible endpoint that Claude Code and opencode can reach.

Your Strix Halo box already runs the model. The hard part is reaching it from your laptop, editor, or coding agent without opening inbound ports. Run the Tokios connector next to your runtime and you get one authenticated endpoint at `https://api.tokios.com` that speaks both OpenAI and Anthropic request formats, so Claude Code, opencode, or any OpenAI-compatible client can reach the box without touching firewall rules.

## After it runs: the endpoint gap

Framework Desktop, mini-PC, and other AMD Ryzen AI Max+ 395 ("Strix Halo") owners buy the 128GB unified-memory configuration specifically to run large local models. On this silicon, the community's general experience is that `llama.cpp` with the **Vulkan** backend is more reliable than ROCm/HIP.

Once the model answers on localhost, the next thing most people want is to plug in a coding agent — opencode, Claude Code, or a similar tool — against it. That means:

* **An OpenAI-compatible endpoint reachable from elsewhere.** The runtime binds to `127.0.0.1` on the Strix Halo box by default. Reaching it from a laptop or a coding agent running elsewhere means binding to `0.0.0.0`, opening a port, or standing up a tunnel — each with its own setup and no per-client auth in front of it.
* **API shape.** Claude Code speaks the Anthropic Messages format; `llama.cpp`, Ollama, and LM Studio speak OpenAI Chat Completions. Bridging the two usually means writing and maintaining a small proxy.
* **A stable place to point tools.** Swapping between `llama.cpp`, Ollama, or LM Studio on the same box, or changing which model is loaded, shouldn't mean reconfiguring every client that talks to it.

Tokios is the connect layer that sits on top of the runtime you already run — it does not replace `llama.cpp`, Ollama, LM Studio, vLLM, or Lemonade Server.

> [!WARNING]
>
> 2 Strix Halo boxes can have enough aggregate memory for `DeepSeek-V4-Flash-0731` GGUF weights, but the required `llama.cpp` RPC path is fragile, insecure, and proof-of-concept. Do not treat it as a production deployment. See [DeepSeek V4 Flash local hardware](https://docs.tokios.com/blog/deepseek-v4-flash-local-hardware.md) for the experimental boundary.

## Set it up on your Strix Halo box

**1. Start your runtime on the box**

Run your model server as you normally would, for example with `llama.cpp` on the Vulkan backend:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
llama-server -m your-model.gguf --port 8080
```

or with LM Studio's local server, or Ollama:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run llama3.1
```

**Lemonade Server** is another option — it auto-detects your Strix Halo hardware and selects the Vulkan backend for you, so you skip the manual backend selection:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
lemonade pull your-model
lemonade run your-model
```

Lemonade Server listens on port `13305` by default and exposes an OpenAI-compatible API at `/v1`. It also supports AMD Ryzen AI NPUs, which no other backend here does. See [Lemonade Server remote access](https://docs.tokios.com/guides/lemonade-server-remote-access.md) for the full setup guide.

Confirm it answers on its own OpenAI-compatible `/v1` API before adding the connector.

**2. Get a claim code and pair the connector**

In the [dashboard](https://tokios.com/console), open the **Setup** tab and select **Start pairing** to get a one-time claim code shaped `TKS-XXXX-XXXX` (valid for about 15 minutes).

On Linux:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

On Windows (PowerShell):

```powershell theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

Back in the dashboard, approve the device on the **Setup** or **Connectors** tab. The connector's token is delivered over the tunnel — you never paste it into a file.

**3. Point the connector at your runtime's /v1 endpoint**

The connector's upstream is the runtime's own OpenAI-compatible API on the box, not a port you expose externally. Default ports: `llama.cpp` `8080`, LM Studio `1234`, Ollama `11434`, Lemonade Server `13305` (all with `/v1` appended; for Ollama use its `/v1` OpenAI-compatible API, not the native `/api`). No inbound firewall rule is needed — the connector only dials out.

**4. Register a deployment**

On the **Models** tab, map your upstream model id to a public deployment name such as `strix-coder`. Clients send this deployment name in the `model` field, not the upstream id.

**5. Create an API key**

On the **Keys** tab, create a key. Keys are `sk-tok-…` Bearer tokens, shown once, and can be scoped to specific deployment name patterns.

## Use it from your coding agent

Once the connector is paired and a deployment is registered, point your coding agent at Tokios instead of the box's local address.

```bash Claude Code theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model strix-coder
```

```python OpenAI SDK / opencode theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="strix-coder",
    messages=[{"role": "user", "content": "Hello from my laptop"}],
)
```

> [!NOTE]
>
> Claude Code's model picker only lists `claude*` ids, so your Tokios deployment won't appear in the list — pass `--model strix-coder` explicitly.

If you run more than one backend on the box — say `llama.cpp` for one model and Lemonade Server or Ollama for another — use the [connector config file](https://docs.tokios.com/reference/connector-config.md) to define multiple routes and upstreams instead of the single-upstream pairing flow. See [Model routing](https://docs.tokios.com/guides/model-routing.md) for how to register and switch between several deployments behind the same endpoint.

## Why one endpoint

* **Swap backends without touching agent config.** Move from `llama.cpp` to LM Studio or Ollama on the same box, and every agent that already targets `strix-coder` keeps working — you only change the connector's upstream, not the client.
* **Scoped keys instead of a shared bind address.** Give a teammate a key limited to specific deployment patterns instead of opening the runtime's port to your network.
* **No inbound ports on the box.** The connector only makes outbound connections, so you don't bind the runtime to `0.0.0.0` or maintain your own tunnel just to reach it from a laptop or coding agent.

## Can Strix Halo run Kimi K3?

No. Strix Halo's 128 GB unified memory pool (\~90–110 GB available after OS overhead) is far below what Kimi K3 requires. K3 has **2.8 trillion total parameters** and needs about **1.4 TB** at its native MXFP4 precision (4 bits per parameter) — before block scales, runtime buffers, activations, and KV cache. Even if you could use every byte of the 128 GB pool for weights, you would still be roughly **1.27 TB short**.

Moonshot recommends serving K3 on a supernode with 64 or more accelerators. For the full memory calculation and cluster sizing, see the [Kimi K3 hardware deep dive](https://docs.tokios.com/blog/kimi-k3-hardware-deep-dive.md).

> [!TIP]
>
> Looking for a model that actually fits Strix Halo? See [what to run instead of Kimi K3](https://docs.tokios.com/models/kimi-k3-alternatives.md) — the best coding and general model for 128 GB unified memory.

## Next steps

> [card] [Team access](https://docs.tokios.com/use-cases/team-access.md)
>
> Full walkthrough for giving your team API access to your local models.

> [card] [What is a local LLM?](https://docs.tokios.com/guides/what-is-local-llm.md)
>
> Complete guide to running large language models locally.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Go from zero to a working connector, deployment, and API key.

> [card] [Install the connector](https://docs.tokios.com/setup/connector-install.md)
>
> Full connector install and pairing reference for all platforms.

> [card] [Use Claude Code](https://docs.tokios.com/guides/claude-code.md)
>
> Configure `ANTHROPIC_BASE_URL` and `ANTHROPIC_API_KEY` in detail.

> [card] [Connector config reference](https://docs.tokios.com/reference/connector-config.md)
>
> Serve multiple models from one box with `tokios-connector.json`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.