> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Turn your RTX workstation into a private model endpoint

> Give the model on your RTX 5090, 4090, or 3090 desktop one OpenAI- and Anthropic-compatible endpoint that your laptop, coding agent, or teammates can reach.

Your RTX workstation already runs the model. The hard part is reaching it from your laptop, a coding agent, or a teammate's machine without exposing your gaming PC to the internet. Run the Tokios connector next to your runtime and you get one authenticated endpoint at `https://api.tokios.com` that speaks both OpenAI and Anthropic request formats, so Claude Code, an OpenAI SDK script, or anyone you invite can reach the box without touching your router.

## After it runs: the endpoint gap

Desktop RTX owners — 5090, 4090, 3090, and similar cards — typically get a model answering locally with Ollama, LM Studio, `llama.cpp`, or vLLM, then hit the next problem: using that model from somewhere other than the desktop itself.

* **An endpoint reachable from elsewhere.** The runtime binds to `127.0.0.1` by default. Reaching it from a laptop, a coding agent running elsewhere, or a teammate's machine means binding to `0.0.0.0`, forwarding a port on your home router, or standing up a tunnel — each with its own setup and no per-client auth in front of it.
* **API shape.** Claude Code speaks the Anthropic Messages format; Ollama, LM Studio, `llama.cpp`, and vLLM speak OpenAI Chat Completions. Bridging the two usually means writing and maintaining a small proxy.
* **A stable place to point tools.** Swapping between Ollama, LM Studio, `llama.cpp`, or vLLM on the same box, or changing which model is loaded, shouldn't mean reconfiguring every client that talks to it.

Tokios is the connect layer that sits on top of the runtime you already run — it does not replace Ollama, LM Studio, `llama.cpp`, or vLLM.

> [!NOTE]
>
> `DeepSeek-V4-Flash-0731` is not a typical single-workstation model. A pair of RTX PRO 6000 Blackwell cards has enough `192 GB` aggregate VRAM for its weight floor, but this 2-card path is community-validation territory. See [DeepSeek V4 Flash local hardware](https://docs.tokios.com/blog/deepseek-v4-flash-local-hardware.md) for the runtime and context caveats.

## Set it up on your workstation

**1. Start your runtime on the workstation**

Run your model server as you normally would, for example:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run llama3.1
```

or with LM Studio's local server, `llama.cpp`, or vLLM:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
vllm serve meta-llama/Llama-3.1-8B-Instruct
```

Confirm it answers on its own OpenAI-compatible `/v1` API before adding the connector.

**2. Get a claim code and pair the connector**

In the [dashboard](https://tokios.com/console), open the **Setup** tab and select **Start pairing** to get a one-time claim code shaped `TKS-XXXX-XXXX` (valid for about 15 minutes).

On Windows (PowerShell):

```powershell theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

On Linux:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Back in the dashboard, approve the device on the **Setup** or **Connectors** tab. The connector's token is delivered over the tunnel — you never paste it into a file.

**3. Point the connector at your runtime's /v1 endpoint**

The connector's upstream is the runtime's own OpenAI-compatible API on the workstation, not a port you forward on your router. Default ports: Ollama `11434`, LM Studio `1234`, `llama.cpp` `8080`, vLLM `8000` (all with `/v1` appended; for Ollama use its `/v1` OpenAI-compatible API, not the native `/api`). No inbound firewall rule or port forward is needed — the connector only dials out.

**4. Register a deployment**

On the **Models** tab, map your upstream model id to a public deployment name such as `rtx-coder`. Clients send this deployment name in the `model` field, not the upstream id.

**5. Create an API key**

On the **Keys** tab, create a key. Keys are `sk-tok-…` Bearer tokens, shown once, and can be scoped to specific deployment name patterns.

## Use it from your laptop or agent

Once the connector is paired and a deployment is registered, point your client at Tokios instead of your workstation's local network address.

```bash Claude Code theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model rtx-coder
```

```python OpenAI SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="rtx-coder",
    messages=[{"role": "user", "content": "Hello from my laptop"}],
)
```

> [!NOTE]
>
> Claude Code's model picker only lists `claude*` ids, so your Tokios deployment won't appear in the list — pass `--model rtx-coder` explicitly.

If you run more than one backend on the workstation — say Ollama for one model and vLLM for another — use the [connector config file](https://docs.tokios.com/reference/connector-config.md) to define multiple routes and upstreams instead of the single-upstream pairing flow. See [Model routing](https://docs.tokios.com/guides/model-routing.md) for how to register and switch between several deployments behind the same endpoint.

## How big a model can I run?

Your VRAM decides which model sizes and quantizations are realistic on a 5090, 4090, or 3090, and that's a hardware question, not a Tokios one. See [Choose a local model](https://docs.tokios.com/guides/model-fit.md) for sizing guidance, and [Test your deployment](https://docs.tokios.com/guides/benchmark-your-deployment.md) for how to compare latency and throughput across deployments once traffic is flowing through Tokios.

> [!TIP]
>
> Meta tested [Muse Glimmer 30B](https://docs.tokios.com/models/muse-glimmer.md) specifically on an RTX 5090, and reports its largest DFlash speedup there — 3.1x. The 5090's 32 GB runs the `-dflash` build comfortably. A 24 GB 3090 or 4090 runs the plain 4-bit build (18 GB) fully resident.

## Why one endpoint

* **Swap backends without touching client config.** Move from Ollama to LM Studio or vLLM on the same box, and every agent and script that already targets `rtx-coder` keeps working — you only change the connector's upstream, not the client.
* **No inbound ports on a home gaming PC.** The connector only makes outbound connections, so you don't forward a port on your router, bind the runtime to `0.0.0.0`, or maintain your own tunnel just to reach it from a laptop or coding agent.
* **Scoped keys instead of a shared bind address.** Give a teammate a key limited to specific deployment patterns instead of opening your runtime's port to your network.

## Next steps

> [card] [Run Qwen3.6-35B on RTX 5090](https://docs.tokios.com/guides/run-qwen36-35b-a3b-on-rtx-5090.md)
>
> Step-by-step deployment guide for the ideal RTX 5090 model.

> [card] [Team access](https://docs.tokios.com/use-cases/team-access.md)
>
> Full walkthrough for giving your team API access to your local models.

> [card] [What is a local LLM?](https://docs.tokios.com/guides/what-is-local-llm.md)
>
> Complete guide to running large language models locally.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Go from zero to a working connector, deployment, and API key.

> [card] [Install the connector](https://docs.tokios.com/setup/connector-install.md)
>
> Full connector install and pairing reference for all platforms.

> [card] [Use Claude Code](https://docs.tokios.com/guides/claude-code.md)
>
> Configure `ANTHROPIC_BASE_URL` and `ANTHROPIC_API_KEY` in detail.

> [card] [Connector config reference](https://docs.tokios.com/reference/connector-config.md)
>
> Serve multiple models from one workstation with `tokios-connector.json`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.