> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Serve models from your homelab GPU server to your team

> Give the models on your headless homelab GPU box a single authenticated endpoint your teammates and apps can reach, even behind CGNAT, with no inbound ports.

Your homelab box already serves models fine on its own LAN. The hard part is letting your teammates — or an app that isn't sitting on the same network — reach it, especially when your ISP's CGNAT makes port forwarding impossible anyway. Run the `tokios-connector` next to your runtime and you get one authenticated endpoint at `https://api.tokios.com` that anyone you invite can call, with no inbound ports, no router changes, and no shared raw access to the box.

## The homelab reachability problem

A homelab GPU server is usually headless, tucked in a closet or rack, and running Ollama, vLLM, or llama.cpp for whoever's on the LAN. Sharing it beyond that runs into a few walls at once:

* **No public IP to forward to.** If your ISP puts you behind carrier-grade NAT (CGNAT), there's no unique inbound address for a port-forward rule to attach to in the first place — see [Serve a local LLM from behind CGNAT](https://docs.tokios.com/guides/llm-behind-cgnat.md) for why that's a router setting you can't fix yourself.
* **You don't want to expose it even if you could.** Binding the runtime to `0.0.0.0` and forwarding a port turns your GPU box into an unauthenticated service on the open internet. Anyone who finds the port can send it requests.
* **Sharing access means sharing everything.** Handing a teammate your Tailscale node or a raw bound port gives them (and anything on their machine) the same access you have — there's no per-person key, and no way to cut off one person without disrupting everyone else.

Tokios is the connect layer that sits on top of the runtime you already run — it doesn't replace Ollama, vLLM, or llama.cpp.

## Set it up on your server

**1. Run your runtime(s) on the server**

Start your model server as you normally would, for example:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama serve
```

or

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
vllm serve meta-llama/Llama-3.1-8B-Instruct
```

Leave it listening on its default loopback address. Confirm it answers on its own OpenAI-compatible `/v1` API before adding the connector.

**2. Get a claim code and pair the connector**

In the [dashboard](https://tokios.com/console), open the **Setup** tab and select **Start pairing** to get a one-time claim code shaped `TKS-XXXX-XXXX` (valid for about 15 minutes). On the server, run:

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Already have `tokios-connector` installed? Pass the code directly instead:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
tokios-connector TKS-XXXX-XXXX
```

Back in the dashboard, **approve** the device on the **Setup** or **Connectors** tab. The connector's token (`ct-tok-…`) is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

**3. Point the connector at your upstream(s)**

The connector's upstream is your runtime's own OpenAI-compatible API on the server, not a port you expose externally. Default ports: Ollama `11434`, llama.cpp `8080`, vLLM `8000`, LM Studio `1234` — all with `/v1` appended. For Ollama, point at its `/v1` OpenAI-compatible API, not the native `/api`. No inbound firewall rule is needed — the connector only dials out.

**4. Register one deployment per model**

On the **Models** tab, map each upstream model id (for example `llama3.1`) to a public deployment name your team will use, such as `homelab-llama`. Clients send this deployment name in the `model` field of every request, never the upstream id.

**5. Create per-teammate scoped keys**

On the **Keys** tab, click **Create key** for each teammate or app. Scope each key to specific deployment name patterns (comma-separated globs, `*` for all) so a teammate only reaches the models you intend. Keys are `sk-tok-…` Bearer tokens, shown once at creation — copy it before leaving the page.

## Serve several models at once

If your box runs more than one model, or splits traffic across Ollama, llama.cpp, and vLLM at the same time, use the [connector config file](https://docs.tokios.com/reference/connector-config.md) instead of the single-upstream pairing flow. In `tokios-connector.json`, define `Routes` and `Upstreams` under the `Connector` section — one `Upstream` per backend, each with its own `BaseUrl` (including `/v1`) — and pass the file with `--config`. See [Model routing](https://docs.tokios.com/guides/model-routing.md) for how to register and switch between the resulting deployments behind the same endpoint.

## Use it from anywhere

```bash curl theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "homelab-llama",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello from the team laptop"}]
  }'
```

```python OpenAI SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="homelab-llama",
    messages=[{"role": "user", "content": "Hello from the team laptop"}],
)
```

> [!TIP]
>
> The `model` field is always the registered **deployment** name, never the upstream model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) for calling your homelab server from `claude --model homelab-llama`.

## Why one endpoint for a team

* **No inbound ports on the box, ever.** The connector only makes outbound connections, so teammates never need a VPN peer, an SSH tunnel, or a bound `0.0.0.0` port to reach the server — this holds whether or not you're behind CGNAT.
* **Revoke one teammate's key without disrupting anyone else.** Because each person has their own `sk-tok-…` key, disabling or deleting one on the **Keys** tab stops that person immediately and leaves every other key working.
* **Swap models without touching client config.** Point a deployment name at a different upstream model or backend, and every teammate and script already targeting `homelab-llama` keeps working.

## Next steps

> [card] [Team access](https://docs.tokios.com/use-cases/team-access.md)
>
> Full walkthrough for giving your team API access to your local models.

> [card] [What is a local LLM?](https://docs.tokios.com/guides/what-is-local-llm.md)
>
> Complete guide to running large language models locally.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Go from zero to a working connector, deployment, and API key.

> [card] [LLM behind CGNAT](https://docs.tokios.com/guides/llm-behind-cgnat.md)
>
> Why port forwarding fails on residential ISPs, and how the connector sidesteps it.

> [card] [Connector config reference](https://docs.tokios.com/reference/connector-config.md)
>
> Serve multiple models from one server with `tokios-connector.json`.

> [card] [Model routing](https://docs.tokios.com/guides/model-routing.md)
>
> Register and switch between several deployments behind one endpoint.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.