> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Use a vLLM server from anywhere — safely

> Reach your vLLM OpenAI-compatible server from other machines, agents, or teammates without exposing port 8000 — using an outbound-only connector and API key.

By default, vLLM's OpenAI-compatible server listens on `http://127.0.0.1:8000/v1` — it isn't reachable from another machine unless you bind it wider or forward the port. vLLM does ship an `--api-key` flag, but vLLM's own docs are explicit that you shouldn't rely on it exclusively. Keep vLLM on localhost and run the Tokios connector next to it instead: the connector dials out to `https://api.tokios.com`, and your tools, teammates, or agents call one authenticated endpoint from anywhere.

> [!NOTE]
>
> **Advanced runtime.** vLLM is a Python GPU server aimed at high throughput, with more setup and tuning than a desktop tool. If you just want the easiest way to serve a model, start with [Ollama](https://docs.tokios.com/guides/ollama-remote-access.md) or [LM Studio](https://docs.tokios.com/guides/lm-studio-remote-access.md) — Tokios connects to any of them the same way.

## vLLM's `--api-key` isn't the whole story

`vllm serve --api-key <key>` requires a bearer token on requests, which is a reasonable first layer. The catch, per vLLM's own security documentation, is scope: the key check only covers `/v1` paths, and other endpoints on the same server are not protected by it. vLLM's documented recommendation is to put a reverse proxy in front that allowlists only the endpoints you intend to expose — `--api-key` alone isn't meant to be your perimeter.

> [!NOTE]
>
> This isn't a knock on vLLM — `--api-key` does what it says for the paths it covers. It's solving a different problem than "authenticate and route requests to this server from anywhere." Tokios is purpose-built for that: it never opens an inbound port on your vLLM host at all, so there's no listening surface to allowlist in the first place.

## The outbound-only way

**1. Run vLLM locally, as usual**

Keep vLLM on `127.0.0.1` — don't bind it wider than loopback. Start the OpenAI-compatible server:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
vllm serve mistralai/Mistral-7B-Instruct-v0.2
```

Leave it listening on the default `http://127.0.0.1:8000`.

**2. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing**. Tokios shows a one-time **claim code** like `TKS-XXXX-XXXX`, valid for about 15 minutes.

On the machine running vLLM, run the install one-liner for your OS:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Already have `tokios-connector` installed? Pass the code directly instead:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
tokios-connector TKS-XXXX-XXXX
```

Back in the console, **approve** the device on the **Setup** or **Connectors** tab. The connector's token is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

**3. Point the connector at vLLM's /v1 API**

vLLM's OpenAI-compatible server already speaks the surface Tokios expects. For the basic single-model path, the console's Setup wizard uses `http://127.0.0.1:8000/v1` as the upstream automatically. If you're serving multiple models through a `tokios-connector.json` config, set the upstream `BaseUrl` explicitly:

```json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:8000/v1"
  }
}
```

See [Connector config](https://docs.tokios.com/reference/connector-config.md) for the full multi-model file format.

**4. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the vLLM model and choose a **public name** — for example, `mistral-tunnel`. That public name, not vLLM's local model id, is what clients send in the `model` field.

**5. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Optionally scope it to specific deployments with model-name patterns (comma-separated globs, `*` for all). Copy the `sk-tok-…` key now — it's shown only once.

> [!TIP]
>
> Serving LoRA adapters? vLLM can serve multiple adapters from one process when you start it with `--enable-lora` and `--lora-modules`, addressing each adapter by name in the `model` field of the request. Register each adapter as its own Tokios deployment with its own public name, the same way you'd register any other model — see [Model routing](https://docs.tokios.com/guides/model-routing.md) for how the gateway maps a public deployment name to the id you send upstream.

## Use it from anywhere

Once you have a deployment name and an API key, any OpenAI- or Anthropic-compatible client can reach your vLLM model — from another machine, a teammate's laptop, or a hosted agent.

```bash curl theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

```python Python (OpenAI SDK) theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="mistral-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")
```

```bash Claude Code theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model mistral-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`mistral-tunnel` above), never vLLM's local model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) for persisting the environment variables across sessions.

## FAQ

> [accordion] Does streaming work?
>
> Yes. Set `"stream": true` and Tokios returns standard server-sent events (SSE) in OpenAI or Anthropic format, depending on which API surface you called.

> [accordion] Can I serve more than one model or adapter through vLLM?
>
> Yes. Register additional deployments in the [Models](https://tokios.com/console/models) tab, or use a `tokios-connector.json` config file to define multiple upstreams and routes for a single connector — useful if you run more than one vLLM process, or one process serving several LoRA adapters via `--lora-modules`. See [Connector config](https://docs.tokios.com/reference/connector-config.md).

> [accordion] Do I still need vLLM's --api-key flag?
>
> You can keep it as defense in depth on the loopback connection between the connector and vLLM, but it isn't what authenticates callers reaching you through Tokios. Client requests to `https://api.tokios.com` are authenticated with your `sk-tok-…` key; the connector is what's allowed to reach `127.0.0.1:8000` at all.

> [accordion] Can I share access with teammates without exposing vLLM directly?
>
> Yes. Create a separate `sk-tok-…` key per teammate in the [Keys](https://tokios.com/console/keys) tab, and scope each key to only the deployments they need. Revoking a key stops it from authenticating immediately, so you can cut off access without touching vLLM or the connector.

## Your model never leaves your machine — your endpoint works everywhere

vLLM keeps doing exactly what it does today, on localhost, with no inbound ports and no endpoints exposed beyond `/v1`. The connector only dials out. Everything past that — auth, routing, streaming — is one endpoint you can call from any machine, teammate, or agent you choose to give a key to.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create, scope, and rotate `sk-tok-…` keys for teammates and agents.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.