> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Use a llama.cpp server from anywhere — safely

> Reach your llama.cpp OpenAI-compatible server from other machines or teammates without exposing port 8080 — via an outbound-only connector and a real API key.

By default, `llama-server` listens on `http://127.0.0.1:8080/v1` — it isn't reachable from another machine unless you bind it wider or forward the port. `llama-server` does ship an optional `--api-key` flag, but it's off by default, and putting TLS in front of it is left entirely to you. Keep `llama-server` on localhost and run the Tokios connector next to it instead: the connector dials out to `https://api.tokios.com`, and your tools, teammates, or agents call one authenticated endpoint from anywhere.

## `llama-server`'s auth is optional and off by default

`llama-server` is the OpenAI-compatible server that ships with llama.cpp, built to serve a GGUF model over HTTP. Run it and, unless you say otherwise, it accepts requests from anyone who can reach the port — no credential required:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
llama-server -m model.gguf --port 8080
```

You can pass `--api-key <key>` to require a bearer token, and that's a reasonable first layer. But there's no default, and TLS termination is a separate, manual concern — `llama-server` doesn't set that up for you.

> [!NOTE]
>
> This isn't a knock on `llama-server` — it's a lightweight, embeddable inference server, and leaving auth and TLS as opt-in flags keeps it simple to run on a trusted machine or behind your own infrastructure. Tokios is purpose-built for a different problem: turning that local endpoint into an authenticated API you can call from anywhere, without opening an inbound port or hand-rolling TLS. llama.cpp is a first-class Tokios backend.

## The outbound-only way

**1. Run llama-server locally, as usual**

Keep `llama-server` on `127.0.0.1` — don't bind it wider than loopback. Start it with your GGUF model:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
llama-server -m model.gguf --port 8080
```

Leave it listening on the default `http://127.0.0.1:8080`.

**2. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing**. Tokios shows a one-time **claim code** like `TKS-XXXX-XXXX`, valid for about 15 minutes.

On the machine running `llama-server`, run the install one-liner for your OS:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Already have `tokios-connector` installed? Pass the code directly instead:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
tokios-connector TKS-XXXX-XXXX
```

Back in the console, **approve** the device on the **Setup** or **Connectors** tab. The connector's token is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

**3. Point the connector at llama-server's /v1 API**

`llama-server` already speaks the OpenAI Chat Completions surface Tokios expects. For the basic single-model path, the console's Setup wizard uses `http://127.0.0.1:8080/v1` as the upstream automatically. If you're serving multiple models through a `tokios-connector.json` config, set the upstream `BaseUrl` explicitly:

```json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:8080/v1"
  }
}
```

See [Connector config](https://docs.tokios.com/reference/connector-config.md) for the full multi-model file format.

**4. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the llama.cpp model and choose a **public name** — for example, `llama-tunnel`. That public name, not the GGUF file or its internal model id, is what clients send in the `model` field.

**5. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Optionally scope it to specific deployments with model-name patterns (comma-separated globs, `*` for all). Copy the `sk-tok-…` key now — it's shown only once.

## Use it from anywhere

Once you have a deployment name and an API key, any OpenAI- or Anthropic-compatible client can reach your llama.cpp model — from another machine, a teammate's laptop, or a hosted agent.

```bash curl theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

```python Python (OpenAI SDK) theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="llama-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")
```

```bash Claude Code theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model llama-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`llama-tunnel` above), never the GGUF file name or a local model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) for persisting the environment variables across sessions.

## FAQ

> [accordion] Does streaming work?
>
> Yes. Set `"stream": true` and Tokios returns standard server-sent events (SSE) in OpenAI or Anthropic format, depending on which API surface you called.

> [accordion] I fine-tuned a GGUF model — can I serve that?
>
> Yes. `llama-server` serves whatever GGUF file you point `-m` at, fine-tunes included. Register it as its own deployment with its own public name, the same way you'd register any other model. See [Serve your fine-tune](https://docs.tokios.com/guides/serve-your-fine-tune.md) for the end-to-end path from a fine-tuned model to a Tokios deployment.

> [accordion] Can I serve more than one GGUF model?
>
> Yes. Register additional deployments in the [Models](https://tokios.com/console/models) tab, or use a `tokios-connector.json` config file to define multiple upstreams and routes for a single connector — useful if you run more than one `llama-server` process, each on its own port. See [Connector config](https://docs.tokios.com/reference/connector-config.md).

> [accordion] Do I still need llama-server's --api-key flag?
>
> You can keep it as defense in depth on the loopback connection between the connector and `llama-server`, but it isn't what authenticates callers reaching you through Tokios. Client requests to `https://api.tokios.com` are authenticated with your `sk-tok-…` key; the connector is what's allowed to reach `127.0.0.1:8080` at all.

## Your model never leaves your machine — your endpoint works everywhere

`llama-server` keeps doing exactly what it does today, on localhost, with no inbound ports and no manual TLS to configure. The connector only dials out. Everything past that — auth, routing, streaming — is one endpoint you can call from any machine, teammate, or agent you choose to give a key to.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create, scope, and rotate `sk-tok-…` keys for teammates and agents.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.