> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# You fine-tuned a model. Now make it an API.

> Turn a fine-tuned model already running in Ollama or llama.cpp into a keyed https://api.tokios.com endpoint your tools, teammates, or coding agents can call.

If you've already exported your fine-tune to GGUF and it's running in Ollama or llama.cpp, you have a `localhost` endpoint but no way to call it from anywhere else. Keep the fine-tune exactly where it is, run the Tokios connector next to it, and register a deployment name for it — you get a real `https://api.tokios.com` endpoint, secured with an `sk-tok-…` key, that any OpenAI- or Anthropic-compatible client can call.

## The last mile after fine-tuning

The documented fine-tuning path — for example with [Unsloth](https://unsloth.ai) — gets you as far as exporting a GGUF file and loading it with `ollama create` from a Modelfile, or serving it with llama.cpp's server; check the exact export and `ollama create` command syntax against your Unsloth version. Both of those get your fine-tune answering requests on `127.0.0.1`.

That's where the documentation usually stops. A `localhost` endpoint works from the same machine, in the same terminal, for as long as that process is running — it doesn't work from a teammate's laptop, a CI job, a hosted agent, or your own phone. Turning it into something callable from anywhere is the undocumented last mile, and it's the specific problem Tokios solves: it doesn't touch how the fine-tune was made or how it's served, it just makes the endpoint reachable and authenticated.

> [!NOTE]
>
> Tokios doesn't fine-tune, quantize, or export anything, and it doesn't fix chat-template mismatches. It picks up wherever your runtime already exposes an OpenAI-compatible `/v1` API and makes that endpoint callable outside `localhost`. Getting the fine-tune serving correctly, with the right chat template, is your runtime's job — see [Choose a local model](https://docs.tokios.com/guides/choose-a-local-model.md) if you're still deciding how to serve it.

## Steps

**1. Confirm your fine-tune is serving locally**

This guide assumes your GGUF is already loaded and answering requests through Ollama or llama.cpp's server on localhost. For Ollama, that's typically a model you loaded with `ollama create <name> -f Modelfile` and can already query with `ollama run <name>`. For llama.cpp, that's `llama-server` listening on its default port — confirm the exact port against your install. Either way, don't change how it's bound — leave it on `127.0.0.1`.

**2. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing**. Tokios shows a one-time **claim code** like `TKS-XXXX-XXXX`, valid for about 15 minutes.

On the machine running your fine-tune, run the install one-liner for your OS:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Already have `tokios-connector` installed? Pass the code directly instead:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
tokios-connector TKS-XXXX-XXXX
```

Back in the console, **approve** the device on the **Setup** or **Connectors** tab. The connector's token is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

**3. Point the connector at your runtime's /v1 API**

Tokios talks to whatever OpenAI-compatible `/v1` surface your runtime exposes — it doesn't matter that the model underneath is a fine-tune rather than a stock checkpoint. For the basic single-model path, the console's Setup wizard fills in the upstream automatically based on the runtime you pick (`http://127.0.0.1:11434/v1` for Ollama, `http://127.0.0.1:8080/v1` for llama.cpp). If you're serving multiple models or adapters through a `tokios-connector.json` config, set the upstream `BaseUrl` explicitly:

```json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:11434/v1"
  }
}
```

See [Connector config](https://docs.tokios.com/reference/connector-config.md) for the full multi-model file format.

**4. Register a deployment name for your fine-tune**

In the [Models](https://tokios.com/console/models) tab, register your fine-tune and choose a **public name** — for example, `my-finetune-tunnel`. That public name, not the local model id you gave it in `ollama create` or llama.cpp, is what clients send in the `model` field. It's also what tells your fine-tune apart from any base models or other fine-tunes you serve through the same connector.

**5. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Optionally scope it to specific deployments with model-name patterns (comma-separated globs, `*` for all) — useful if you want a key that can only reach your fine-tune and nothing else. Copy the `sk-tok-…` key now — it's shown only once.

## Use your fine-tune from a coding agent

Once you have a deployment name and an API key, your fine-tune is reachable the same way any hosted model is — including from Claude Code or any OpenAI SDK client.

```bash Claude Code theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model my-finetune-tunnel
```

```python Python (OpenAI SDK) theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="my-finetune-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")
```

```bash curl theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-finetune-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

> [!TIP]
>
> Claude Code's model picker only lists `claude*` ids, so your fine-tune's deployment name won't show up in it. Pass `--model my-finetune-tunnel` explicitly, every time. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) for persisting the environment variables across sessions.

> [!WARNING]
>
> If responses look malformed, truncated, or off-tone, that's almost always a chat-template mismatch between how the model was fine-tuned and how the runtime is prompting it — not something in the Tokios path. Tokios passes requests straight through to your runtime's `/v1` API and doesn't rewrite prompts or templates. Fix the Modelfile or server prompt template at the runtime level, then re-test.

## FAQ

> [accordion] Does Tokios care that this is a fine-tune and not a base model?
>
> No. Tokios only sees an OpenAI-compatible `/v1` endpoint and a deployment name you mapped to it. Whether the weights behind that endpoint are a stock checkpoint, a merged fine-tune, or a LoRA adapter makes no difference to the connector or the gateway.

> [accordion] Can I serve a base model and a fine-tune side by side?
>
> Yes. Register each as its own deployment in the [Models](https://tokios.com/console/models) tab, or define multiple upstreams and routes in a `tokios-connector.json` config for one connector serving several models. See [Connector config](https://docs.tokios.com/reference/connector-config.md) and [Model routing](https://docs.tokios.com/guides/model-routing.md).

> [accordion] Does streaming work with a fine-tune the same way?
>
> Yes. Set `"stream": true` and Tokios returns standard server-sent events (SSE) in OpenAI or Anthropic format, depending on which API surface you called — the same as for any other deployment.

> [accordion] My fine-tune's chat template is wrong — can Tokios fix that?
>
> No. Chat-template correctness — matching the special tokens and turn format the fine-tune was trained on — is entirely your runtime's responsibility (Ollama's Modelfile `TEMPLATE`, or llama.cpp's server prompt template). Tokios forwards the request body as-is to your runtime's `/v1` API.

## Your fine-tune never leaves your machine — your endpoint works everywhere

The weights stay on your hardware, in the same Ollama or llama.cpp process you already trust. The connector only dials out — no inbound port, no exposed API on your machine. Everything past that — auth, routing, streaming — is one endpoint you can call from any machine, teammate, or agent you choose to give a key to.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create, scope, and rotate `sk-tok-…` keys for teammates and agents.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.