> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Choose a local model and backend

> Choose a Tokios deployment setup based on your workflow, local hardware, backend, and performance needs.

Use this guide before you register your first deployment or when you want to compare several local models. Tokios does not choose the model for you; it gives each local model a stable deployment name that clients can call.

> [!TIP]
>
> Once you pick a model and backend, [register your first deployment with Tokios](https://docs.tokios.com/quickstart.md) — 1 authenticated endpoint at `https://api.tokios.com`, about 5 minutes.

## Start with the workflow

| Workflow | Optimize for | Good first check |
| - | - | - |
| Coding agents | Instruction following, repository reasoning, streaming stability | Ask the agent to inspect and modify a small project |
| Chat apps | Low latency and helpful answers | Send short prompts and compare time to first token |
| SDK experiments | API compatibility and predictable responses | Run the same request from `curl` and your SDK |
| Shared local endpoint | Stability and clear deployment names | Register one deployment per model and test `/v1/models` |

## Choose a backend

Tokios forwards requests to a backend that serves an OpenAI-compatible Chat Completions API. Start with the backend you already use. If you are choosing from scratch, use the simplest option for your operating system and hardware.

| Backend | Often useful when... | Default Tokios upstream base |
| - | - | - |
| Ollama | You want a quick CLI-first local model setup | `http://127.0.0.1:11434/v1` |
| LM Studio | You prefer a desktop app and visual model management | `http://127.0.0.1:1234/v1` |
| llama.cpp | You run GGUF models directly | `http://127.0.0.1:8080/v1` |
| vLLM | You need a server-oriented GPU runtime | `http://127.0.0.1:8000/v1` |

See [Backends](https://docs.tokios.com/setup/supported-backends.md) for current setup examples.

## Name deployments for clients

The model id your backend serves does not need to be the name your clients use. Give each Tokios deployment a short public name that describes its role.

| Upstream model id | Deployment name | Why it helps |
| - | - | - |
| `qwen2.5-coder` | `coder-local` | Coding agents can keep one stable model setting |
| `gemma4:e4b` | `chat-local` | Chat clients do not need to know the backend id |
| `mistral-small` | `fast-draft` | You can signal the intended use in the name |
| `muse-glimmer:30b-q4_K_M` | `agent-local` | Keeps agent clients on one name if you swap quantization tags later |

> [!TIP]
>
> Keep deployment names stable. If you move a model to a different backend or machine later, update the deployment or connector route instead of changing every client.

## Compare models carefully

Change one thing at a time. Keep the same prompt, client, backend settings, and hardware when you compare two deployments. Record both speed and answer quality.

| Metric | How to use it |
| - | - |
| Time to first token | Measures how quickly a request starts producing output |
| Tokens per second | Measures generation speed after output starts |
| Total task time | Matters most for coding agents and multi-step workflows |
| Completion quality | Captures whether the model actually did the job |
| Stability | Catches backend restarts, timeouts, and streaming failures |

For a repeatable test flow, use [Test your deployment](https://docs.tokios.com/guides/benchmark-your-deployment.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.