> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Which local model for which task?

> Pick a local model family by task — coding, chat, reasoning, long-context, vision, tool-calling, and embeddings — before registering a deployment.

Pick by **selection criteria**, not by whichever name is trending this week: native tool-calling support if you're running an agent, context window versus your VRAM budget, license, and how well the model follows instructions. For DeepSeek, the current checkpoint to evaluate is `deepseek-ai/DeepSeek-V4-Flash-0731`; use R1 distills only when your hardware cannot support the V4 weight floor.

> [!WARNING]
>
> Specific model names and version numbers date fast — a model that's state of the art this quarter may be superseded within months. Treat the tables below as a starting point for families and criteria, follow the source links, and re-check the current best release in each family before you commit hardware to it.

## Match the task to a family

| Task | Open families (2025–2026) | Selection criteria | Source |
| - | - | - | - |
| Coding / agents | `DeepSeek-V4-Flash-0731`, `Qwen3` / `Qwen-Coder`, `flint` | Reliable tool calls, agentic benchmarks, context that fits your KV-cache budget | [DeepSeek V4 Flash 0731 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731), [Flint](https://docs.tokios.com/models/flint.md) |
| General chat | `Qwen3` (0.6B–32B dense), `Llama 3.1`/`3.3`, `gpt-oss` | Instruction-following, license, size fit for your hardware | [Qwen3 blog](https://qwenlm.github.io/blog/qwen3/), [gpt-oss model card](https://openai.com/index/gpt-oss-model-card/) |
| Reasoning | `DeepSeek-V4-Flash-0731` (`low` / `high` / `max` effort), `Qwen3`, `gpt-oss-120b`/`20b` | Reasoning controls, benchmark setup, and output-token budget | [DeepSeek V4 Flash 0731 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731), [gpt-oss model card](https://openai.com/index/gpt-oss-model-card/) |
| Long-context | `DeepSeek-V4-Flash-0731` (1M), `Qwen3` (256K), `Llama 3.1` (128K) | Native context length and measured KV-cache capacity — verify recall at the length you need | [DeepSeek V4 Flash 0731 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731), [Meta Llama 3.1](https://ai.meta.com/blog/meta-llama-3-1/) |
| Vision / multimodal | `Qwen3-VL`, `Qwen3.8-27B`, `Gemma`, `Llama 4` (multimodal) | OCR, document parsing, visual localization quality; whether your runtime loads the vision projector | [Koyeb multimodal roundup](https://www.koyeb.com/blog) |
| Tool-calling / agentic | `DeepSeek-V4-Flash-0731`, `Qwen3` (with `Qwen-Agent`/MCP), `Llama 3.1` | Native function calling, structured-output reliability, and exact harness settings | [DeepSeek V4 Flash 0731 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731), [Qwen3 blog](https://qwenlm.github.io/blog/qwen3/) |
| Embeddings | `Qwen3-Embedding` (8B, Apache-2.0), `BGE-M3`, `Nomic` | MTEB score, embedding dimension and latency, license | [Baseten](https://www.baseten.co/), [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard) |

> [!NOTE]
>
> Tokios doesn't serve embeddings on the gateway today — the API surface covers chat and messages endpoints only. The embeddings row above is for picking a model to run locally with your backend directly, not through a Tokios deployment.

> [!TIP]
>
> For the `DeepSeek-V4-Flash-0731` checkpoint, start with the [local hardware guide](https://docs.tokios.com/blog/deepseek-v4-flash-local-hardware.md). Its 1M-token specification does not mean every local configuration has enough KV-cache capacity for that context length.

## Choosing a model for agents specifically

Coding agents and other tool-using clients are pickier than a chat UI. Use three criteria:

1. **Native tool/function calling.** Agent loops depend on the model returning structured tool calls the client can parse. `DeepSeek-V4-Flash-0731` is the current DeepSeek option for this workload. Validate it with the exact serving runtime, reasoning effort, and chat encoding you will use.
2. **Context window of at least 32–64K, ideally 128K or more.** Agents accumulate file contents, diffs, and tool outputs turn over turn, so a short context fills up fast. Budget VRAM for the KV cache at your target context length on top of the model weights — see [Fit a model to your GPU or unified memory](https://docs.tokios.com/guides/model-fit.md).
3. **Instruction-following, structured output, and a permissive license.** Apache-2.0 families like `Qwen3` and `gpt-oss` are easier to adopt without licensing friction.

> [!WARNING]
>
> Clients like [Roo Code](https://docs.tokios.com/guides/roo-code.md) hard-require native tool calling. If the underlying model doesn't support it, the client can't parse its responses into actions and will retry the same step instead of making progress — this is a model limitation, not a Tokios or connector issue.

## Fit it to your hardware

Picking the right family is only half the decision — the model also has to fit in the VRAM or unified memory you actually have, at the context length you need. See [Fit a model to your GPU or unified memory](https://docs.tokios.com/guides/model-fit.md) for sizing estimates by parameter count and quantization.

For the workflow, backend, and naming side of the decision — which backend to run, and what to call the deployment once you've registered it — see [Choose a local model and backend](https://docs.tokios.com/guides/choose-a-local-model.md).

## Once you've picked one

Tokios doesn't choose the model for you; it gives whatever you're running a stable deployment endpoint you can call from anywhere.

> [card] [Register your deployment](https://docs.tokios.com/quickstart.md)
>
> Turn your chosen model into a Tokios deployment with a public name and API key.

> [card] [Compare candidates](https://docs.tokios.com/guides/benchmark-your-deployment.md)
>
> Test two or more deployments side by side before you settle on one.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.