> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Local LLM models: pick one, self-host it, reach it anywhere

> Choose a local LLM to self-host — compare the open-weight families Tokios works with (Qwen, DeepSeek, Llama, Gemma, and more) and match one to your hardware.

A **local LLM** (also called a **self-hosted LLM**) is an open-weight model you run on your own hardware — a laptop, a workstation GPU, a Mac, or a homelab server — instead of calling a hosted API you don't control. This section covers the open-weight families that run well locally and how to pick between them.

Tokios doesn't pick, host, or fine-tune the model. You choose and run it locally; Tokios turns whatever you're running into a stable, authenticated endpoint at `https://api.tokios.com` that any OpenAI- or Anthropic-compatible client can reach — no inbound ports, no VPN.

## How to choose in three steps

**1. Match the task**

Coding agent, general chat, reasoning, vision, or edge/CPU work? The task narrows the family fast. See [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md) for a task-by-task breakdown with agent-specific criteria.

**2. Fit it to your hardware**

A model only helps if it fits in your VRAM or unified memory at the context length you need. See [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math, and [Quantization](https://docs.tokios.com/concepts/quantization.md) for the size/quality dial.

**3. Check the license**

Licenses vary widely — Apache-2.0 and MIT are permissive; others (Llama Community, Gemma, NVIDIA Open Model License) carry conditions. See [How to read a model card](https://docs.tokios.com/concepts/read-a-model-card.md) for what to look for before you build on a model.

## The families, and what's current

Every family below has a dedicated page with tags, sizes, hardware-fit tables, and setup. Model versions move fast — this table is a starting point; follow each page for the current release.

> [!NOTE]
>
> Last verified: October 2026.

| Family (vendor) | Best known for | Current open-weight release | License | Guide |
| - | - | - | - | - |
| **Qwen** (Alibaba) | All-round local models; also coding and vision variants | Qwen3.8-27B (Aug 2026) | Apache-2.0 | [Qwen3.8-27B](https://docs.tokios.com/models/qwen3.8.md) |
| **DeepSeek** | Coding agents, tool use, and long-context reasoning | DeepSeek-V4.1-Flash (Sep 2026) | MIT | [DeepSeek V4.1 Flash](https://docs.tokios.com/models/deepseek.md) |
| **Gemma** (Google) | Efficient multimodal, edge → desktop | Gemma 4 (Apr 2026) | Apache-2.0 (Gemma 4) | [Gemma](https://docs.tokios.com/models/gemma.md) |
| **Muse** (Meta) | Agentic tasks and tool use — consumer GPU (Glimmer) to frontier workstation (Spark Open) | Muse Spark Open 816B/42B (Oct 2026); Muse Glimmer 30B (Aug 2026) | Muse Model License (Spark Open); Apache-2.0 (Glimmer) | [Muse Spark Open](https://docs.tokios.com/models/muse-spark-open.md) · [Muse Glimmer](https://docs.tokios.com/models/muse-glimmer.md) |
| **Llama** (Meta) | General chat; huge-context MoE | Llama 4; Llama 3.3 for a single GPU | Llama Community License | [Llama 4](https://docs.tokios.com/models/llama-4.md) |
| **gpt-oss** (OpenAI) | Open reasoning MoE | gpt-oss-20b / 120b | Apache-2.0 | [gpt-oss](https://docs.tokios.com/models/gpt-oss) |
| **Mistral** | Task-specific tiers (chat, coding, reasoning, edge) | Mistral Small 4, Devstral, Magistral, Ministral; Large 4 announced (weights pending) | Apache-2.0 | [Mistral](https://docs.tokios.com/models/mistral) |
| **GLM** (Z.ai) | Coding and agentic workflows | GLM-5.3 (flagship), GLM-5.3-Flash; GLM-4, GLM-4.5-Air, GLM-4.6 | Custom (GLM-5.3); MIT (GLM-5.3-Flash) | [GLM](https://docs.tokios.com/models/glm.md) |
| **Kimi** (Moonshot AI) | Frontier multimodal agentic & coding (server-class and up) | K2.6 / K2.7-Code; [K3](https://docs.tokios.com/models/kimi-k3.md) (Jul 2026) | Modified MIT (K2.x); Kimi K3 License | [Kimi](https://docs.tokios.com/models/kimi.md) |
| **MiMo** (Xiaomi) | Multimodal agents (text/image/video/audio), 1M context, MIT | MiMo V2.6 Pro / Flash / Distill (Sep 2026) | MIT | [MiMo](https://docs.tokios.com/models/mimo.md) |
| **MiniMax** | Long-context, agentic & coding line (big-memory) | M3 (1M context); M2 | Varies by version | [MiniMax](https://docs.tokios.com/models/minimax.md) |
| **Nemotron** (NVIDIA) | Long-context reasoning, hybrid Mamba-2 | Nemotron 3 (Nano → Ultra) | NVIDIA Open Model License | [Nemotron](https://docs.tokios.com/models/nemotron) |
| **Phi** (Microsoft) | Small models, strong reasoning per parameter | Phi-4 | MIT | [Phi](https://docs.tokios.com/models/phi) |
| **Granite** (IBM) | Enterprise RAG and tool-calling, hybrid Mamba-2 | Granite 4 | Apache-2.0 | [Granite](https://docs.tokios.com/models/granite) |

> [!NOTE]
>
> "Current" means the newest open-weight release you can download and run yourself. Some current releases still require multi-GPU hardware, including `DeepSeek-V4.1-Flash`, GLM-5.3, and the frontier-scale Kimi and MiniMax MoEs. Each family page calls out what is downloadable now, what needs a server, and what is still pending.

> [!TIP]
>
> Need a current frontier-scale model you can still host yourself? [DeepSeek-V4.1-Flash local hardware](https://docs.tokios.com/blog/deepseek-v4-flash-local-hardware.md) compares the 2× RTX PRO 6000, 2× DGX Spark, and 2× Strix Halo paths, including their support maturity and memory limits.

## Pick by what you're doing

> [card] [General chat & agents](https://docs.tokios.com/models/qwen3.6.md)
>
> Qwen3.6, Llama, Gemma, and gpt-oss are strong all-rounders. [Qwen3.8-27B](https://docs.tokios.com/models/qwen3.8.md) is the newest Qwen release (vLLM/SGLang only for now); Qwen3.6 is the easiest verified path, and [Muse Glimmer](https://docs.tokios.com/models/muse-glimmer.md) is the agent-focused pick that stays resident on a 24 GB card.

> [card] [Coding](https://docs.tokios.com/models/qwen3-coder)
>
> Qwen3-Coder, Mistral's Devstral, and GLM target agentic, repository-scale coding.

> [card] [Reasoning and agents](https://docs.tokios.com/models/deepseek.md)
>
> Start with DeepSeek-V4.1-Flash for the current DeepSeek reasoning, coding-agent, and tool-calling release. Use R1 distills for smaller hardware. For frontier-scale agentic coding on workstation hardware, [Muse Spark Open](https://docs.tokios.com/models/muse-spark-open.md) leads the open-weight field.

> [card] [Vision & multimodal](https://docs.tokios.com/models/qwen3-vl.md)
>
> Qwen3-VL and Gemma 4 read text, images, and video — start with Qwen3-VL. [MiMo V2.6](https://docs.tokios.com/models/mimo.md) adds native audio input across all 3 variants.

> [card] [Small, edge & CPU](https://docs.tokios.com/models/phi)
>
> Gemma's E2B, Phi-4-mini, Granite, and Mistral's Ministral run on laptops and edge boxes.

> [card] [Formats & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Which weight format — GGUF, MLX, NVFP4, AWQ — runs on your GPU or Mac.

## Once you've picked one

Tokios doesn't choose the model for you; it gives whatever you're running a stable, authenticated endpoint you can call from anywhere.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector, register a deployment, and create your first key end to end.

> [card] [Choose a model and backend](https://docs.tokios.com/guides/choose-a-local-model.md)
>
> Match a model to your workflow, backend, and deployment naming.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.