> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Kimi models: which one can you run locally?

> Moonshot AI's Kimi family compared — K3 (weights released July 27) versus downloadable K2.6 and K2.7-Code — with honest hardware requirements for self-hosting each.

> [!NOTE]
>
> Last verified: 2026-10-14

Kimi is Moonshot AI's frontier-scale model family. The current lineup spans three models: **Kimi K3**, the new 2.8-trillion-parameter flagship (weights released July 27, 2026), and the downloadable **Kimi K2.6** and **Kimi K2.7-Code** — both 1-trillion-parameter open-weight MoEs you can self-host today on serious hardware. This page is the honest guide to which Kimi you can actually run yourself, and how to reach it from anywhere through Tokios.

## The Kimi lineup at a glance

| Property | Kimi K3 | Kimi K2.6 | Kimi K2.7-Code |
| - | - | - | - |
| Status | Open weights available (released July 27, 2026) | Open weights available | Open weights available |
| Focus | Long-horizon coding, knowledge work, reasoning | General flagship: writing, analysis, agentic work | Agentic coding |
| Parameters | 2.8T total | 1T total / \~32B active | 1T total / \~32B active |
| Context window | 1M | 256K | 256K |
| Modalities | Text, image, video | Text, image, video | Text, image, video |
| Local hardware | Supernode with 64+ accelerators | Multi-GPU server or big-RAM offload | Multi-GPU server or big-RAM offload |
| License | Kimi K3 License | Modified MIT | Modified MIT |
| Guide | [Kimi K3 page](https://docs.tokios.com/models/kimi-k3.md) | This page, below | [K2.7-Code section](#kimi-k2-7-code-for-coding-agents) |

Wondering whether you can [run Kimi K3 locally](https://docs.tokios.com/models/kimi-k3.md)? The weights are out (released July 27, 2026), but the floor is roughly **1.4 TB** of memory. The rest of this page covers the Kimi models you *can* download today.

> [!NOTE]
>
> If you just want Kimi's answers and don't care where it runs, Moonshot's hosted API ([platform.kimi.com](https://platform.kimi.com)) is the easy path. Tokios is for the case where you want the weights on **your** hardware — a serious multi-GPU or high-RAM rig — and a private, authenticated endpoint in front of it.

## Can you run Kimi locally?

Honestly: not on a normal desktop. Kimi K2.6 ships already at **native INT4** — that's the *small* format — and it's still about **595 GB** across 64 checkpoint shards. There is no consumer GPU, and no single datacenter GPU, that holds it. The realistic options:

| Setup | Can it run Kimi K2.6? |
| - | - |
| Single consumer GPU (8–24 GB) | No — off by 20–70× |
| One 80 GB datacenter GPU (H100/H200) | No — needs several |
| Multi-GPU server (e.g. 8× 80 GB) | Yes, via vLLM / SGLang |
| CPU + GPU offload, 256–512 GB system RAM | Yes, slowly, via community GGUF |
| Apple Silicon / Strix Halo (≤128 GB unified) | No — not enough memory for a usable quant |

Community [re-quantizations](https://docs.tokios.com/concepts/quantization.md) trade quality for size but don't change the conclusion: [`bartowski/moonshotai_Kimi-K2.6-GGUF`](https://huggingface.co/bartowski/moonshotai_Kimi-K2.6-GGUF) ranges from an extreme `IQ1_S` at \~207 GB up to `Q4_0` at \~584 GB, and Unsloth's dynamic 2-bit build (\~340–350 GB) still wants a **350 GB+** combined VRAM/RAM budget. As a rule of thumb, plan for RAM + VRAM roughly equal to the quant size. For most people, Kimi is a **cloud/API model** — and that's a fine answer. Run it locally only if you have the rig for it.

> [!TIP]
>
> Already serving Kimi on your own hardware? [Register it with Tokios](https://docs.tokios.com/setup/register-model.md) to reach it from any client. Not sure your rig can hold it? See [what to run instead of Kimi K3](https://docs.tokios.com/models/kimi-k3-alternatives.md).

## "Kimi on Ollama" is cloud, not local

Kimi K2.x models *are* in the Ollama library — `kimi-k2.6`, `kimi-k2.7-code`, `kimi-k2.5` — but every one of them exposes only a **`:cloud`** tag. Ollama's cloud models don't download; requests are proxied to Ollama's hosted infrastructure while you stay signed in. So `ollama run kimi-k2.6:cloud` is convenient, but it is **not** self-hosting — the weights never touch your machine, and there's nothing local for a connector to point at.

If your goal is Kimi on hardware you control, skip the `:cloud` tag and self-host the weights with a server runtime, below. Tokios then fronts *your* deployment, not someone else's cloud.

## Run it on your own hardware

Moonshot's deployment docs target production-grade runtimes — **vLLM**, **SGLang**, and **KTransformers** — not single-GPU stacks. On a multi-GPU server, serve the native INT4 weights and expose an OpenAI-compatible endpoint:

> [tab] vLLM (official path, multi-GPU server)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve moonshotai/Kimi-K2.6 \
>   --tensor-parallel-size 8 \
>   --trust-remote-code
> ```
>
> Tensor-parallel across every GPU in the box. vLLM serves an OpenAI-compatible API on `http://127.0.0.1:8000/v1` — that's what the Tokios connector points at. Check the model card for the current recommended flags and any tool-call parser before serving.

> [tab] Community GGUF (big-RAM rig)
>
> On a workstation with 256 GB+ of system RAM (plus a GPU for offload), a community GGUF via `llama.cpp` or LM Studio can run a low-bit Kimi quant on CPU + GPU offload — slowly. Start from [`bartowski/moonshotai_Kimi-K2.6-GGUF`](https://huggingface.co/bartowski/moonshotai_Kimi-K2.6-GGUF) or Unsloth's dynamic quants and pick the largest quant your RAM + VRAM can hold. Expect single-digit tokens/sec, not interactive speed.

> [!WARNING]
>
> Kimi K2.6 is distributed at native INT4 already. Don't try to "upgrade" it to BF16 to run it — a BF16 upcast is roughly 2.1 TB and buys you nothing. The INT4 weights are the intended format.

## Variants & specs

Every value below is from the Moonshot model cards on Hugging Face and Moonshot's platform docs. Verify against the model card before you standardize on one — Moonshot ships new Kimi releases roughly monthly.

| Property | Kimi K2.6 | Kimi K2.7-Code | Kimi K2.5 |
| - | - | - | - |
| Hugging Face | [`moonshotai/Kimi-K2.6`](https://huggingface.co/moonshotai/Kimi-K2.6) | [`moonshotai/Kimi-K2.7-Code`](https://huggingface.co/moonshotai/Kimi-K2.7-Code) | [`moonshotai/Kimi-K2.5`](https://huggingface.co/moonshotai/Kimi-K2.5) |
| Ollama tag | `kimi-k2.6:cloud` (cloud only) | `kimi-k2.7-code:cloud` (cloud only) | `kimi-k2.5:cloud` (cloud only) |
| Parameters | 1T total / **32B active** (384 experts, 8/token) | 1T total / **32B active** | 1T total / **32B active** |
| Context window | 256K (262,144) | 256K (262,144) | 256K (262,144) |
| Modality | Text + image + video | Text + image + video | Text + image + video |
| Native format | INT4 (\~595 GB on disk) | INT4 (\~595 GB on disk) | INT4 |
| License | Modified MIT | Modified MIT | Modified MIT |
| Best for | General flagship: writing, analysis, agentic work | Agentic coding (default in Moonshot's Kimi Code CLI) | Prior flagship |

Sources: [`moonshotai/Kimi-K2.6`](https://huggingface.co/moonshotai/Kimi-K2.6), [`moonshotai/Kimi-K2.7-Code`](https://huggingface.co/moonshotai/Kimi-K2.7-Code), [platform.kimi.com/docs/models](https://platform.kimi.com/docs/models), [ollama.com/library/kimi-k2.6](https://ollama.com/library/kimi-k2.6).

> [!NOTE]
>
> Kimi's **Modified MIT** license is permissive for most uses, but adds an attribution clause for very large commercial deployments (displaying "Kimi K2.6" in your product's UI above a high MAU/revenue threshold). Read the `LICENSE` file on the model card before you build a commercial product on it.

## Capabilities

* **Agentic tool calling** — K2.x is trained for tool use and multi-step agent loops. When you serve it with vLLM or SGLang, enable the tool-call parser the model card recommends so `tools` requests work through your endpoint.
* **Native multimodal input** — text, image, and video understanding across the K2.x line.
* **Reasoning** — K2.x exposes thinking through Moonshot's `thinking` API parameter. Kimi K3 replaces it with a top-level `reasoning_effort` — don't mix the two conventions; see the [Kimi K3 page](https://docs.tokios.com/models/kimi-k3.md) for K3's protocol details.
* **256K context** — long enough for repository-scale coding sessions, but budget [KV-cache memory](https://docs.tokios.com/concepts/kv-cache-and-context.md) on top of the weights when you size a long-context deployment.

## Which Kimi should you run?

* **K2.6** is Moonshot's general-purpose downloadable flagship — reach for it for writing, analysis, conversation, and mixed agentic work.
* **K2.7-Code** is scoped to coding and agentic tasks — see the [dedicated section below](#kimi-k2-7-code-for-coding-agents).
* **K2.5** is the prior flagship, still available if you've already standardized on it.
* **K3** is the frontier flagship (weights released July 27, 2026), but it moves a full hardware tier upward — see [Kimi K3](https://docs.tokios.com/models/kimi-k3.md) for hardware requirements, or [what to run instead](https://docs.tokios.com/models/kimi-k3-alternatives.md) if it won't fit your hardware.

## Kimi K2.7-Code for coding agents

Kimi K2.7-Code (June 2026) is the coding-specialized sibling of K2.6: same 1T-MoE architecture, size, and hardware requirements, but tuned for agentic engineering. It cuts reasoning-token usage roughly 30% versus K2.6 and is the default model in Moonshot's Kimi Code CLI. If coding is the job, it's the Kimi to serve.

It shares K2.6's deployment story exactly — the same INT4 weights size, the same vLLM/SGLang serving path (swap the model id for `moonshotai/Kimi-K2.7-Code`), and the same `:cloud`-only situation on Ollama. Once it's behind your Tokios endpoint, it pairs naturally with coding agents: see [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) and [Use OpenAI Codex](https://docs.tokios.com/guides/openai-codex.md) for client setup.

## Connect it with Tokios

Once Kimi is running on your own server — vLLM or SGLang exposing an OpenAI-compatible `/v1` — pair the Tokios connector so you can reach it as `https://api.tokios.com` from Claude Code, Cline, or any other compatible client, instead of only from inside your LAN.

**1. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Install and run the connector on the server hosting Kimi:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Point the connector at your runtime's /v1 API**

vLLM and SGLang expose an OpenAI-compatible surface — typically `http://127.0.0.1:8000/v1`. Set that as the upstream in `tokios-connector.json` — see [Connector config](https://docs.tokios.com/reference/connector-config.md).

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the model and choose a public deployment name, for example `kimi-tunnel`. This name — not the upstream model id — is what clients send in the `model` field.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kimi-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Plan and scaffold a small CLI tool, step by step."}]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model kimi-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`kimi-tunnel` above), never the upstream model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) and [Use OpenAI Codex](https://docs.tokios.com/guides/openai-codex.md) for full client setup — Kimi K2.7-Code pairs naturally with a coding agent.

## Caveats

* Kimi K2.x is a **1T-parameter MoE** — all trillion parameters must fit in memory even though [only \~32B are active per token](https://docs.tokios.com/concepts/how-inference-works.md). This is a multi-GPU-server or big-RAM-workstation model, not a single-GPU one.
* **Ollama's `kimi-*` tags are `:cloud`-only** — they run on Ollama's infrastructure, not yours. There's nothing local for a connector to reach; self-host with vLLM/SGLang if you want Kimi on your own hardware.
* Kimi ships at **native INT4**; don't upcast to BF16 to run it.
* **Kimi K3 weights are available** (released July 27, 2026) — the [Kimi K3 page](https://docs.tokios.com/models/kimi-k3.md) covers hardware requirements and what to verify before self-hosting it.
* Moonshot releases Kimi versions frequently and deprecates older previews — confirm the current model id on [platform.kimi.com/docs/models](https://platform.kimi.com/docs/models) before you standardize.

> [card] [Kimi K3](https://docs.tokios.com/models/kimi-k3.md)
>
> The 2.8T frontier flagship: what's available now and the hardware reality.

> [card] [What to run instead of K3](https://docs.tokios.com/models/kimi-k3-alternatives.md)
>
> Can't fit K3? The best model for your exact hardware, with the deploy guide.

> [card] [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> The sizing math behind why a 1T MoE needs a server, not a desktop.

> [card] [Homelab GPU server](https://docs.tokios.com/use-cases/homelab-gpu-server.md)
>
> Building the multi-GPU rig a model like Kimi actually needs.

> [card] [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md)
>
> Lighter models that run on hardware you already own.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.