> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Kimi K3 won't fit — what to run instead, by hardware

> Kimi K3 needs ~1.4 TB and no consumer or prosumer rig holds it. Here's the closest model to run instead on Strix Halo, 1–4 DGX Sparks, or a single GPU.

> [!NOTE]
>
> Last verified: 2026-10-14

Kimi K3 is the model you came for, but it does not fit on hardware you own. Its 2.8T parameters need about **1.4 TB** at the native 4-bit floor — even 8 DGX Sparks (1 TB pooled) fall 376 GB short, and Moonshot targets K3 at a supernode with 64 or more accelerators. See [Run Kimi K3 locally](https://docs.tokios.com/models/kimi-k3.md) for the availability picture and the [DGX Spark memory math](https://docs.tokios.com/blog/kimi-k3-hardware-deep-dive.md) for the full calculation.

So the useful question is not "how do I run K3" — it's "what is the closest model that actually runs on *my* machine, today?" This page answers that by hardware tier.

> [!TIP]
>
> Already running one of the models below? [Register it with Tokios](https://docs.tokios.com/setup/register-model.md) to reach it as `https://api.tokios.com` from any OpenAI- or Anthropic-compatible client — see [Connect your pick with Tokios](#connect-your-pick-with-tokios) at the bottom.

## The short answer by hardware

| Your hardware | Best coding / agent model | Best general / reasoning model | Deploy guide |
| - | - | - | - |
| Strix Halo (128 GB) | Qwen3-Coder-30B (Q4\_K\_M) | GLM-4.5-Air, Qwen3.5-122B-A10B, or [Qwen3.8-27B](https://docs.tokios.com/models/qwen3.8.md) for vision | [Strix Halo compatibility](https://docs.tokios.com/guides/strix-halo-model-compatibility.md) |
| 1 DGX Spark (128 GB) | Qwen3.5-35B-A3B (NVFP4) | GLM-4.5-Air (106B MoE), or [Qwen3.8-27B](https://docs.tokios.com/models/qwen3.8.md) at FP8 | [DGX Spark compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) |
| 2 DGX Sparks (256 GB) | `DeepSeek-V4-Flash-0731` or Qwen3.5-35B-A3B | `DeepSeek-V4-Flash-0731` | [DeepSeek V4 Flash hardware](https://docs.tokios.com/blog/deepseek-v4-flash-local-hardware.md) |
| 4 DGX Sparks (512 GB) | Qwen3-Coder-480B | `DeepSeek-V4-Flash-0731`; DeepSeek-R1 671B (Q4) for legacy R1 workloads | [DGX Spark compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) |
| Big-RAM / multi-GPU server (256 GB+) | **Kimi K2.7-Code** | **Kimi K2.6** | [Kimi models](https://docs.tokios.com/models/kimi.md) |
| Single consumer GPU (8–24 GB) | Qwen3-Coder-30B-A3B (Q4\_K\_M) | gpt-oss-20b (MXFP4), or [Qwen3.8-27B](https://docs.tokios.com/models/qwen3.8.md) at `Q4_K_M` on 24 GB | [Model fit](https://docs.tokios.com/guides/model-fit.md) |

Every model above is one you can download and serve today. The rows are ordered by memory, not by capability — pick your hardware, then read that tier below. One caveat on the big-RAM row: Kimi K2.6 and K2.7-Code hit interactive speed on a multi-GPU server, but on a RAM-offload-only workstation they run at single-digit tokens per second — fine for batch work, slow for coding agents.

> [card] [Found your tier? Connect it next](https://docs.tokios.com/quickstart.md)
>
> Pair the Tokios connector and register a deployment name in about 5 minutes, then call your model from anywhere.

## The closest thing to K3 itself: Kimi K2.6 and K2.7-Code

If you want K3 specifically because it is *Kimi*, the honest substitute is the rest of the Kimi family. **Kimi K2.6** (general flagship) and **Kimi K2.7-Code** (coding-specialized, the default in Moonshot's Kimi Code CLI) are 1T-parameter open-weight MoEs — one full hardware tier below K3. They ship at native INT4 (\~595 GB on disk), so they need a multi-GPU server or a big-RAM workstation, not a desktop, but they *are* downloadable now.

This is the strongest "run instead" answer on the page because it is not a substitute at all — it is the same family, with the same agentic tool-calling and multimodal input, at a size that real hardware can hold. See [Kimi models](https://docs.tokios.com/models/kimi.md) for the vLLM/SGLang serving path and quant options.

> [!NOTE]
>
> Kimi's `kimi-*` tags on Ollama are **`:cloud`-only** — they run on Ollama's infrastructure, not yours, so there is nothing local for a connector to reach. Self-host the weights with vLLM or SGLang if you want Kimi on hardware you control.

## Strix Halo (128 GB unified)

An AMD Ryzen AI Max+ 395 exposes roughly 96 GB to the GPU after BIOS and driver allocation. MoE models in the 35B–122B range are the sweet spot — they run 3–14× faster than a dense model of the same parameter count.

* **Coding / agents:** Qwen3-Coder-30B at Q4\_K\_M (\~19 GB) is the community's go-to for Cline, Aider, and opencode, at roughly 10–15 tok/s.
* **General / reasoning:** GLM-4.5-Air (106B MoE, \~60 GB) is the quality pick; Qwen3.5-122B-A10B (\~70 GB) is the maximum-capability option. Both land around 10–15 tok/s at Q4\_K\_M.

Full fit tables, quant formats, and the Vulkan-vs-ROCm story are in [Strix Halo model compatibility](https://docs.tokios.com/guides/strix-halo-model-compatibility.md) and [Serve a model on Strix Halo](https://docs.tokios.com/use-cases/strix-halo.md).

## 1 DGX Spark (128 GB unified)

A single Spark has \~115–120 GB usable after system overhead. It runs dense models up to \~70B at Q4 and MoE models up to \~120B total at NVFP4 — [NVFP4 is the recommended default](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) on the SM121 GPU.

* **Coding / agents:** Qwen3.5-35B-A3B at NVFP4 (\~18 GB) sustains 47–50 tok/s; the newer Qwen3.6-35B-A3B reaches up to \~120 tok/s with the Atlas engine. Qwen3-Coder-30B is the coder-tuned alternative.
* **General / reasoning:** GLM-4.5-Air (106B MoE, \~53 GB) is the widely reported best-quality pick; Qwen3.5-122B-A10B (\~61 GB) is the largest MoE that fits, at \~30 tok/s.

See [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) for the full model list and [vLLM on DGX Spark](https://docs.tokios.com/guides/vllm-dgx-spark.md) for the launch commands.

## 2 DGX Sparks (256 GB combined)

A second Spark does not unlock a "bigger Kimi." It does unlock `DeepSeek-V4-Flash-0731`: 2 nodes provide 256 GB of aggregate unified memory for its roughly 162–167 GB weight floor. The route is community-tested rather than vendor-verified for 0731, so start at 32K context and revalidate model load, tool calls, and sustained generation. A 405B dense model at FP8 needs roughly 405 GB for weights alone and does **not** fit in 2 Sparks.

If your goal is a large *coding* model rather than more concurrency, skip to 4 Sparks — [Qwen3-Coder-480B](https://docs.tokios.com/models/qwen3-coder) needs a \~250 GB+ memory floor and only gets real KV-cache headroom at 512 GB. See [One vs two DGX Sparks](https://docs.tokios.com/guides/one-vs-two-dgx-sparks.md) for the decision framework and worked throughput numbers.

## 4 DGX Sparks (512 GB combined)

4 nodes is NVIDIA's documented maximum Spark cluster, positioned for models up to about 700B parameters. This is the tier that makes the K3 verdict concrete: even doubling past 4 Sparks to 8 still falls 376 GB short of K3 — so 4 Sparks is where you run the largest models that *do* fit.

* **Coding / agents:** [Qwen3-Coder-480B](https://docs.tokios.com/models/qwen3-coder) (large MoE coder, \~250 GB+ memory floor) runs here with the KV-cache headroom it lacked at 256 GB.
* **General / reasoning:** Start with `DeepSeek-V4-Flash-0731`, the current official release with tool calling and 1M context. DeepSeek-R1 671B at `Q4_K_M` (\~404 GB) also fits when you specifically need the older R1 family.

The repo does not publish measured tok/s at this cluster size, so treat any throughput figure here as an estimate until you benchmark your own nodes. See [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) and the [Kimi K3 hardware deep dive](https://docs.tokios.com/blog/kimi-k3-hardware-deep-dive.md) for the cluster math.

## Big-RAM or multi-GPU server (256 GB+)

This is the tier where the truest K3 alternatives live, because it is where the rest of the Kimi family runs. On an 8×80 GB datacenter box, serve Kimi K2.6 or K2.7-Code with vLLM at `--tensor-parallel-size 8` for interactive speed. On a 256 GB+ RAM workstation with GPU offload, a community GGUF runs a low-bit quant slowly — single-digit tokens per second, not interactive.

* **Coding / agents:** Kimi K2.7-Code — Moonshot's coding-specialized 1T MoE.
* **General / reasoning:** Kimi K2.6 for mixed agentic work; MiniMax M3 (428B / 23B active) or GLM-5.2 are strong non-Kimi options at this tier.

Deployment commands and quant options are on [Kimi models](https://docs.tokios.com/models/kimi.md); [Homelab GPU server](https://docs.tokios.com/use-cases/homelab-gpu-server.md) covers building the rig, and [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md) helps you choose.

## Single consumer GPU (8–24 GB)

A single 24 GB card is a different class of tool from K3 — set expectations accordingly. It is excellent for fast, local coding help and general chat, not for frontier reasoning.

* **Coding / agents:** Qwen3-Coder-30B-A3B at Q4\_K\_M (19 GB) is a comfortable fit on a 24 GB card.
* **General / reasoning:** gpt-oss-20b at native MXFP4 (14 GB) fits comfortably and exposes low/medium/high reasoning effort.

Check any other model against your exact card with the [model-fit calculator](https://docs.tokios.com/guides/model-fit.md), and see [RTX workstation](https://docs.tokios.com/use-cases/rtx-workstation.md) for the full setup.

## Connect your pick with Tokios

Once your chosen model is running on a local runtime — Ollama, llama.cpp, vLLM, or SGLang exposing an OpenAI-compatible `/v1` — the Tokios connector dials out and gives you one authenticated endpoint at `https://api.tokios.com`, with no inbound ports and no VPN. Each model page above carries the full connect walkthrough; the short version:

**1. Pair the connector**

Sign in at [tokios.com/console](https://tokios.com/console), open **Setup**, click **Start pairing**, and run the installer on the machine serving your model with the one-time `TKS-XXXX-XXXX` claim code.

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Register a deployment**

Point the connector at your runtime's `/v1` base URL and register the upstream model id under a public deployment name — see [register a model](https://docs.tokios.com/setup/register-model.md).

**3. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key** and copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "YOUR_DEPLOYMENT_NAME",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Explain async/await in Python."}
    ]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model YOUR_DEPLOYMENT_NAME
```

## FAQ

> [accordion] What is the best local alternative to Kimi K3?
>
> It depends on your hardware. If you want Kimi specifically, run **Kimi K2.6** (general) or **Kimi K2.7-Code** (coding) on a multi-GPU server — same family, one tier down. On a single DGX Spark, the best all-round pick is **GLM-4.5-Air** for quality or **Qwen3.5-35B-A3B** for speed. On Strix Halo, **Qwen3-Coder-30B** for coding and **GLM-4.5-Air** for general use.

> [accordion] What is the best coding model for a DGX Spark?
>
> **Qwen3.5-35B-A3B** at NVFP4 — 47–50 tok/s on 1 Spark (the newer **Qwen3.6-35B-A3B** reaches up to \~120 tok/s with the Atlas engine) — or the coder-tuned **Qwen3-Coder-30B**. For a large coding MoE you need a 4-Spark cluster (Qwen3-Coder-480B). See [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md).

> [accordion] Is there a model like Kimi K3 that runs on 128 GB?
>
> Not K3 itself — its floor is \~1.4 TB. But 128 GB (1 DGX Spark or a Strix Halo box) comfortably runs 100B-class MoE models like GLM-4.5-Air that deliver strong agentic and reasoning quality. That is the realistic frontier-adjacent tier for a single 128 GB machine.

> [accordion] Can I run any Kimi model locally?
>
> Yes — **Kimi K2.6** and **Kimi K2.7-Code** are open-weight 1T MoEs at native INT4 (\~595 GB), so they run on a multi-GPU server (interactive) or a big-RAM workstation with offload (slowly — single-digit tokens per second). Only **K3** is out of reach on self-owned hardware. See [Kimi models](https://docs.tokios.com/models/kimi.md).

> [accordion] Why not just run K3?
>
> The weights are out (released July 27, 2026 — see [Run Kimi K3 locally](https://docs.tokios.com/models/kimi-k3.md)), but K3's \~1.4 TB floor rules out every consumer and prosumer setup, so the models on this page remain your realistic local options.

> [card] [Run Kimi K3 locally](https://docs.tokios.com/models/kimi-k3.md)
>
> The availability picture and why no rig you own holds K3 today.

> [card] [Kimi models](https://docs.tokios.com/models/kimi.md)
>
> Self-host K2.6 and K2.7-Code — the closest downloadable Kimi models.

> [card] [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md)
>
> The full fit tables, quant formats, and recommended models per use case.

> [card] [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> Check any model against your exact hardware before you commit.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.