> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run Qwen3.6 locally and use it from anywhere

> Serve Qwen3.6 — Qwen's most widely supported multimodal family — with Ollama, LM Studio, vLLM, or SGLang, and reach it through a Tokios endpoint.

> [!NOTE]
>
> Last verified: 2026-07-21

Qwen3.6 is Qwen's most widely supported open-weight family: a multimodal mixture-of-experts model that reasons by default, handles up to a 1M-token context, and ships with native tool-calling for agentic coding. Its successor, [Qwen3.8-27B](https://docs.tokios.com/models/qwen3.8.md), shipped in August 2026 under the same Apache-2.0 license and already has community GGUF builds, but no Ollama tag — so Qwen3.6 remains the easiest verified local setup. It comes in two sizes — a `35B-A3B` MoE (35B total, 3B active) and a `27B` dense model. Run whichever fits your hardware locally, then pair the Tokios connector to call it from any machine, teammate, or hosted agent through `https://api.tokios.com` — no inbound ports, no VPN.

> [!TIP]
>
> Already running Qwen3.6 locally? [Register it with Tokios](https://docs.tokios.com/setup/register-model.md) to reach it as `https://api.tokios.com` from any client.

> [!NOTE]
>
> Looking for **Qwen 3.5**? Qwen3.6 is the generation that replaced it. Alibaba's dense Qwen3.6-27B outperforms the previous-generation Qwen3.5-397B-A17B MoE on several agentic coding tasks despite being far smaller — so if you were about to pull Qwen3.5, this is the model to run instead.

## Run it locally with Ollama

Qwen3.6 is in the Ollama library. Pull and run the default tag (the `35B-A3B` MoE):

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run qwen3.6
```

For the smaller dense model, run `ollama run qwen3.6:27b`. See the [Ollama library page](https://ollama.com/library/qwen3.6) for the full tag list — this page covers connecting it, not every runtime detail.

> [!NOTE]
>
> Ollama's default context window is small (2048 tokens) — far below Qwen3.6's native 256K. Raise `num_ctx` in your Modelfile or client request to use more of its context. Qwen recommends keeping at least 128K to preserve reasoning quality on complex tasks.

## Variants & tags

Every value below is from the model card and the Ollama library. Tags and download sizes change as new quantizations ship — confirm against [ollama.com/library/qwen3.6](https://ollama.com/library/qwen3.6) before you standardize on one.

| Property | `35B-A3B` (MoE) | `27B` (dense) |
| - | - | - |
| Ollama tag | `qwen3.6:35b` (alias `:latest`) | `qwen3.6:27b` |
| Download size (`Q4_K_M`) | \~24 GB | \~17 GB |
| Parameters | 35B total / **3B active** (256 experts, 8 routed + 1 shared) | 27B dense |
| Context window | 262,144 native, extensible to \~1M | 262,144 native |
| Modality | Text + image + video | Text + image + video |
| Reasoning | Thinking mode **on by default** | Thinking mode **on by default** |
| License | Apache-2.0 | Apache-2.0 |
| Hugging Face | [`Qwen/Qwen3.6-35B-A3B`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) | [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B) |

Sources: [ollama.com/library/qwen3.6](https://ollama.com/library/qwen3.6), [Qwen/Qwen3.6-35B-A3B model card](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B).

> [!TIP]
>
> The `35B-A3B` is a mixture-of-experts model with only **3B active parameters** per token, so it decodes about as fast as a 3B model while reasoning like a much larger one — but all 35B parameters still have to fit in memory. Don't size your GPU off the active-parameter count alone. See [how inference works](https://docs.tokios.com/concepts/how-inference-works.md) for why.

## Quantization & formats

Ollama's default tags are `GGUF` `Q4_K_M`. The library also publishes `-q8_0`, `-bf16`, `-nvfp4` (NVIDIA Blackwell), `-mxfp8`, `-mlx` (Apple Silicon), and `-mtp-*` (multi-token-prediction speculative-decoding) variants for both sizes. Which format you can run depends on your hardware — see [Quantization formats and the hardware they need](https://docs.tokios.com/models/quantization-and-hardware.md) for the full matrix, and [what quantization is](https://docs.tokios.com/concepts/quantization.md) for how to read the `Q4_K_M`-style names.

Unlike Gemma, Qwen doesn't publish official **quantization-aware-trained (QAT)** checkpoints — Qwen's tags above are post-training quantizations. If you want QAT-grade quality at 4-bit, fine-tune your own with a tool like Unsloth; see [quantization-aware training](https://docs.tokios.com/concepts/qat.md) for when that's worth it.

## KV-cache quantization

The [KV cache](https://docs.tokios.com/concepts/kv-cache-and-context.md) is the per-token memory a model builds as it reads your prompt and generates a reply — and it grows with every token of context. For a long-context model like Qwen3.6 (262K native, extensible to \~1M), the KV cache can rival or exceed the weight footprint at full context. Quantizing the KV cache is a separate lever from weight quantization that cuts that memory cost roughly in half.

### Enable FP8 KV cache

On **vLLM**, add `--kv-cache-dtype fp8` to your serve command:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
vllm serve Qwen/Qwen3.6-35B-A3B \
  --max-model-len 262144 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder
```

On **SGLang**, the equivalent flag is `--kv-cache-dtype fp8`:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
python -m sglang.launch_server \
  --model-path Qwen/Qwen3.6-35B-A3B \
  --context-length 262144 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder
```

> [!NOTE]
>
> Ollama and LM Studio manage the KV cache internally and do not expose a `--kv-cache-dtype` flag. If you need FP8 KV cache, use vLLM or SGLang as the serving backend.

### What you get

| Metric | FP16 KV cache | FP8 KV cache |
| - | - | - |
| Bytes per cache entry | 2 | 1 |
| Memory reduction | — | \~50% |
| Quality impact | Baseline | \<2% degradation on standard benchmarks |

The practical payoff is **longer context on the same hardware**. A Qwen3.6 deployment that fits 32K tokens with FP16 KV cache can typically serve 64K or more with FP8 KV cache in the same memory budget — because the cache takes half the space per token. On memory-constrained hardware like a single DGX Spark, that difference can be the gap between a usable long-context deployment and an OOM. For hardware-specific tuning, see [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md).

> [!TIP]
>
> FP8 KV cache is a quality-for-memory trade-off, not a free win. On most tasks — chat, summarization, code generation — the \<2% quality loss is negligible. If your workload is sensitive to precision (long-form reasoning, mathematical proofs), benchmark with and without FP8 KV cache on your actual prompts before committing.

## Hardware fit

Comfort at roughly `Q4` for a modest context length. Treat every entry as an estimate — the ground truth is the actual `GGUF` size plus the KV cache for your context. See [what size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math.

| Target | `27B` dense (\~17 GB) | `35B-A3B` MoE (\~24 GB) |
| - | - | - |
| RTX 3090 / 4090 (24 GB) | Comfortable | Tight — reduce context |
| RTX 5090 (32 GB, Blackwell) | Comfortable; `nvfp4` tag available | Comfortable; `nvfp4` tag available |
| RTX 6000 Ada / A6000 (48 GB) | Comfortable, long context | Comfortable, long context |
| RTX 5080 / 4080 (16 GB) | Tight — short context | Won't fit at `Q4` |
| Apple Silicon (36 GB+ unified) | Comfortable; `mlx` tag available | Comfortable; `mlx` tag available |
| AMD Strix Halo (ROCm/Vulkan, ≤96 GB) | Comfortable | Comfortable |
| DGX Spark (128 GB unified, Blackwell) | Comfortable (\~14–15 GB at NVFP4, estimated 40–55 tok/s) | Excellent fit (\~18 GB at NVFP4, estimated 47–50 tok/s; up to 120 tok/s with Atlas) |

### Can a DGX Spark run Qwen3.6?

Yes, and it is one of the best matches for this family. The dense `27B` lands near 14 GB at NVFP4 and the `35B-A3B` MoE near 18 GB, against 128 GB of unified memory. Because the weights claim so little of the pool, the rest is available for KV cache — which makes the 262K native context practical here rather than theoretical.

Prefer the MoE for throughput: only 3B parameters are active per token, so it decodes far faster than its total size suggests. See [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) for tuning and measured behaviour.

### Can Strix Halo run Qwen3.6?

Yes. AMD Strix Halo can allocate up to 96 GB to the GPU, so both variants load with room left for long context.

Capacity is not the limit on this machine — memory bandwidth is. Decode trails a discrete GPU holding the same weights, and prompt processing trails it further, so favour the `35B-A3B` MoE where its small active-parameter count does the most good. [Running vLLM on Strix Halo](https://docs.tokios.com/guides/vllm-strix-halo.md) and [Strix Halo model compatibility](https://docs.tokios.com/guides/strix-halo-model-compatibility.md) cover the ROCm and Vulkan paths.

### Can a 24 GB GPU run Qwen3.6?

Yes for the dense `27B` at `Q4_K_M`, which needs about 17 GB and leaves roughly 7 GB for KV cache — comfortable at everyday context lengths. The `35B-A3B` MoE needs about 24 GB, so on a 24 GB card it is tight: shorten the context or move to a 32 GB card.

On 16 GB, the dense `27B` runs only at short context, and the MoE does not fit at `Q4`. Offloading to system RAM works but costs throughput — see [memory and offloading](https://docs.tokios.com/concepts/memory-and-offloading.md).

## Fastest way to run Qwen3.6 locally

For the most tokens per second on modest hardware, reach for the **`35B-A3B` MoE**: only 3B of its 35B parameters are active per token, so it decodes about as fast as a 3B model while reasoning like a much larger one — provided all 35B fit in memory (\~24 GB at `Q4_K_M`). If you specifically want the **dense `27B`**, pull `Q4_K_M` on Ollama or LM Studio (\~17 GB); on Apple Silicon the `-mlx` tag runs fastest. Raise `num_ctx` only as far as you actually need — a shorter context leaves more memory for weights and speeds up decode.

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
# Fastest general pick: the MoE (only ~3B active per token)
ollama run qwen3.6

# Dense 27B at Q4_K_M
ollama run qwen3.6:27b
```

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run qwen3.6
> ```
>
> Leave Ollama listening on the default `http://127.0.0.1:11434`. Raise `num_ctx` to use more context.

> [tab] LM Studio (GUI)
>
> Search for **Qwen3.6** in LM Studio's model catalog, download a `GGUF` quant that fits your VRAM (start with `Q4_K_M`), then start LM Studio's local server. It exposes an OpenAI-compatible API on `http://127.0.0.1:1234/v1`.

> [tab] vLLM (advanced)
>
> Qwen3.6 needs its reasoning and tool-call parsers set explicitly. For tool use:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve Qwen/Qwen3.6-35B-A3B \
>   --max-model-len 262144 \
>   --reasoning-parser qwen3 \
>   --enable-auto-tool-choice \
>   --tool-call-parser qwen3_coder
> ```
>
> Add `--tensor-parallel-size <N>` to shard across GPUs. Flags are from the model card's vLLM section.

> [tab] SGLang (advanced)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> python -m sglang.launch_server \
>   --model-path Qwen/Qwen3.6-35B-A3B \
>   --context-length 262144 \
>   --reasoning-parser qwen3 \
>   --tool-call-parser qwen3_coder
> ```
>
> Add `--tp-size <N>` for tensor parallelism. Flags are from the model card's SGLang section.

## Capabilities

* **Reasoning:** Qwen3.6 runs in **thinking mode by default**, emitting a `<think>…</think>` block before its answer. To get a direct response, disable it with `"chat_template_kwargs": {"enable_thinking": false}` in your request (or `"enable_thinking": false` on Alibaba Cloud Model Studio). It also adds a "thinking preservation" option that retains reasoning context across turns.
* **Native tool-calling:** Yes. On vLLM/SGLang, set `--tool-call-parser qwen3_coder` (with `--reasoning-parser qwen3`); Ollama applies the template for you. Built for agentic coding — repository-level reasoning and frontend workflows.
* **Vision:** Multimodal — accepts text, images, and video.
* **Long context:** 262,144 tokens natively, extensible to \~1M. Keep at least 128K for complex reasoning tasks.

## Connect it with Tokios

Once Qwen3.6 is running locally, pair the Tokios connector so you can reach it as `https://api.tokios.com` from Claude Code, Cline, or any other compatible client — instead of only from `localhost:11434`.

**1. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Install and run the connector on the machine running Ollama:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Point the connector at Ollama's /v1 API**

Ollama exposes an OpenAI-compatible surface at `/v1`. For a single-model setup, the console's Setup wizard fills this in automatically as `http://127.0.0.1:11434/v1`. For multiple models, set it explicitly in `tokios-connector.json` — see [Connector config](https://docs.tokios.com/reference/connector-config.md).

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the model and choose a public deployment name, for example `qwen3.6-tunnel`. This name — not Ollama's local model id — is what clients send in the `model` field.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.6-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Refactor this function and explain your reasoning."}]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3.6-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`qwen3.6-tunnel` above), never Ollama's local model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) and [Use Cline](https://docs.tokios.com/guides/cline.md) for full client setup.

## Caveats

* Qwen3.6 emits `<think>…</think>` reasoning by default. If a downstream client shows the raw thinking or you want lower latency, disable thinking as shown above.
* Tool-call behavior depends on the runtime applying the `qwen3_coder` parser. Ollama handles this for you; if you self-host with vLLM/SGLang, set the flags above or tool calls won't parse.
* The `35B-A3B` MoE needs all 35B parameters in memory even though only 3B are active per token — size your hardware off the full download, not the active count.

> [!NOTE]
>
> **New same-class alternative.** If you are weighing the `27B` dense build for agent work, Meta's [Muse Glimmer 30B](https://docs.tokios.com/models/muse-glimmer.md) is now a third option at that tier — also dense, also Apache 2.0, and trained specifically for tool calling and multi-step task completion. Meta benchmarks it against Qwen3.6 `27B` directly and its numbers favour Muse Glimmer, but they are vendor-run with no independent reproduction yet. Qwen3.6 has the longer track record and the wider variant range; test both before you commit an agent workflow to either.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> Match Qwen3.6 to your available VRAM before you pull it.

> [card] [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Which of Qwen3.6's formats — `GGUF`, `NVFP4`, `MLX` — runs on your hardware.

> [card] [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md)
>
> See how Qwen3.6 compares to other local models by task.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.