> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Qwen3.8-27B: release date, hardware, local deployment

> Qwen3.8-27B is out under Apache-2.0 with native vision and 262K context. GGUF is 17.11 GB at Q4_K_M, so a 24 GB GPU runs it — Ollama tag is live.

> [!NOTE]
>
> Last verified: 2026-10-14

**Qwen3.8-27B is released.** The weights went live on 2026-08-14 at [Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B) and [ModelScope](https://modelscope.cn/models/Qwen/Qwen3.8-27B) under **Apache-2.0**, alongside an official FP8 build. It is a dense `27B` **native vision-language model** — text, image, and video input — with a switchable thinking mode, tunable reasoning depth, and a 262K-token native context extensible to about 1M. This page covers its download, hardware requirements, and local deployment.

> [!NOTE]
>
> GGUF builds landed the same day: [unsloth/Qwen3.8-27B-GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) publishes the full quantization ladder plus the vision projector, also under Apache-2.0. The Ollama tag is now live at [ollama.com/library/qwen3.8/tags](https://ollama.com/library/qwen3.8/tags).

## Qwen3.8-27B release date and status

| Item | Status |
| - | - |
| Released | 2026-08-14, on [Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B) and [ModelScope](https://modelscope.cn/models/Qwen/Qwen3.8-27B) |
| License | **Apache-2.0** |
| Architecture | Dense, `27B`, hybrid Gated DeltaNet + Gated Attention, 64 layers |
| Modality | Native vision-language: text, image, and video input |
| Thinking mode | Switchable (`enable_thinking`), with `reasoning_effort` levels `xhigh`, `medium`, `low` |
| Context window | 262,144 native, extensible to \~1M |
| Official downloads | `Qwen/Qwen3.8-27B` (BF16, 55.6 GB) and `Qwen/Qwen3.8-27B-FP8` (30.9 GB) |
| Runtime support | Ollama, Transformers, vLLM, SGLang, TokenSpeed (per model card); llama.cpp via the community GGUF |
| GGUF | [unsloth/Qwen3.8-27B-GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) — `Q4_K_M` 17.11 GB, plus `mmproj` for vision |
| Ollama tag | `qwen3.8:27b` and `qwen3.8:27b-q4_K_M` — [ollama.com/library/qwen3.8/tags](https://ollama.com/library/qwen3.8/tags) |
| Tokios-verified serving command | `vllm serve "Qwen/Qwen3.8-27B"` |

> [!NOTE]
>
> Confirm the flags for your vLLM version against the [model card](https://huggingface.co/Qwen/Qwen3.8-27B).

The originally announced window — the week beginning 2026-08-10 — passed with only the 2.4T flagship shipping; the 27B followed on 2026-08-14.

Qwen3.8-27B is the local successor to [Qwen3.6](https://docs.tokios.com/models/qwen3.6.md), and its Apache-2.0 license carries over. If you need a path this site has already verified end-to-end with Tokios, Qwen3.6 remains that model until the tested Qwen3.8 commands land here; if you are comfortable with vLLM or SGLang, nothing blocks running Qwen3.8-27B today.

## Qwen3.8-27B hardware requirements

Every figure below is a measured file size, not an estimate. The BF16 release weighs **55.6 GB** and the official FP8 build **30.9 GB**; the [Unsloth GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) supplies the quantized ladder, where `Q4_K_M` lands at **17.11 GB**.

| Precision | Download | Practical floor |
| - | - | - |
| `IQ2_XXS` | 9.01 GB | 12 GB VRAM — quality cost is real at 2-bit |
| `Q3_K_M` | 13.82 GB | 16 GB VRAM |
| `IQ4_XS` | 15.71 GB | 16 GB VRAM, short context |
| `Q4_K_M` | 17.11 GB | 24 GB VRAM for a comfortable context |
| `UD-Q4_K_XL` | 17.92 GB | 24 GB VRAM — Unsloth's recommended build |
| `Q5_K_M` | 19.83 GB | 24 GB VRAM, shorter context |
| `Q6_K` | 22.88 GB | 32 GB VRAM |
| `Q8_0` | 29.05 GB | 32 GB VRAM, or unified memory |
| FP8 (official) | 30.9 GB | 48 GB VRAM, or a unified-memory pool |
| BF16 (official) | 55.6 GB | 64 GB and up, or multiple GPUs |

Add **0.93 GB** for the `mmproj` projector if you want image and video input — see [vision in llama.cpp](#vision-needs-the-mmproj-file) below.

### Fit by device

Every verdict below assumes a `Q4_K_M` build.

| Target | Verdict |
| - | - |
| RTX 5080 / 4080 (16 GB) | Workable at `Q3_K_M` or `IQ4_XS`, short context |
| RTX 3090 / 4090 (24 GB) | Comfortable |
| RTX 5090 (32 GB, Blackwell) | Comfortable, longer context |
| RTX 6000 Ada / A6000 (48 GB) | Comfortable, long context |
| Apple Silicon (36 GB and up, unified) | Comfortable |
| AMD Strix Halo (ROCm/Vulkan, up to 96 GB) | Comfortable, long context |
| DGX Spark (128 GB unified, Blackwell) | Comfortable, very long context |
| Two 24 GB GPUs (48 GB pooled) | Comfortable, with headroom to spare |

### Can a DGX Spark run Qwen3.8-27B?

Yes, with room to spare — a [DGX Spark](https://docs.tokios.com/use-cases/dgx-spark.md) pairs 128 GB of unified memory with a Blackwell GPU, so even the unquantized 55.6 GB BF16 release fits outright, and the official 30.9 GB FP8 build leaves roughly 90 GB for KV cache, which is what long context and concurrent sessions actually consume.

You now have a choice of format. FP8 through vLLM or SGLang keeps the most quality, while the 17.11 GB `Q4_K_M` GGUF through llama.cpp leaves over 100 GB of the pool for KV cache. NVFP4 would be more efficient still, but no NVFP4 build has been published for this model. See [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) for per-machine tuning.

### Can Strix Halo run Qwen3.8-27B?

Yes. AMD Strix Halo can allocate up to 96 GB to the GPU, so even the 55.6 GB BF16 release fits, and the 17.11 GB `Q4_K_M` GGUF leaves plenty for context.

The binding constraint here is memory bandwidth rather than capacity. The weights load fine, but decode trails a discrete GPU holding the same model, and prompt processing trails it further. Serve it through ROCm or Vulkan — [running vLLM on Strix Halo](https://docs.tokios.com/guides/vllm-strix-halo.md) and [Strix Halo model compatibility](https://docs.tokios.com/guides/strix-halo-model-compatibility.md) cover setup and measured behaviour.

### Can a 24 GB GPU run Qwen3.8-27B?

Yes, and this is the size class the model was built for. An RTX 3090 or 4090 holds the 17.11 GB `Q4_K_M` GGUF with roughly 7 GB spare for KV cache — fine for everyday context lengths, tight beyond that. Neither official safetensors release fits a 24 GB card directly, since FP8 is 30.9 GB, so the GGUF is the path here.

A 16 GB card also works now, at `Q3_K_M` (13.82 GB) or `IQ4_XS` (15.71 GB), with little room left for context or the vision projector. Below that, offloading costs throughput; see [memory and offloading](https://docs.tokios.com/concepts/memory-and-offloading.md).

For two or more cards, an 8-bit build across 48 GB is comfortable and keeps more quality than a single-card `Q4`. [Model sharding](https://docs.tokios.com/guides/model-sharding.md) covers how runtimes split weights.

### Context is a separate budget

The weight figures above cover the model only. KV cache is added on top and grows with context length and concurrency, so a long context can cost more memory than the weights. Size it with [what size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) and see [KV cache and context](https://docs.tokios.com/concepts/kv-cache-and-context.md) for the mechanics.

> [!NOTE]
>
> These are published file sizes, so size from them directly. What they do not include is the KV cache, which grows with context length and concurrency, or the 0.93 GB projector if you need vision.

For device-by-device comparisons, see the [hardware reference](https://docs.tokios.com/guides/hardware-reference.md). For how weight formats map to hardware, see [quantization and hardware](https://docs.tokios.com/models/quantization-and-hardware.md).

## Bonsai 2 27B: ternary compression

[Bonsai 2 27B](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf) is a ternary-compressed variant of Qwen3.8-27B by PrismML. Every weight is stored as `-1`, `0`, or `+1` — roughly 1.58 bits per weight — which shrinks the model from 54 GB to **5.9 GB**. It runs under 6 GB of VRAM on a consumer GPU, the smallest hardware footprint for any 27B-class model.

| Item | Detail |
| - | - |
| Compressed from | Qwen3.8-27B |
| Compressed by | PrismML |
| Weight format | Ternary (`-1`, `0`, `+1`), \~1.58 bits per weight |
| File size | 5.9 GB (vs 54 GB original, vs 17.11 GB at `Q4_K_M`) |
| VRAM | Under 6 GB |
| Quality retention | 98.2% of Qwen3.8-27B (per PrismML) |
| License | Apache-2.0 |
| Formats | GGUF, MLX (2-bit) |
| HuggingFace | [prism-ml/Ternary-Bonsai-2-27B-gguf](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf), [prism-ml/Ternary-Bonsai-2-27B-mlx-2bit](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit) |

Bonsai 2 is best for laptop or single-GPU deployment where every gigabyte counts, and for testing whether extreme compression is viable for your use case. It is text-only — the ternary compression does not include the vision projector, so you lose the image and video input that the full Qwen3.8-27B supports.

> [!WARNING]
>
> Ternary compression is lossy. PrismML reports 98.2% capability retention, but that is an aggregate figure. Tasks that depend on precise numerical reasoning or niche knowledge may show larger degradation. Test your specific workload before relying on Bonsai 2 for production.

> [!TIP]
>
> At 5.9 GB, Bonsai 2 fits a 8 GB phone or a 12 GB laptop GPU with room for context. Pair it with the [Tokios connector](https://docs.tokios.com/quickstart.md) to reach it as `https://api.tokios.com` from any client.

### Bonsai 2 vs standard quantizations

| | Bonsai 2 (ternary) | `Q4_K_M` (GGUF) | `Q3_K_M` (GGUF) |
| - | - | - | - |
| File size | 5.9 GB | 17.11 GB | 13.82 GB |
| VRAM floor | Under 6 GB | 24 GB | 16 GB |
| Quality | 98.2% of base (per PrismML) | Close to base | Visible degradation |
| Vision | No | Yes (with `mmproj`) | Yes (with `mmproj`) |
| Runtime | llama.cpp (GGUF), MLX | llama.cpp, LM Studio, Ollama | llama.cpp, LM Studio, Ollama |

The standard GGUF ladder preserves the full model architecture, including the vision encoder. Bonsai 2 trades vision capability and a small quality margin for a 9x size reduction over the base model and a 3x reduction over `Q4_K_M`.

## Qwen3.8-27B download and Ollama

The official downloads are live in two precisions:

| Repository | Precision | Size | License |
| - | - | - | - |
| [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) | BF16 safetensors | 55.6 GB | Apache-2.0 |
| [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8) | FP8 safetensors | 30.9 GB | Apache-2.0 |

Both are mirrored on [ModelScope](https://modelscope.cn/models/Qwen/Qwen3.8-27B). The model card documents serving through Transformers, vLLM, SGLang, and TokenSpeed. Thinking is switchable per request (`enable_thinking`), with `reasoning_effort` accepting `xhigh`, `medium`, or `low`; Qwen recommends `temperature=1.0, top_p=0.95, top_k=20` with thinking on, and `temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5` with it off.

### GGUF for llama.cpp and LM Studio

Qwen did not publish a GGUF, but [unsloth/Qwen3.8-27B-GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) shipped the same day under Apache-2.0 with the full ladder from `IQ2_XXS` (9.01 GB) to `Q8_0` (29.05 GB), plus BF16. Unsloth publishes the run commands for their build:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL
```

That tag is the 17.92 GB dynamic build they recommend; plain `Q4_K_M` at 17.11 GB is the conventional choice. LM Studio can load the same files.

> [!TIP]
>
> **Ollama is live.** Run `ollama run qwen3.8:27b` for the default quantization, or `ollama run qwen3.8:27b-q4_K_M` for the explicit `Q4_K_M` build. See [ollama.com/library/qwen3.8/tags](https://ollama.com/library/qwen3.8/tags) for the full tag list.

> [!TIP]
>
> **Apple Silicon?** MLX builds of Qwen3.8-27B are available from [mlx-community](https://huggingface.co/mlx-community) on Hugging Face in 4-bit, 8-bit, and BF16 quantizations. Serve them with `mlx_lm.server` for native unified-memory inference on Mac Studio, Mac mini, and MacBook Pro — see [MLX-LM Server](https://docs.tokios.com/setup/supported-backends.md#mlx-lm-server) in the backends reference.

### Vision needs the mmproj file

Image and video input is not part of the main weights file. The GGUF repository publishes a separate **`mmproj-F16.gguf` (0.93 GB)** projector, and llama.cpp needs it loaded alongside the model for any multimodal request — without it you get a text-only model. Budget that extra gigabyte on top of whichever quantization you pick, and note that the official FP8 and BF16 safetensors releases carry the vision encoder inline instead, so they need no separate file.

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run qwen3.8:27b
> ```
>
> Or pin the explicit `Q4_K_M` build:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run qwen3.8:27b-q4_K_M
> ```
>
> Leave Ollama on its default `http://127.0.0.1:11434` and raise `num_ctx` from its small default only as far as your free memory allows. See [Ollama remote access](https://docs.tokios.com/guides/ollama-remote-access.md).

> [tab] llama.cpp
>
> ```bash macOS / Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
> curl -LsSf https://llama.app/install.sh | sh
> llama serve -hf Qwen/Qwen3.8-27B-GGUF
> ```
>
> ```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
> winget install llama.cpp
> llama serve -hf Qwen/Qwen3.8-27B-GGUF
> ```
>
> See [llama.cpp remote access](https://docs.tokios.com/guides/llama-cpp-remote-access.md) to reach it from another machine.

> [tab] vLLM (GPU serving)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve "Qwen/Qwen3.8-27B"
> ```
>
> This serves the BF16 checkpoint, so budget \~56 GB plus KV cache. Confirm the flags for your vLLM version against the model card. See [vLLM remote access](https://docs.tokios.com/guides/vllm-remote-access.md).

## Connect it with Tokios

A local model is one you want to reach when you are not at the machine running it. Tokios sits after your runtime: it does not download the model or replace Ollama.

**1. Keep your local runtime running**

Start Qwen3.8-27B in Ollama, llama.cpp, vLLM, or another OpenAI-compatible runtime and leave it bound to loopback.

**2. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Run the installer on the machine that runs the model:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the upstream model id (for example `qwen3.8:27b`) under a public deployment name like `qwen3.8`. Clients send the deployment name in the `model` field, never the local id — so re-quantizing later changes nothing on the client side.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it is shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Describe this image in detail."}
    ]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3.8
```

## Qwen3.8-27B vs Qwen3.8-Max

The same announcement covered a second, far larger model — and that one has now shipped. **Qwen3.8-2.4T-A95B**, the open-weight model behind the hosted Qwen3.8-Max, is live on [Hugging Face](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B) and [ModelScope](https://modelscope.cn/models/Qwen/Qwen3.8-2.4T-A95B) with a model card, BF16 safetensors, and an official FP8 variant. The model card settles what community reporting could only estimate: a mixture-of-experts model with **2.4T total parameters and 95B activated**, a 262K-token native context extensible to about 1M, and — unlike the 27B — **text-only input with thinking always on**. Vision input, a non-thinking mode, and 1M default context are features of the hosted Qwen3.8-Max API, not of the open weights. The license is Qwen's custom `qwen3.8-max` license, not Apache-2.0, so read its terms before commercial use.

That makes it the first open-weight release at Qwen's Max tier, after Qwen3.7-Max and Qwen3.6-Max-Preview both shipped API-only. It does not make it a homelab model.

| | Qwen3.8-27B | Qwen3.8-2.4T-A95B |
| - | - | - |
| Status | Released 2026-08-14 | Released, week of 2026-08-10 |
| Parameters | `27B` dense | `2.4T` total, `95B` active, MoE |
| Modality | Text, image, and video input | Text only (vision is hosted-Max only) |
| Thinking | Switchable, tunable depth | Always on |
| Context window | 262K native, \~1M extended | 262K native, \~1M extended |
| License | Apache-2.0 | Custom `qwen3.8-max` |
| Runs on one desktop GPU | Yes | No |
| Weight floor | 30.9 GB FP8 official, \~17 GB at 4-bit | \~1.2 TB at 4-bit |
| Who it is for | Local and homelab use | Datacenter and cluster serving |

The split matters for how you size each one. A dense model loads every parameter for every token, so the `27B` behaves predictably: memory tracks its parameter count, and decode speed tracks it too. The `2.4T` is the opposite case — only `95B` of it activates per token, which keeps compute manageable while the full weight set still has to be resident.

Alibaba also serves the model through its hosted Token Plan, Qoder, and QoderWork products — see [Alibaba's WAIC announcement](https://www.alibabagroup.com/en-US/document-2016703577908576256). Hosted access is not a Tokios deployment; Tokios fronts a model **you** run.

### Can a DGX Spark or Strix Halo run Qwen3.8-Max?

No — not a single machine, and not a small cluster either. In a MoE model every expert weight must stay resident even though only a fraction activates per token — here `95B` of `2.4T` — so memory scales with total parameters rather than active ones. At 4 bits, `2.4T` parameters need roughly **1.2 TB** for weights alone.

| Configuration | Pooled memory | Verdict |
| - | - | - |
| One DGX Spark or Strix Halo (128 GB) | 128 GB | Nowhere close |
| Four DGX Sparks | 512 GB | Still far short |
| Ten DGX Sparks | 1.28 TB | Marginal, almost no context |
| Sixteen H100 (80 GB) | 1.28 TB | Marginal |
| Thirty-two H100 (80 GB) | 2.56 TB | Workable at 8-bit |

Those are weight-only floors. KV cache, runtime buffers, and system reservation all come on top, so treat the marginal rows as unusable in practice. If you want a Qwen3.8 you can actually serve at home, that is the `27B`.

> [!TIP]
>
> Want the largest Qwen you can serve on one machine today? [Qwen3.6-35B-A3B](https://docs.tokios.com/models/qwen3.6.md) is a released MoE that fits a single 128 GB unified-memory box. Size it from total weight size, not active parameters.

## Capabilities

* **Native vision.** Text, image, and video input — the vision encoder is part of the model architecture, not a bolt-on. For GGUF in llama.cpp, load the separate `mmproj` projector to enable it.
* **Switchable thinking mode.** Toggle reasoning on or off per request with `enable_thinking`, and tune the depth with `reasoning_effort` at `xhigh`, `medium`, or `low`.
* **Long context.** 262K tokens native, extensible to about 1M.
* **Multilingual.** Qwen models support a wide range of languages.
* **Tunable sampling.** Qwen recommends `temperature=1.0, top_p=0.95, top_k=20` with thinking on, and `temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5` with it off.

## Caveats

* **vLLM or SGLang for safetensors.** The official BF16 and FP8 releases serve through vLLM, SGLang, TokenSpeed, or Transformers per the model card. For Ollama, llama.cpp, or LM Studio, use the [Unsloth GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF).
* **Benchmarks are vendor-reported.** Qwen published them on release day. Wait for independent evaluations before treating them as settled.
* **Vision support depends on your runtime.** Not every backend handles multimodal input yet. Tokios passes image and video content through to the upstream endpoint — it does not add capabilities your runtime lacks. Check [supported backends](https://docs.tokios.com/setup/supported-backends.md).
* **262K tokens is a ceiling, not a target.** Long contexts add KV-cache memory on top of the model weights, and on a 24 GB card that is the headroom left after the 17.11 GB `Q4_K_M` build.
* **Sampling settings differ by mode.** Do not carry over settings from another model without testing — Qwen's recommendations change depending on whether thinking is on or off.

## FAQ

> [accordion] How much VRAM does Qwen3.8-27B need?
>
> Depends on the format. The [Unsloth GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) is **17.11 GB at `Q4_K_M`**, which a 24 GB card holds with room for everyday context, and 16 GB cards can run `Q3_K_M` (13.82 GB) or `IQ4_XS` (15.71 GB). The official safetensors releases are larger: **30.9 GB at FP8** and **55.6 GB at BF16**. Add 0.93 GB for the vision projector when using GGUF.

> [accordion] Can Qwen3.8-27B run on a 24 GB GPU?
>
> Yes — this is the size class the model was built for. An RTX 3090 or 4090 holds the 17.11 GB `Q4_K_M` GGUF with roughly 7 GB spare for KV cache. Neither official safetensors release fits a 24 GB card directly (FP8 is 30.9 GB), so the GGUF is the path for a single consumer GPU.

> [accordion] When did the Qwen3.8 weights ship?
>
> Both are out. The 2.4T flagship (`Qwen/Qwen3.8-2.4T-A95B`, plus FP8) shipped first, in the week of 2026-08-10. Qwen3.8-27B followed on **2026-08-14**, with BF16 and FP8 repositories live on [Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B) and [ModelScope](https://modelscope.cn/models/Qwen/Qwen3.8-27B).

> [accordion] What is the difference between Qwen3.8-27B and Qwen3.8-Max?
>
> Size, and therefore who can run it — but also modality and licensing, in the direction you might not expect. The `27B` is a dense **vision-language** model (image and video input, switchable thinking) under **Apache-2.0**, sized for a single desktop GPU or unified-memory machine. The open-weight Max-class model, `Qwen3.8-2.4T-A95B`, is a mixture-of-experts model at `2.4T` total parameters (`95B` active) that is **text-only** with thinking always on, released under a custom license, with a 4-bit weight floor around 1.2 TB — cluster hardware, not homelab. Vision input and 1M default context on the big model belong to the hosted Qwen3.8-Max API built on it.

> [accordion] Is Qwen3.8 open source?
>
> The 27B ships under **Apache-2.0** — the same permissive license as Qwen3.6, fine for commercial use. The 2.4T flagship instead uses Qwen's custom `qwen3.8-max` license with scale-tiered terms, so read it before commercial deployment. Same generation, two different licenses — check the model you actually plan to run.

> [accordion] Is there an Ollama tag for Qwen3.8 yet?
>
> Yes — `ollama run qwen3.8:27b` works, and `qwen3.8:27b-q4_K_M` is also available. Check [ollama.com/library/qwen3.8/tags](https://ollama.com/library/qwen3.8/tags) for the full tag list. GGUF files also exist via [unsloth/Qwen3.8-27B-GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) for llama.cpp and LM Studio, and Qwen's own releases run on vLLM or SGLang.

> [accordion] Why does the serving command only show vLLM?
>
> The official BF16 and FP8 safetensors releases require vLLM, SGLang, TokenSpeed, or Transformers per the model card. If you want broader runtime support — Ollama, llama.cpp, LM Studio — use the [Unsloth GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF), which ships the full quantization ladder under Apache-2.0.

> [accordion] How do I use Qwen3.8-27B from another machine?
>
> Serve it locally on any OpenAI-compatible runtime, then pair the [Tokios connector](https://docs.tokios.com/quickstart.md) so the machine holding the GPU needs no inbound ports. You call `https://api.tokios.com` with an `sk-tok-…` key from any OpenAI- or Anthropic-compatible client, sending your deployment name in the `model` field rather than the upstream model id.

> [card] [Qwen3.6](https://docs.tokios.com/models/qwen3.6.md)
>
> Run a verified local Qwen model today.

> [card] [Model fit](https://docs.tokios.com/guides/model-fit.md)
>
> Estimate the model and context size your hardware can hold.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair the connector and register a local deployment.

> [card] [Quantization and hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Learn what a model's release formats mean for your hardware.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.