Skip to main content
Last verified: 2026-10-14
Qwen3.8-27B is released. The weights went live on 2026-08-14 at Hugging Face and ModelScope under Apache-2.0, alongside an official FP8 build. It is a dense 27B native vision-language model — text, image, and video input — with a switchable thinking mode, tunable reasoning depth, and a 262K-token native context extensible to about 1M. This page covers its download, hardware requirements, and local deployment.
GGUF builds landed the same day: unsloth/Qwen3.8-27B-GGUF publishes the full quantization ladder plus the vision projector, also under Apache-2.0. The Ollama tag is now live at ollama.com/library/qwen3.8/tags.

Qwen3.8-27B release date and status

Confirm the flags for your vLLM version against the model card.
The originally announced window — the week beginning 2026-08-10 — passed with only the 2.4T flagship shipping; the 27B followed on 2026-08-14. Qwen3.8-27B is the local successor to Qwen3.6, and its Apache-2.0 license carries over. If you need a path this site has already verified end-to-end with Tokios, Qwen3.6 remains that model until the tested Qwen3.8 commands land here; if you are comfortable with vLLM or SGLang, nothing blocks running Qwen3.8-27B today.

Qwen3.8-27B hardware requirements

Every figure below is a measured file size, not an estimate. The BF16 release weighs 55.6 GB and the official FP8 build 30.9 GB; the Unsloth GGUF supplies the quantized ladder, where Q4_K_M lands at 17.11 GB. Add 0.93 GB for the mmproj projector if you want image and video input — see vision in llama.cpp below.

Fit by device

Every verdict below assumes a Q4_K_M build.

Can a DGX Spark run Qwen3.8-27B?

Yes, with room to spare — a DGX Spark pairs 128 GB of unified memory with a Blackwell GPU, so even the unquantized 55.6 GB BF16 release fits outright, and the official 30.9 GB FP8 build leaves roughly 90 GB for KV cache, which is what long context and concurrent sessions actually consume. You now have a choice of format. FP8 through vLLM or SGLang keeps the most quality, while the 17.11 GB Q4_K_M GGUF through llama.cpp leaves over 100 GB of the pool for KV cache. NVFP4 would be more efficient still, but no NVFP4 build has been published for this model. See DGX Spark model compatibility for per-machine tuning.

Can Strix Halo run Qwen3.8-27B?

Yes. AMD Strix Halo can allocate up to 96 GB to the GPU, so even the 55.6 GB BF16 release fits, and the 17.11 GB Q4_K_M GGUF leaves plenty for context. The binding constraint here is memory bandwidth rather than capacity. The weights load fine, but decode trails a discrete GPU holding the same model, and prompt processing trails it further. Serve it through ROCm or Vulkan — running vLLM on Strix Halo and Strix Halo model compatibility cover setup and measured behaviour.

Can a 24 GB GPU run Qwen3.8-27B?

Yes, and this is the size class the model was built for. An RTX 3090 or 4090 holds the 17.11 GB Q4_K_M GGUF with roughly 7 GB spare for KV cache — fine for everyday context lengths, tight beyond that. Neither official safetensors release fits a 24 GB card directly, since FP8 is 30.9 GB, so the GGUF is the path here. A 16 GB card also works now, at Q3_K_M (13.82 GB) or IQ4_XS (15.71 GB), with little room left for context or the vision projector. Below that, offloading costs throughput; see memory and offloading. For two or more cards, an 8-bit build across 48 GB is comfortable and keeps more quality than a single-card Q4. Model sharding covers how runtimes split weights.

Context is a separate budget

The weight figures above cover the model only. KV cache is added on top and grows with context length and concurrency, so a long context can cost more memory than the weights. Size it with what size model fits your GPU? and see KV cache and context for the mechanics.
These are published file sizes, so size from them directly. What they do not include is the KV cache, which grows with context length and concurrency, or the 0.93 GB projector if you need vision.
For device-by-device comparisons, see the hardware reference. For how weight formats map to hardware, see quantization and hardware.

Bonsai 2 27B: ternary compression

Bonsai 2 27B is a ternary-compressed variant of Qwen3.8-27B by PrismML. Every weight is stored as -1, 0, or +1 — roughly 1.58 bits per weight — which shrinks the model from 54 GB to 5.9 GB. It runs under 6 GB of VRAM on a consumer GPU, the smallest hardware footprint for any 27B-class model. Bonsai 2 is best for laptop or single-GPU deployment where every gigabyte counts, and for testing whether extreme compression is viable for your use case. It is text-only — the ternary compression does not include the vision projector, so you lose the image and video input that the full Qwen3.8-27B supports.
Ternary compression is lossy. PrismML reports 98.2% capability retention, but that is an aggregate figure. Tasks that depend on precise numerical reasoning or niche knowledge may show larger degradation. Test your specific workload before relying on Bonsai 2 for production.
At 5.9 GB, Bonsai 2 fits a 8 GB phone or a 12 GB laptop GPU with room for context. Pair it with the Tokios connector to reach it as https://api.tokios.com from any client.

Bonsai 2 vs standard quantizations

The standard GGUF ladder preserves the full model architecture, including the vision encoder. Bonsai 2 trades vision capability and a small quality margin for a 9x size reduction over the base model and a 3x reduction over Q4_K_M.

Qwen3.8-27B download and Ollama

The official downloads are live in two precisions: Both are mirrored on ModelScope. The model card documents serving through Transformers, vLLM, SGLang, and TokenSpeed. Thinking is switchable per request (enable_thinking), with reasoning_effort accepting xhigh, medium, or low; Qwen recommends temperature=1.0, top_p=0.95, top_k=20 with thinking on, and temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5 with it off.

GGUF for llama.cpp and LM Studio

Qwen did not publish a GGUF, but unsloth/Qwen3.8-27B-GGUF shipped the same day under Apache-2.0 with the full ladder from IQ2_XXS (9.01 GB) to Q8_0 (29.05 GB), plus BF16. Unsloth publishes the run commands for their build:
That tag is the 17.92 GB dynamic build they recommend; plain Q4_K_M at 17.11 GB is the conventional choice. LM Studio can load the same files.
Ollama is live. Run ollama run qwen3.8:27b for the default quantization, or ollama run qwen3.8:27b-q4_K_M for the explicit Q4_K_M build. See ollama.com/library/qwen3.8/tags for the full tag list.
Apple Silicon? MLX builds of Qwen3.8-27B are available from mlx-community on Hugging Face in 4-bit, 8-bit, and BF16 quantizations. Serve them with mlx_lm.server for native unified-memory inference on Mac Studio, Mac mini, and MacBook Pro — see MLX-LM Server in the backends reference.

Vision needs the mmproj file

Image and video input is not part of the main weights file. The GGUF repository publishes a separate mmproj-F16.gguf (0.93 GB) projector, and llama.cpp needs it loaded alongside the model for any multimodal request — without it you get a text-only model. Budget that extra gigabyte on top of whichever quantization you pick, and note that the official FP8 and BF16 safetensors releases carry the vision encoder inline instead, so they need no separate file.

Run it — beginner to advanced

Or pin the explicit Q4_K_M build:
Leave Ollama on its default http://127.0.0.1:11434 and raise num_ctx from its small default only as far as your free memory allows. See Ollama remote access.

Connect it with Tokios

A local model is one you want to reach when you are not at the machine running it. Tokios sits after your runtime: it does not download the model or replace Ollama.
1

Keep your local runtime running

Start Qwen3.8-27B in Ollama, llama.cpp, vLLM, or another OpenAI-compatible runtime and leave it bound to loopback.
2

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing to get a one-time claim code (TKS-XXXX-XXXX). Run the installer on the machine that runs the model:
Windows
macOS
Linux
Then approve the device on the Setup or Connectors tab.
3

Register a deployment

In the Models tab, register the upstream model id (for example qwen3.8:27b) under a public deployment name like qwen3.8. Clients send the deployment name in the model field, never the local id — so re-quantizing later changes nothing on the client side.
4

Create an API key

In the Keys tab, click Create key. Copy the sk-tok-… key now — it is shown only once.

Call it from anywhere

Qwen3.8-27B vs Qwen3.8-Max

The same announcement covered a second, far larger model — and that one has now shipped. Qwen3.8-2.4T-A95B, the open-weight model behind the hosted Qwen3.8-Max, is live on Hugging Face and ModelScope with a model card, BF16 safetensors, and an official FP8 variant. The model card settles what community reporting could only estimate: a mixture-of-experts model with 2.4T total parameters and 95B activated, a 262K-token native context extensible to about 1M, and — unlike the 27B — text-only input with thinking always on. Vision input, a non-thinking mode, and 1M default context are features of the hosted Qwen3.8-Max API, not of the open weights. The license is Qwen’s custom qwen3.8-max license, not Apache-2.0, so read its terms before commercial use. That makes it the first open-weight release at Qwen’s Max tier, after Qwen3.7-Max and Qwen3.6-Max-Preview both shipped API-only. It does not make it a homelab model. The split matters for how you size each one. A dense model loads every parameter for every token, so the 27B behaves predictably: memory tracks its parameter count, and decode speed tracks it too. The 2.4T is the opposite case — only 95B of it activates per token, which keeps compute manageable while the full weight set still has to be resident. Alibaba also serves the model through its hosted Token Plan, Qoder, and QoderWork products — see Alibaba’s WAIC announcement. Hosted access is not a Tokios deployment; Tokios fronts a model you run.

Can a DGX Spark or Strix Halo run Qwen3.8-Max?

No — not a single machine, and not a small cluster either. In a MoE model every expert weight must stay resident even though only a fraction activates per token — here 95B of 2.4T — so memory scales with total parameters rather than active ones. At 4 bits, 2.4T parameters need roughly 1.2 TB for weights alone. Those are weight-only floors. KV cache, runtime buffers, and system reservation all come on top, so treat the marginal rows as unusable in practice. If you want a Qwen3.8 you can actually serve at home, that is the 27B.
Want the largest Qwen you can serve on one machine today? Qwen3.6-35B-A3B is a released MoE that fits a single 128 GB unified-memory box. Size it from total weight size, not active parameters.

Capabilities

  • Native vision. Text, image, and video input — the vision encoder is part of the model architecture, not a bolt-on. For GGUF in llama.cpp, load the separate mmproj projector to enable it.
  • Switchable thinking mode. Toggle reasoning on or off per request with enable_thinking, and tune the depth with reasoning_effort at xhigh, medium, or low.
  • Long context. 262K tokens native, extensible to about 1M.
  • Multilingual. Qwen models support a wide range of languages.
  • Tunable sampling. Qwen recommends temperature=1.0, top_p=0.95, top_k=20 with thinking on, and temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5 with it off.

Caveats

  • vLLM or SGLang for safetensors. The official BF16 and FP8 releases serve through vLLM, SGLang, TokenSpeed, or Transformers per the model card. For Ollama, llama.cpp, or LM Studio, use the Unsloth GGUF.
  • Benchmarks are vendor-reported. Qwen published them on release day. Wait for independent evaluations before treating them as settled.
  • Vision support depends on your runtime. Not every backend handles multimodal input yet. Tokios passes image and video content through to the upstream endpoint — it does not add capabilities your runtime lacks. Check supported backends.
  • 262K tokens is a ceiling, not a target. Long contexts add KV-cache memory on top of the model weights, and on a 24 GB card that is the headroom left after the 17.11 GB Q4_K_M build.
  • Sampling settings differ by mode. Do not carry over settings from another model without testing — Qwen’s recommendations change depending on whether thinking is on or off.

FAQ

Depends on the format. The Unsloth GGUF is 17.11 GB at Q4_K_M, which a 24 GB card holds with room for everyday context, and 16 GB cards can run Q3_K_M (13.82 GB) or IQ4_XS (15.71 GB). The official safetensors releases are larger: 30.9 GB at FP8 and 55.6 GB at BF16. Add 0.93 GB for the vision projector when using GGUF.
Yes — this is the size class the model was built for. An RTX 3090 or 4090 holds the 17.11 GB Q4_K_M GGUF with roughly 7 GB spare for KV cache. Neither official safetensors release fits a 24 GB card directly (FP8 is 30.9 GB), so the GGUF is the path for a single consumer GPU.
Both are out. The 2.4T flagship (Qwen/Qwen3.8-2.4T-A95B, plus FP8) shipped first, in the week of 2026-08-10. Qwen3.8-27B followed on 2026-08-14, with BF16 and FP8 repositories live on Hugging Face and ModelScope.
Size, and therefore who can run it — but also modality and licensing, in the direction you might not expect. The 27B is a dense vision-language model (image and video input, switchable thinking) under Apache-2.0, sized for a single desktop GPU or unified-memory machine. The open-weight Max-class model, Qwen3.8-2.4T-A95B, is a mixture-of-experts model at 2.4T total parameters (95B active) that is text-only with thinking always on, released under a custom license, with a 4-bit weight floor around 1.2 TB — cluster hardware, not homelab. Vision input and 1M default context on the big model belong to the hosted Qwen3.8-Max API built on it.
The 27B ships under Apache-2.0 — the same permissive license as Qwen3.6, fine for commercial use. The 2.4T flagship instead uses Qwen’s custom qwen3.8-max license with scale-tiered terms, so read it before commercial deployment. Same generation, two different licenses — check the model you actually plan to run.
Yes — ollama run qwen3.8:27b works, and qwen3.8:27b-q4_K_M is also available. Check ollama.com/library/qwen3.8/tags for the full tag list. GGUF files also exist via unsloth/Qwen3.8-27B-GGUF for llama.cpp and LM Studio, and Qwen’s own releases run on vLLM or SGLang.
The official BF16 and FP8 safetensors releases require vLLM, SGLang, TokenSpeed, or Transformers per the model card. If you want broader runtime support — Ollama, llama.cpp, LM Studio — use the Unsloth GGUF, which ships the full quantization ladder under Apache-2.0.
Serve it locally on any OpenAI-compatible runtime, then pair the Tokios connector so the machine holding the GPU needs no inbound ports. You call https://api.tokios.com with an sk-tok-… key from any OpenAI- or Anthropic-compatible client, sending your deployment name in the model field rather than the upstream model id.

Qwen3.6

Run a verified local Qwen model today.

Model fit

Estimate the model and context size your hardware can hold.

Quickstart

Pair the connector and register a local deployment.

Quantization and hardware

Learn what a model’s release formats mean for your hardware.