Last verified: 2026-10-14
27B native vision-language model — text, image, and video input — with a switchable thinking mode, tunable reasoning depth, and a 262K-token native context extensible to about 1M. This page covers its download, hardware requirements, and local deployment.
GGUF builds landed the same day: unsloth/Qwen3.8-27B-GGUF publishes the full quantization ladder plus the vision projector, also under Apache-2.0. The Ollama tag is now live at ollama.com/library/qwen3.8/tags.
Qwen3.8-27B release date and status
Confirm the flags for your vLLM version against the model card.
Qwen3.8-27B hardware requirements
Every figure below is a measured file size, not an estimate. The BF16 release weighs 55.6 GB and the official FP8 build 30.9 GB; the Unsloth GGUF supplies the quantized ladder, whereQ4_K_M lands at 17.11 GB.
Add 0.93 GB for the
mmproj projector if you want image and video input — see vision in llama.cpp below.
Fit by device
Every verdict below assumes aQ4_K_M build.
Can a DGX Spark run Qwen3.8-27B?
Yes, with room to spare — a DGX Spark pairs 128 GB of unified memory with a Blackwell GPU, so even the unquantized 55.6 GB BF16 release fits outright, and the official 30.9 GB FP8 build leaves roughly 90 GB for KV cache, which is what long context and concurrent sessions actually consume. You now have a choice of format. FP8 through vLLM or SGLang keeps the most quality, while the 17.11 GBQ4_K_M GGUF through llama.cpp leaves over 100 GB of the pool for KV cache. NVFP4 would be more efficient still, but no NVFP4 build has been published for this model. See DGX Spark model compatibility for per-machine tuning.
Can Strix Halo run Qwen3.8-27B?
Yes. AMD Strix Halo can allocate up to 96 GB to the GPU, so even the 55.6 GB BF16 release fits, and the 17.11 GBQ4_K_M GGUF leaves plenty for context.
The binding constraint here is memory bandwidth rather than capacity. The weights load fine, but decode trails a discrete GPU holding the same model, and prompt processing trails it further. Serve it through ROCm or Vulkan — running vLLM on Strix Halo and Strix Halo model compatibility cover setup and measured behaviour.
Can a 24 GB GPU run Qwen3.8-27B?
Yes, and this is the size class the model was built for. An RTX 3090 or 4090 holds the 17.11 GBQ4_K_M GGUF with roughly 7 GB spare for KV cache — fine for everyday context lengths, tight beyond that. Neither official safetensors release fits a 24 GB card directly, since FP8 is 30.9 GB, so the GGUF is the path here.
A 16 GB card also works now, at Q3_K_M (13.82 GB) or IQ4_XS (15.71 GB), with little room left for context or the vision projector. Below that, offloading costs throughput; see memory and offloading.
For two or more cards, an 8-bit build across 48 GB is comfortable and keeps more quality than a single-card Q4. Model sharding covers how runtimes split weights.
Context is a separate budget
The weight figures above cover the model only. KV cache is added on top and grows with context length and concurrency, so a long context can cost more memory than the weights. Size it with what size model fits your GPU? and see KV cache and context for the mechanics.These are published file sizes, so size from them directly. What they do not include is the KV cache, which grows with context length and concurrency, or the 0.93 GB projector if you need vision.
Bonsai 2 27B: ternary compression
Bonsai 2 27B is a ternary-compressed variant of Qwen3.8-27B by PrismML. Every weight is stored as-1, 0, or +1 — roughly 1.58 bits per weight — which shrinks the model from 54 GB to 5.9 GB. It runs under 6 GB of VRAM on a consumer GPU, the smallest hardware footprint for any 27B-class model.
Bonsai 2 is best for laptop or single-GPU deployment where every gigabyte counts, and for testing whether extreme compression is viable for your use case. It is text-only — the ternary compression does not include the vision projector, so you lose the image and video input that the full Qwen3.8-27B supports.
Bonsai 2 vs standard quantizations
The standard GGUF ladder preserves the full model architecture, including the vision encoder. Bonsai 2 trades vision capability and a small quality margin for a 9x size reduction over the base model and a 3x reduction over
Q4_K_M.
Qwen3.8-27B download and Ollama
The official downloads are live in two precisions:
Both are mirrored on ModelScope. The model card documents serving through Transformers, vLLM, SGLang, and TokenSpeed. Thinking is switchable per request (
enable_thinking), with reasoning_effort accepting xhigh, medium, or low; Qwen recommends temperature=1.0, top_p=0.95, top_k=20 with thinking on, and temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5 with it off.
GGUF for llama.cpp and LM Studio
Qwen did not publish a GGUF, but unsloth/Qwen3.8-27B-GGUF shipped the same day under Apache-2.0 with the full ladder fromIQ2_XXS (9.01 GB) to Q8_0 (29.05 GB), plus BF16. Unsloth publishes the run commands for their build:
Q4_K_M at 17.11 GB is the conventional choice. LM Studio can load the same files.
Vision needs the mmproj file
Image and video input is not part of the main weights file. The GGUF repository publishes a separatemmproj-F16.gguf (0.93 GB) projector, and llama.cpp needs it loaded alongside the model for any multimodal request — without it you get a text-only model. Budget that extra gigabyte on top of whichever quantization you pick, and note that the official FP8 and BF16 safetensors releases carry the vision encoder inline instead, so they need no separate file.
Run it — beginner to advanced
- Ollama (easiest)
- llama.cpp
- vLLM (GPU serving)
Q4_K_M build:http://127.0.0.1:11434 and raise num_ctx from its small default only as far as your free memory allows. See Ollama remote access.Connect it with Tokios
A local model is one you want to reach when you are not at the machine running it. Tokios sits after your runtime: it does not download the model or replace Ollama.1
Keep your local runtime running
Start Qwen3.8-27B in Ollama, llama.cpp, vLLM, or another OpenAI-compatible runtime and leave it bound to loopback.
2
Pair the Tokios connector
Sign in at tokios.com/console, open the Setup tab, and click Start pairing to get a one-time claim code (Then approve the device on the Setup or Connectors tab.
TKS-XXXX-XXXX). Run the installer on the machine that runs the model:Windows
macOS
Linux
3
Register a deployment
In the Models tab, register the upstream model id (for example
qwen3.8:27b) under a public deployment name like qwen3.8. Clients send the deployment name in the model field, never the local id — so re-quantizing later changes nothing on the client side.4
Create an API key
In the Keys tab, click Create key. Copy the
sk-tok-… key now — it is shown only once.Call it from anywhere
Qwen3.8-27B vs Qwen3.8-Max
The same announcement covered a second, far larger model — and that one has now shipped. Qwen3.8-2.4T-A95B, the open-weight model behind the hosted Qwen3.8-Max, is live on Hugging Face and ModelScope with a model card, BF16 safetensors, and an official FP8 variant. The model card settles what community reporting could only estimate: a mixture-of-experts model with 2.4T total parameters and 95B activated, a 262K-token native context extensible to about 1M, and — unlike the 27B — text-only input with thinking always on. Vision input, a non-thinking mode, and 1M default context are features of the hosted Qwen3.8-Max API, not of the open weights. The license is Qwen’s customqwen3.8-max license, not Apache-2.0, so read its terms before commercial use.
That makes it the first open-weight release at Qwen’s Max tier, after Qwen3.7-Max and Qwen3.6-Max-Preview both shipped API-only. It does not make it a homelab model.
The split matters for how you size each one. A dense model loads every parameter for every token, so the
27B behaves predictably: memory tracks its parameter count, and decode speed tracks it too. The 2.4T is the opposite case — only 95B of it activates per token, which keeps compute manageable while the full weight set still has to be resident.
Alibaba also serves the model through its hosted Token Plan, Qoder, and QoderWork products — see Alibaba’s WAIC announcement. Hosted access is not a Tokios deployment; Tokios fronts a model you run.
Can a DGX Spark or Strix Halo run Qwen3.8-Max?
No — not a single machine, and not a small cluster either. In a MoE model every expert weight must stay resident even though only a fraction activates per token — here95B of 2.4T — so memory scales with total parameters rather than active ones. At 4 bits, 2.4T parameters need roughly 1.2 TB for weights alone.
Those are weight-only floors. KV cache, runtime buffers, and system reservation all come on top, so treat the marginal rows as unusable in practice. If you want a Qwen3.8 you can actually serve at home, that is the
27B.
Capabilities
- Native vision. Text, image, and video input — the vision encoder is part of the model architecture, not a bolt-on. For GGUF in llama.cpp, load the separate
mmprojprojector to enable it. - Switchable thinking mode. Toggle reasoning on or off per request with
enable_thinking, and tune the depth withreasoning_effortatxhigh,medium, orlow. - Long context. 262K tokens native, extensible to about 1M.
- Multilingual. Qwen models support a wide range of languages.
- Tunable sampling. Qwen recommends
temperature=1.0, top_p=0.95, top_k=20with thinking on, andtemperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5with it off.
Caveats
- vLLM or SGLang for safetensors. The official BF16 and FP8 releases serve through vLLM, SGLang, TokenSpeed, or Transformers per the model card. For Ollama, llama.cpp, or LM Studio, use the Unsloth GGUF.
- Benchmarks are vendor-reported. Qwen published them on release day. Wait for independent evaluations before treating them as settled.
- Vision support depends on your runtime. Not every backend handles multimodal input yet. Tokios passes image and video content through to the upstream endpoint — it does not add capabilities your runtime lacks. Check supported backends.
- 262K tokens is a ceiling, not a target. Long contexts add KV-cache memory on top of the model weights, and on a 24 GB card that is the headroom left after the 17.11 GB
Q4_K_Mbuild. - Sampling settings differ by mode. Do not carry over settings from another model without testing — Qwen’s recommendations change depending on whether thinking is on or off.
FAQ
How much VRAM does Qwen3.8-27B need?
How much VRAM does Qwen3.8-27B need?
Depends on the format. The Unsloth GGUF is 17.11 GB at
Q4_K_M, which a 24 GB card holds with room for everyday context, and 16 GB cards can run Q3_K_M (13.82 GB) or IQ4_XS (15.71 GB). The official safetensors releases are larger: 30.9 GB at FP8 and 55.6 GB at BF16. Add 0.93 GB for the vision projector when using GGUF.Can Qwen3.8-27B run on a 24 GB GPU?
Can Qwen3.8-27B run on a 24 GB GPU?
Yes — this is the size class the model was built for. An RTX 3090 or 4090 holds the 17.11 GB
Q4_K_M GGUF with roughly 7 GB spare for KV cache. Neither official safetensors release fits a 24 GB card directly (FP8 is 30.9 GB), so the GGUF is the path for a single consumer GPU.When did the Qwen3.8 weights ship?
When did the Qwen3.8 weights ship?
Both are out. The 2.4T flagship (
Qwen/Qwen3.8-2.4T-A95B, plus FP8) shipped first, in the week of 2026-08-10. Qwen3.8-27B followed on 2026-08-14, with BF16 and FP8 repositories live on Hugging Face and ModelScope.What is the difference between Qwen3.8-27B and Qwen3.8-Max?
What is the difference between Qwen3.8-27B and Qwen3.8-Max?
Size, and therefore who can run it — but also modality and licensing, in the direction you might not expect. The
27B is a dense vision-language model (image and video input, switchable thinking) under Apache-2.0, sized for a single desktop GPU or unified-memory machine. The open-weight Max-class model, Qwen3.8-2.4T-A95B, is a mixture-of-experts model at 2.4T total parameters (95B active) that is text-only with thinking always on, released under a custom license, with a 4-bit weight floor around 1.2 TB — cluster hardware, not homelab. Vision input and 1M default context on the big model belong to the hosted Qwen3.8-Max API built on it.Is Qwen3.8 open source?
Is Qwen3.8 open source?
The 27B ships under Apache-2.0 — the same permissive license as Qwen3.6, fine for commercial use. The 2.4T flagship instead uses Qwen’s custom
qwen3.8-max license with scale-tiered terms, so read it before commercial deployment. Same generation, two different licenses — check the model you actually plan to run.Is there an Ollama tag for Qwen3.8 yet?
Is there an Ollama tag for Qwen3.8 yet?
Yes —
ollama run qwen3.8:27b works, and qwen3.8:27b-q4_K_M is also available. Check ollama.com/library/qwen3.8/tags for the full tag list. GGUF files also exist via unsloth/Qwen3.8-27B-GGUF for llama.cpp and LM Studio, and Qwen’s own releases run on vLLM or SGLang.Why does the serving command only show vLLM?
Why does the serving command only show vLLM?
The official BF16 and FP8 safetensors releases require vLLM, SGLang, TokenSpeed, or Transformers per the model card. If you want broader runtime support — Ollama, llama.cpp, LM Studio — use the Unsloth GGUF, which ships the full quantization ladder under Apache-2.0.
How do I use Qwen3.8-27B from another machine?
How do I use Qwen3.8-27B from another machine?
Serve it locally on any OpenAI-compatible runtime, then pair the Tokios connector so the machine holding the GPU needs no inbound ports. You call
https://api.tokios.com with an sk-tok-… key from any OpenAI- or Anthropic-compatible client, sending your deployment name in the model field rather than the upstream model id.Qwen3.6
Run a verified local Qwen model today.
Model fit
Estimate the model and context size your hardware can hold.
Quickstart
Pair the connector and register a local deployment.
Quantization and hardware
Learn what a model’s release formats mean for your hardware.