> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# AMD Strix Halo model compatibility: what fits on 128GB unified

> Which models run on AMD Strix Halo (Ryzen AI Max+ 395/495) with 128GB or 192GB unified memory? Vulkan vs ROCm backend choice, best models, and system vendors.

AMD Strix Halo (Ryzen AI Max+ 395/495) offers up to 192 GB of unified memory in a compact mini-PC form factor. The 128GB configuration is the sweet spot for 70B models at Q4, while the 192GB developer platform (Ryzen AI Max+ Pro 495) unlocks 120B+ MoE models. This guide covers what fits, what doesn't, and the critical Vulkan vs ROCm backend decision.

> [!NOTE]
>
> Last verified: 2026-10-21. ROCm support for Strix Halo is evolving rapidly. Check the [ROCm documentation](https://rocm.docs.amd.com/) and community benchmarks for the latest compatibility updates.

## Hardware overview

**AMD Strix Halo** is a system-on-chip (SoC) that combines AMD's Zen 5 CPU cores with RDNA 3.5 integrated graphics and up to 192 GB of unified LPDDR5X memory.

| Spec | Ryzen AI Max+ 395 | Ryzen AI Max+ Pro 495 |
| - | - | - |
| CPU | 16-core Zen 5 | 16-core Zen 5 |
| GPU | RDNA 3.5 (40 CU) | RDNA 3.5 (40 CU) |
| Memory | Up to 128 GB LPDDR5X-8000 | Up to 192 GB LPDDR5X-8000 |
| GPU-accessible memory | 128 GB (all of it) | 160 GB (of 192 GB total) |
| Memory bandwidth | 256 GB/s | 256 GB/s |
| TDP | 15-54W | 15-54W |
| Form factor | Mini-PC, laptop, desktop | Mini-PC, desktop (developer platform) |
| Price range | \$1,500-3,000 | \$3,000-5,000 |

> [!TIP]
>
> **The 192GB Pro 495 has 160 GB GPU-accessible.** The remaining 32 GB is reserved for the system. This is the key advantage over standard 128GB systems — you get 25% more memory for model weights.

## What fits (and what doesn't)

### 128 GB configuration (Ryzen AI Max+ 395)

**Fits comfortably:**

* Qwen3.5-122B-A10B at Q4\_K\_M (\~70 GB, 10-18 tok/s)
* GPT-OSS 120B at Q4 (\~59 GB, 35-51 tok/s)
* GLM-4.5-Air at Q6\_K\_XL (\~60 GB, 10-15 tok/s)
* Qwen3.6-35B-A3B at Q4 (\~20 GB, 54+ tok/s)
* Qwen3 70B at Q4 (\~40 GB, 12-20 tok/s)

**Does NOT fit:**

* DeepSeek V4.1 Flash (763B) — smallest GGUF is \~233 GB, nearly 2x the 128GB
* DeepSeek V4-Flash-0731 (284B) — needs \~155 GB, exceeds 128GB
* GLM-5.2 (753B) — needs \~377 GB at Q4, far beyond 128GB
* Qwen3.5-397B at Q4 (\~224 GB) — exceeds 128GB

**Best use case:** 70B dense models at Q4 or 120B MoE models at Q4 with room for KV cache.

### 192 GB configuration (Ryzen AI Max+ Pro 495)

**Fits comfortably:**

* Everything that fits on 128GB, plus:
* Qwen3 235B at Q3\_K\_M (\~130 GB, 8-12 tok/s)
* DeepSeek V4-Flash-0731 (284B) at Q3 (\~130 GB, 10-15 tok/s) — tight fit
* Multiple 70B models simultaneously

**Does NOT fit:**

* DeepSeek V4.1 Flash (763B) — still too large
* GLM-5.2 (753B) — still too large
* Qwen3.5-397B at Q4 (\~224 GB) — exceeds 192GB

**Best use case:** 120B+ MoE models at higher quantization (Q6-Q8) or 200B+ models at Q3.

> [!WARNING]
>
> **DeepSeek V4.1 Flash does not fit on any Strix Halo configuration.** The model's 763B total parameters require at least \~233 GB at the smallest usable quantization. For V4.1 Flash, you need 2x DGX Spark (256GB), Mac Studio M5 Ultra 512GB, or 4x RTX PRO 6000 Max-Q (384GB).

## Backend decision: Vulkan vs ROCm

The most important decision for Strix Halo users is which inference backend to use. The choice between Vulkan and ROCm dramatically affects performance, stability, and ease of use.

### Vulkan (recommended for most users)

**Best for:** Stability, ease of use, broad model support

* **Pros:** Works out of the box, no special drivers needed, broad model support via llama.cpp
* **Cons:** 10-20% slower than ROCm (when ROCm works), no native AMD tensor core acceleration
* **Performance:** \~10-18 tok/s on 120B MoE models
* **Setup:** Use llama.cpp with Vulkan backend

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
# Install llama.cpp with Vulkan support
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make GGML_VULKAN=1

# Run a model
./llama-cli -m qwen3.5-122b-a10b-q4_k_m.gguf \
  -p "Explain quantum computing" \
  -n 128 \
  --gpu-layers 999
```

**When to choose Vulkan:**

* You want a stable, supported path
* You're running GGUF quantization models
* You don't want to deal with ROCm driver issues
* You're okay with 10-20% lower performance

### ROCm 7.2 (for advanced users)

**Best for:** Maximum performance, native AMD acceleration

* **Pros:** 20-30% faster than Vulkan (when it works), native RDNA 3.5 acceleration
* **Cons:** Requires `HSA_OVERRIDE_GFX_VERSION=11.5.1` environment variable, limited model support, driver issues
* **Performance:** \~15-25 tok/s on 120B MoE models (when working)
* **Setup:** Use vLLM or llama.cpp with ROCm backend

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
# Install ROCm 7.2
# (Follow AMD's official ROCm installation guide for your Linux distro)

# Set the override for Strix Halo
export HSA_OVERRIDE_GFX_VERSION=11.5.1

# Run with vLLM
vllm serve Qwen/Qwen3.5-122B-A10B \
  --quantization awq \
  --max-model-len 8192
```

**When to choose ROCm:**

* You need maximum performance and accept the tinkering
* You're running AWQ or FP8 quantization (better supported in ROCm)
* You're comfortable debugging driver issues
* You're on Linux (ROCm has limited Windows support)

> [!WARNING]
>
> **ROCm on Strix Halo is not officially supported.** AMD's ROCm targets datacenter and professional GPUs, not consumer APUs. The `HSA_OVERRIDE_GFX_VERSION=11.5.1` workaround makes it work, but you may encounter:
>
> * Crashes on certain model architectures
> * Slower performance than expected
> * Driver incompatibilities after system updates
>
> **Recommendation:** Start with Vulkan. Only switch to ROCm if you need the extra performance and accept the unsupported path.

## Model recommendations by use case

### Individual developer (128 GB config)

| Use case | Recommended model | Quantization | Memory | tok/s | Backend |
| - | - | - | - | - | - |
| Coding assistant | Qwen3-Coder-30B | Q4\_K\_M | \~18 GB | 40-50 | Vulkan |
| General chat | Qwen3.6-35B-A3B | Q4 | \~20 GB | 54+ | Vulkan |
| Reasoning | GLM-4.5-Air (106B MoE) | Q6\_K\_XL | \~60 GB | 10-15 | Vulkan |
| Large MoE | Qwen3.5-122B-A10B | Q4\_K\_M | \~70 GB | 10-18 | Vulkan |
| Fast iteration | GPT-OSS 120B | Q4 | \~59 GB | 35-51 | Vulkan |

### Research / power user (192 GB config)

| Use case | Recommended model | Quantization | Memory | tok/s | Backend |
| - | - | - | - | - | - |
| Large dense | Qwen3 235B | Q3\_K\_M | \~130 GB | 8-12 | Vulkan |
| Frontier (slow) | DeepSeek V4-Flash-0731 | Q3 | \~130 GB | 10-15 | Vulkan |
| High-quality MoE | Qwen3.5-122B-A10B | Q6\_K | \~90 GB | 12-18 | Vulkan |
| Multi-model | Multiple 70B models | Q4 | \~40 GB each | 15-20 each | Vulkan |

### Maximum performance (ROCm path)

| Use case | Recommended model | Quantization | Memory | tok/s | Backend |
| - | - | - | - | - | - |
| Fast MoE | Qwen3.5-122B-A10B | AWQ | \~61 GB | 18-25 | ROCm |
| Fast dense | Qwen3 70B | FP8 | \~70 GB | 20-30 | ROCm |
| Fast chat | Qwen3.6-35B-A3B | AWQ | \~18 GB | 60-70 | ROCm |

## System vendors

Strix Halo is available in mini-PCs, laptops, and desktops from several vendors.

### Mini-PCs

| Vendor | Model | Memory | Price | Notes |
| - | - | - | - | - |
| **GMKtec** | EVO-X2 | 128 GB | \~\$2,500 | Compact, good cooling |
| **Minisforum** | MS-S1 MAX | 128 GB | \~\$2,800 | Popular choice, active community |
| **Minisforum** | MS-S1 MAX-P495 | 192 GB | \~\$4,300 | Developer platform, 160 GB GPU-accessible |
| **Beelink** | GTR9 Pro | 128 GB | \~\$2,600 | Good value |

### Desktops

| Vendor | Model | Memory | Price | Notes |
| - | - | - | - | - |
| **Framework** | Desktop (AMD) | 128 GB | \~\$2,200 + CPU | Modular, upgradeable (except memory) |
| **Various** | Custom builds | 128-192 GB | varies | Check compatibility before buying |

### Laptops

| Vendor | Model | Memory | Price | Notes |
| - | - | - | - | - |
| **Various** | Ryzen AI Max+ laptops | 128 GB | \~\$2,000-3,500 | Limited selection, check thermals |

> [!TIP]
>
> **Memory is soldered — buy the configuration you need at purchase.** There is no SO-DIMM slot or expansion option. The 128GB vs 192GB decision is permanent.

## Performance benchmarks

Community benchmarks show Strix Halo delivers competitive performance for 70B-120B models, but falls behind DGX Spark and Mac Studio on larger models.

### Single-user inference (Vulkan backend)

| Model | Quantization | Memory | tok/s | Notes |
| - | - | - | - | - |
| Qwen3.6-35B-A3B | Q4 | \~20 GB | 54+ | 3B active, very fast |
| GPT-OSS 120B | Q4 | \~59 GB | 35-51 | Surprisingly fast for 120B |
| Qwen3.5-122B-A10B | Q4\_K\_M | \~70 GB | 10-18 | 10B active, good quality |
| GLM-4.5-Air | Q6\_K\_XL | \~60 GB | 10-15 | 12B active, high quality |
| Qwen3 70B | Q4 | \~40 GB | 12-20 | Dense model, slower |

### Comparison to other platforms

| Platform | Model | tok/s | Price | \$/tok/s |
| - | - | - | - | - |
| Strix Halo 128GB | Qwen3.5-122B Q4 | \~14 | \~\$2,500 | \$179 |
| DGX Spark 128GB | Qwen3.5-122B NVFP4 | \~30 | \~\$3,000 | \$100 |
| Mac Studio M5 Ultra 256GB | Qwen3.5-122B Q8 | \~55 | \~\$8,500 | \$155 |
| RTX 5090 32GB | Qwen3 70B Q4 | \~23 | \~\$1,999 | \$87 |
| RTX PRO 6000 96GB | Qwen3.5-122B Q4 | \~35 | \~\$16,000 | \$457 |

**Strix Halo is 2x slower than DGX Spark** on the same model, but costs \~\$500 less. It's a good value for 70B-120B models, but not competitive for larger models or high-throughput serving.

## Setup guide

### Linux (recommended)

1. **Install Ubuntu 22.04 or 24.04**
   * Use a USB installer (Rufus or balenaEtcher)
   * Select "Install alongside Windows" for dual-boot

2. **Install prerequisites**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   sudo apt update
   sudo apt install build-essential git cmake wget
   ```

3. **Build llama.cpp with Vulkan support**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   git clone https://github.com/ggerganov/llama.cpp
   cd llama.cpp
   make GGML_VULKAN=1
   ```

4. **Download a model**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   # Example: Qwen3.5-122B-A10B at Q4_K_M
   wget https://huggingface.co/Qwen/Qwen3.5-122B-A10B-GGUF/resolve/main/qwen3.5-122b-a10b-q4_k_m.gguf
   ```

5. **Run inference**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   ./llama-cli -m qwen3.5-122b-a10b-q4_k_m.gguf \
     -p "Explain quantum computing in simple terms" \
     -n 128 \
     --gpu-layers 999
   ```

### Windows

1. **Install Ollama**
   * Download from [ollama.com](https://ollama.com)
   * Ollama automatically detects Vulkan-capable GPUs

2. **Pull a model**
   ```powershell theme={"theme":{"light":"github-light","dark":"css-variables"}}
   ollama pull qwen3.5:122b-a10b-q4
   ```

3. **Run inference**
   ```powershell theme={"theme":{"light":"github-light","dark":"css-variables"}}
   ollama run qwen3.5:122b-a10b-q4
   ```

### ROCm setup (advanced)

1. **Install ROCm 7.2**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   # Follow AMD's official guide: https://rocm.docs.amd.com/
   wget https://repo.radeon.com/rocm/rocm-7.2.0/rocm-7.2.0.tar.gz
   tar -xzf rocm-7.2.0.tar.gz
   cd rocm-7.2.0
   sudo ./install.sh
   ```

2. **Set environment variable**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   echo 'export HSA_OVERRIDE_GFX_VERSION=11.5.1' >> ~/.bashrc
   source ~/.bashrc
   ```

3. **Build llama.cpp with ROCm support**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   git clone https://github.com/ggerganov/llama.cpp
   cd llama.cpp
   make GGML_HIP=1
   ```

4. **Run inference**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   ./llama-cli -m qwen3.5-122b-a10b-q4_k_m.gguf \
     -p "Explain quantum computing" \
     -n 128 \
     --gpu-layers 999
   ```

## Common issues and troubleshooting

### Issue: Vulkan not detected

**Cause:** Missing Vulkan drivers or llama.cpp not built with Vulkan support.

**Fix:**

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
# Install Vulkan drivers
sudo apt install mesa-vulkan-drivers

# Rebuild llama.cpp with Vulkan
cd llama.cpp
make clean
make GGML_VULKAN=1

# Verify Vulkan is working
./llama-cli --list-devices
# Should show your AMD GPU
```

### Issue: ROCm crashes on certain models

**Cause:** ROCm does not fully support RDNA 3.5 on Strix Halo.

**Fix:**

* Switch to Vulkan backend (recommended)
* Try a different model architecture (some work better than others)
* Update to latest ROCm version
* Check community forums for model-specific workarounds

### Issue: Slow performance on 120B models

**Cause:** Memory bandwidth bottleneck (256 GB/s is lower than DGX Spark's 273 GB/s or Mac Studio's 1,200 GB/s).

**Fix:**

* Use smaller models (70B instead of 120B)
* Use MoE models with fewer active parameters (Qwen3.6-35B-A3B has 3B active)
* Accept the performance — Strix Halo is not designed for maximum throughput
* Upgrade to DGX Spark or Mac Studio for 2-4x faster inference

### Issue: Out of memory on large models

**Cause:** Model + KV cache exceeds available memory.

**Fix:**

* Use lower quantization (Q4 instead of Q8)
* Reduce `--ctx-size` to limit KV cache
* Use a smaller model
* Upgrade to 192GB Pro 495 configuration

## Is Strix Halo right for you?

**Choose Strix Halo if:**

* You need 128GB unified memory in a compact form factor
* You want a quiet, low-power system (15-54W TDP)
* You're running 70B-120B models at Q4-Q6
* You prefer Linux or are comfortable with Vulkan backend
* Budget is \$2,500-4,300

**Avoid Strix Halo if:**

* You need to run frontier models (V4.1 Flash, GLM-5.2) — they don't fit
* You need maximum throughput — DGX Spark is 2x faster
* You want plug-and-play simplicity — Mac Studio is easier
* You need CUDA support — Strix Halo is AMD-only
* You're on a tight budget — used RTX 3090 (24GB) is \$700-900

## Next steps

> [card] [Hardware reference](https://docs.tokios.com/guides/hardware-reference.md)
>
> Compare Strix Halo against DGX Spark, Mac Studio, RTX PRO, and budget hacker tier.

> [card] [Mac Studio M5 Ultra buying guide](https://docs.tokios.com/guides/mac-studio-m5-ultra.md)
>
> Apple alternative with 256-512GB unified memory and 1,200 GB/s bandwidth.

> [card] [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md)
>
> See which models fit on DGX Spark's 128GB unified memory (2x faster than Strix Halo).

> [card] [Model fit](https://docs.tokios.com/guides/model-fit.md)
>
> Calculate which model sizes and quantizations fit your Strix Halo's unified memory.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.