> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run Llama 4 locally: Scout, Maverick, and their hardware

> Serve Meta's Llama 4 Scout and Maverick on your own hardware. MoE memory math, Ollama tags, vLLM setup, and the 10M-token context reality.

> [!NOTE]
>
> Last verified: 2026-10-14

Llama 4 is Meta's most recent Llama release (April 2025): natively multimodal (text + image input) mixture-of-experts models with very long context. Both open-weight releases — **Scout** (109B total / 17B active) and **Maverick** (400B total / 17B active) — activate only 17B parameters per token, but all expert weights must stay resident in memory. That makes them big-unified-memory or multi-GPU models, not single consumer-GPU pulls: Scout's `Q4`-class download is \~67 GB, Maverick's is \~245 GB. Run the one that fits your hardware, then pair the Tokios connector to reach it as `https://api.tokios.com` from any OpenAI- or Anthropic-compatible client.

> [!NOTE]
>
> Meta's model development has moved to the Muse line — the closed-weight Muse Spark and the open-weight [Muse Glimmer 30B](https://docs.tokios.com/models/muse-glimmer.md). Meta has not formally discontinued Llama and the weights stay downloadable, but no Llama 5 has been announced.

> [!TIP]
>
> Want a Llama that fits a single 24 GB GPU? `ollama run llama3.1:8b` or `ollama run llama3.3:70b` — the 3.x line is still the practical single-GPU choice. Llama 4 is what you step up to on 96 GB+ unified memory or a multi-GPU server.

## Variants & tags

Every value below is from the Ollama library and Meta's Hugging Face model cards. Tags and sizes change as new quantizations ship — confirm against [ollama.com/library/llama4](https://ollama.com/library/llama4/tags) before you standardize on one.

| Model | Ollama tag | Parameters | Experts | Context | Modality | Download (`Q4`-class) |
| - | - | - | - | - | - | - |
| Llama 4 Scout | `llama4:scout` (alias `llama4:16x17b`, also `:latest`) | 109B total / **17B active** | 16 | 10M | Text + image in, text out | \~67 GB |
| Llama 4 Maverick | `llama4:maverick` (alias `llama4:128x17b`) | 400B total / **17B active** | 128 | 1M | Text + image in, text out | \~245 GB |

Sources: [ollama.com/library/llama4/tags](https://ollama.com/library/llama4/tags), [Meta's Llama 4 announcement](https://ai.meta.com/blog/llama-4-multimodal-intelligence/), [meta-llama/Llama-4-Scout-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct).

> [!NOTE]
>
> A third model, **Llama 4 Behemoth** (\~2T parameters), was previewed as the teacher model for the family but has not shipped publicly. Only Scout and Maverick are downloadable.

## Why "17B active" still needs 67 GB

MoE routing activates 17B parameters per token, but the router picks a different subset of experts for every token — so **all** expert weights must stay loaded. The memory math is driven by total parameters, not active ones:

* Scout: 109B × \~0.5 bytes at `Q4`-class ≈ 55 GB of weights, \~67 GB as shipped with overhead
* Maverick: 400B × \~0.5 bytes ≈ 200 GB of weights, \~245 GB as shipped

What the active-parameter count buys you is **speed**, not fit: per-token compute is that of a 17B model, so on hardware with enough memory, Llama 4 generates faster than a dense model of similar total size. See [memory and offloading](https://docs.tokios.com/concepts/memory-and-offloading.md) for how runtimes handle models near the memory ceiling.

## Hardware fit

Comfort at roughly `Q4` with a modest context length. Treat every entry as an estimate — the ground truth is the actual download size plus KV cache for your context, and the long-context headline numbers (10M for Scout) multiply KV-cache memory far past what most machines hold. See [what size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math.

| Target | Scout (\~67 GB) | Maverick (\~245 GB) |
| - | - | - |
| RTX 3090 / 4090 / 5090 (24–32 GB) | Won't fit in VRAM | Won't fit |
| RTX 6000 Ada / A6000 (48 GB) | Won't fit in VRAM | Won't fit |
| Apple Silicon 96–128 GB unified | Comfortable | Won't fit |
| DGX Spark / Strix Halo (128 GB unified) | Comfortable | Won't fit |
| Multi-GPU server (4× 80 GB class) | Comfortable | Tight — weights alone need \~245 GB |
| Multi-GPU server (8× 80 GB class) | Comfortable | Comfortable |

> [card] [Serve it from one endpoint once it runs](https://docs.tokios.com/setup/register-model.md)
>
> Register your Llama 4 deployment with Tokios and call it from any machine with an `sk-tok-…` key — no inbound ports on the box holding the GPUs.

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run llama4:scout
> ```
>
> Leave Ollama listening on the default `http://127.0.0.1:11434`. Raise `num_ctx` past Ollama's small default to use the long context — but size it against your free memory, not the 10M headline.

> [tab] vLLM (multi-GPU)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve meta-llama/Llama-4-Scout-17B-16E-Instruct \
>   --tensor-parallel-size 4 \
>   --max-model-len 131072
> ```
>
> For Maverick, use [`meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8) and shard across enough GPUs to hold \~245 GB. Confirm the recommended flags for your vLLM version against the model card before serving.

## Capabilities

* **Multimodal input:** Scout and Maverick accept text and images natively; output is text-only.
* **Context:** Scout advertises a 10M-token window and Maverick 1M. Serving anywhere near those lengths is a KV-cache memory problem of its own — see [KV cache and context](https://docs.tokios.com/concepts/kv-cache-and-context.md) before you plan around them.
* **Languages:** Meta's model cards list 12 supported languages, including English, German, French, Spanish, Portuguese, Hindi, and Vietnamese.
* **Reasoning:** General instruction-tuned models — no separate thinking mode. For step-by-step reasoning output, see [DeepSeek](https://docs.tokios.com/models/deepseek-r1.md) or [gpt-oss](https://docs.tokios.com/models/gpt-oss).

## Connect it with Tokios

**1. Keep your local runtime running**

Start Llama 4 in Ollama, vLLM, or another OpenAI-compatible runtime and leave it bound to loopback.

**2. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Run the installer on the machine that runs the model:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the upstream model id (for example `llama4:scout`) under a public deployment name like `llama4-tunnel`. Clients send the deployment name in the `model` field, never the local id.

**4. Create an API key and call it**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**, then:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama4-tunnel", "stream": true, "messages": [{"role": "user", "content": "Hello!"}]}'
```

## Caveats

* **No consumer-GPU path.** Unlike the 3.x line, neither Llama 4 model fits a 24–48 GB card in VRAM. llama.cpp-based runtimes can offload experts to system RAM, but throughput drops sharply once weights leave the GPU — budget for unified memory or a server before you standardize on Llama 4.
* **License:** Llama 4 uses Meta's **Llama 4 Community License** — not Apache or MIT. It carries an acceptable-use policy and a separate Meta license requirement for products above 700 million monthly active users. Check it before commercial use.
* **Long context is a memory budget, not a free feature.** A 10M-token window at full occupancy needs KV cache far beyond the weight footprint. Size `num_ctx` / `--max-model-len` to what your memory actually holds.
* Tokios passes image content and tool-call fields through to your upstream runtime — it doesn't add capabilities the runtime or quantization doesn't support.

## FAQ

> [accordion] How much VRAM does Llama 4 need?
>
> Scout's `Q4`-class download is \~67 GB (109B total parameters × \~0.5 bytes at Q4 ≈ 55 GB of weights, plus overhead). Maverick's is \~245 GB (400B × \~0.5 bytes ≈ 200 GB of weights, plus overhead). Neither fits in a single consumer GPU's VRAM — Scout needs 96–128 GB of unified memory (Apple Silicon, DGX Spark) or a multi-GPU server; Maverick needs a multi-GPU server with 8× 80 GB class GPUs.

> [accordion] Can Llama 4 run on a consumer GPU?
>
> Not in VRAM — neither model fits a 24–48 GB card. llama.cpp-based runtimes can offload experts to system RAM, but throughput drops sharply once weights leave the GPU. For a single consumer GPU, use the Llama 3.x line instead (`llama3.1:8b` or `llama3.3:70b`), or Meta's [Muse Glimmer 30B](https://docs.tokios.com/models/muse-glimmer.md) which fits one 24 GB card at 4-bit.

> [accordion] Is Llama 4 open source?
>
> Llama 4 uses Meta's **Llama 4 Community License** — not Apache or MIT. It carries an acceptable-use policy and requires a separate Meta license for products above 700 million monthly active users. Check the license terms before commercial use.

> [accordion] What is the difference between Scout and Maverick?
>
> Both activate 17B parameters per token, but Scout has 109B total parameters across 16 experts with a 10M-token context (\~67 GB at Q4), while Maverick has 400B total parameters across 128 experts with a 1M-token context (\~245 GB at Q4). Scout fits 96–128 GB unified memory; Maverick needs a multi-GPU server. Choose Scout if it fits your hardware — Maverick only if you need the larger expert pool and can host it.

> [accordion] Will there be a Llama 5?
>
> Meta's model development has moved to the Muse line — the closed-weight Muse Spark and the open-weight [Muse Glimmer 30B](https://docs.tokios.com/models/muse-glimmer.md). Meta has not formally discontinued Llama and the weights stay downloadable, but no Llama 5 has been announced.

> [card] [Muse Glimmer](https://docs.tokios.com/models/muse-glimmer.md)
>
> Meta's current open-weight line — Apache 2.0, agentic, and fits one 24 GB card.

> [card] [What fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> Size the model to your VRAM or unified memory before downloading.

> [card] [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Which weight formats run on your GPU, Mac, or unified-memory box.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.