> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run Mistral locally and use it from anywhere

> Serve Mistral's open-weight family — chat, agentic coding, reasoning, and edge tiers — with Ollama, LM Studio, vLLM, or SGLang, then reach it via Tokios.

> [!NOTE]
>
> Last verified: 2026-10-14

Mistral AI publishes several open-weight families under Apache-2.0, each aimed at a different job: **Mistral Small** for general chat, **Devstral** for agentic coding, **Magistral** for reasoning, and **Ministral** for edge deployment. All of them run locally. Pick the tier that matches your task and hardware, then pair the Tokios connector to call it from any machine, teammate, or hosted agent through `https://api.tokios.com` — no inbound ports, no VPN.

## Run it locally with Ollama

Each family has its own Ollama library entry. Pull the one that matches your task:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run mistral-small3.2:24b   # general chat
ollama run devstral-small-2:24b   # agentic coding
ollama run magistral:24b          # reasoning
ollama run ministral-3:3b         # edge / low-VRAM
```

See the [Ollama library](https://ollama.com/search?q=mistral) for the full tag list — this page covers connecting a model, not every runtime detail.

> [!NOTE]
>
> Ollama's default context window is small (2048 tokens) — far below any of these models' native context. Raise `num_ctx` in your Modelfile or client request to use more of it.

## Variants & tags

Every value below is from the model card or the Ollama library. Tags and download sizes change as new quantizations ship — confirm against the linked library page before you standardize on one.

| Role | Model | Ollama tag | Context window | License |
| - | - | - | - | - |
| General chat | Mistral Small 3.2 (2506) | `mistral-small3.2:24b` | 131,072 (128K) | Apache-2.0 |
| General chat (larger, dense) | Mistral Large (2411) | `mistral-large:123b` | 128K | Apache-2.0 |
| Agentic coding | Devstral Small 2 (2512) | `devstral-small-2:24b` | 393,216 architecture max; 256K recommended | Apache-2.0 |
| Agentic coding (previous gen) | Devstral Small 1.0 (2505) | `devstral:24b` | 131,072 (128K) | Apache-2.0 |
| Reasoning | Magistral Small (2509) | `magistral:24b` | 128K (best quality ≤40K) | Apache-2.0 |
| Edge / low-VRAM | Ministral 3 (2512) | `ministral-3:3b`, `ministral-3:8b`, `ministral-3:14b` | 262,144 (256K) | Apache-2.0 |
| Classic 7B | Mistral 0.3 | `mistral:7b` | 32K | Apache-2.0 |

Sources: [ollama.com/library/mistral-small3.2](https://ollama.com/library/mistral-small3.2), [ollama.com/library/mistral-large](https://ollama.com/library/mistral-large), [ollama.com/library/devstral-small-2](https://ollama.com/library/devstral-small-2), [ollama.com/library/devstral](https://ollama.com/library/devstral), [ollama.com/library/magistral](https://ollama.com/library/magistral), [ollama.com/library/ministral-3](https://ollama.com/library/ministral-3), [ollama.com/library/mistral](https://ollama.com/library/mistral), and the corresponding Hugging Face model cards ([Mistral-Small-3.2-24B-Instruct-2506](https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506), [Devstral-Small-2-24B-Instruct-2512](https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512), [Magistral-Small-2509](https://huggingface.co/mistralai/Magistral-Small-2509), [Ministral-3-3B-Instruct-2512](https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512)).

> [!TIP]
>
> Mistral also publishes a newer unified flagship, **Mistral Small 4 (2603)**, which merges the Instruct, Reasoning, and Devstral lines into one 119B mixture-of-experts model (128 experts, 4 active; 256K context; toggle reasoning per-request with `reasoning_effort`). As of this writing it isn't in the Ollama library — run it via `vLLM`/`SGLang` from [mistralai/Mistral-Small-4-119B-2603](https://huggingface.co/mistralai/Mistral-Small-4-119B-2603) or a community GGUF, and size your hardware off all 119B parameters even though only 4 experts activate per token.

> [!WARNING]
>
> Devstral Small 2's config lists a 393,216-token (384K) architecture maximum, but Mistral's own model card recommends operating at 256K — treat 256K as the practical ceiling, not the raw config value.

## Quantization & formats

Ollama's default tags are `GGUF` `Q4_K_M`, with `q4`, `q8`, and `fp16` variants published alongside for most of these libraries. Which format you can run depends on your hardware — see [Quantization formats and the hardware they need](https://docs.tokios.com/models/quantization-and-hardware.md) for the full matrix, and [what quantization is](https://docs.tokios.com/concepts/quantization.md) for how to read the `Q4_K_M`-style names.

## Hardware fit

Comfort at roughly `Q4` for a modest context length. Treat every entry as an estimate — the ground truth is the actual `GGUF` size plus the KV cache for your context. See [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math.

| Target | Ministral 3 (`3b`–`14b`) | Mistral Small 3.2 / Devstral / Magistral (`24b`) | Mistral Large (`123b`) |
| - | - | - | - |
| Laptop / CPU only | Comfortable at `3b` | Won't fit at usable speed | Won't fit |
| RTX 3090 / 4090 (24 GB) | Comfortable | Comfortable at `Q4`, tight at `Q8` | Won't fit |
| RTX 5090 (32 GB) | Comfortable | Comfortable, more context headroom | Won't fit |
| RTX 6000 Ada / A6000 (48 GB) | Comfortable | Comfortable, long context | Tight at `Q4`, short context |
| Apple Silicon (36 GB+ unified) | Comfortable | Comfortable | Needs 96 GB+ unified memory |
| AMD Strix Halo (ROCm/Vulkan, ≤96 GB) | Comfortable | Comfortable | Comfortable at `Q4` |

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run mistral-small3.2:24b
> ```
>
> Leave Ollama listening on the default `http://127.0.0.1:11434`. Raise `num_ctx` to use more context; swap the tag for `devstral-small-2:24b`, `magistral:24b`, or `ministral-3:3b` depending on the job.

> [tab] LM Studio (GUI)
>
> Search for **Mistral**, **Devstral**, **Magistral**, or **Ministral** in LM Studio's model catalog, download a `GGUF` quant that fits your VRAM (start with `Q4_K_M`), then start LM Studio's local server. It exposes an OpenAI-compatible API on `http://127.0.0.1:1234/v1`.

> [tab] vLLM (advanced)
>
> Mistral's chat template and tool-calling need explicit flags on vLLM. For Devstral Small 2, for example:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve mistralai/Devstral-Small-2-24B-Instruct-2512 \
>   --tokenizer_mode mistral \
>   --config_format mistral \
>   --load_format mistral \
>   --tool-call-parser mistral \
>   --enable-auto-tool-choice
> ```
>
> Magistral and Mistral Small 4 additionally need `--reasoning-parser mistral` to separate reasoning content from the final answer. Add `--tensor-parallel-size <N>` to shard across GPUs. Flags are from each model's card — confirm against the specific model you run before deploying.

> [tab] SGLang (advanced)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> python -m sglang.launch_server \
>   --model-path mistralai/Devstral-Small-2-24B-Instruct-2512 \
>   --tool-call-parser mistral
> ```
>
> Add `--tp <N>` for tensor parallelism. Flags are from the model card's SGLang section.

## Capabilities

* **Native tool-calling:** Mistral's whole current lineup — Small 3.2, Devstral Small 2, Magistral, Ministral 3 — supports function calling with its own call format. Ollama applies the template for you; on vLLM/SGLang set `--tool-call-parser mistral` (add `--enable-auto-tool-choice` on vLLM) or tool calls won't parse.
* **Reasoning:** Magistral Small (2509) wraps its reasoning trace in `[THINK]…[/THINK]` tokens and needs `--reasoning-parser mistral` on vLLM/SGLang to separate it from the final answer. Mistral's own guidance notes quality can degrade past 40K tokens even though the model accepts up to 128K.
* **Agentic coding:** Devstral Small 2 is purpose-built for coding agents — repository exploration, multi-file edits, and tool use — and is validated against agent harnesses like OpenHands and Mistral's own `mistral-vibe` CLI.
* **Edge deployment:** Ministral 3 targets low-VRAM and embedded use; the `3B` tier can run in roughly 8 GB of VRAM at FP8 per Mistral's own model card, with `8B` and `14B` tiers available if you have more headroom.
* **Vision:** Mistral Small 3.2, Devstral Small 2, and Magistral Small all ship with a vision encoder and accept image input alongside text.

## Connect it with Tokios

Once your chosen Mistral model is running locally, pair the Tokios connector so you can reach it as `https://api.tokios.com` from Claude Code, Cline, or any other compatible client — instead of only from `localhost:11434`.

**1. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Install and run the connector on the machine running Ollama:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Point the connector at Ollama's /v1 API**

Ollama exposes an OpenAI-compatible surface at `/v1`. For a single-model setup, the console's Setup wizard fills this in automatically as `http://127.0.0.1:11434/v1`. For multiple models — say, `mistral-small3.2` for chat and `devstral-small-2` for coding, both tunneled at once — set it explicitly in `tokios-connector.json`. See [Connector config](https://docs.tokios.com/reference/connector-config.md).

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the model and choose a public deployment name, for example `mistral-tunnel`. This name — not Ollama's local model id — is what clients send in the `model` field. Tokios gives whatever you're already running locally a stable, authenticated endpoint; it doesn't pick, host, or fine-tune the model for you.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model mistral-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`mistral-tunnel` above), never Ollama's local model id. Claude Code's model picker only lists `claude*` ids, so pass `--model mistral-tunnel` explicitly.

## Caveats

* **Pick the tier for the job, not just the family name.** "Mistral" now spans four purpose-built lines — Small (chat), Devstral (coding), Magistral (reasoning), Ministral (edge) — with different context windows and tool-calling behavior. Match the tag to the task; see [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md).
* **Tool-call behavior depends on the runtime applying Mistral's parser.** Ollama handles this for you; on vLLM/SGLang, set `--tool-call-parser mistral` (and `--enable-auto-tool-choice` on vLLM) or tool calls won't parse correctly.
* **Devstral Small 2's context is runtime-dependent.** The architecture supports up to 393,216 tokens, but Mistral recommends 256K as the practical operating ceiling — don't assume the full config value is safe for quality.
* **Magistral's reasoning quality can drop off well before its stated context limit.** Mistral's own card recommends keeping reasoning-heavy prompts under roughly 40K tokens even though the model accepts up to 128K.
* **Mistral Small 4 (2603)** is the newest unified flagship but isn't in the Ollama library as of this writing — confirm current availability before assuming an `ollama run` shortcut exists for it.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> Match a Mistral tier to your available VRAM before you pull it.

> [card] [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Which formats — `GGUF`, `FP8`, quantized — run on your hardware.

> [card] [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md)
>
> Compare Mistral's chat, coding, reasoning, and edge tiers against other local models.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.