> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run Qwen3-VL locally and use it from anywhere

> Serve Qwen3-VL — Qwen's dedicated vision-language family — with Ollama, LM Studio, vLLM, or SGLang, and reach it through a Tokios endpoint.

> [!NOTE]
>
> Last verified: 2026-10-14

Qwen3-VL is Qwen's dedicated vision-language family — it reads text, images, **and** video, supports long context on the local-friendly sizes, and ships in both dense and mixture-of-experts (MoE) sizes with separate Instruct and reasoning-enhanced Thinking editions. It supersedes the older Qwen2.5-VL; if you're picking a vision model to run today, start here — it is the line with verified Ollama tags, while the newer [Qwen3.8-27B](https://docs.tokios.com/models/qwen3.8.md) also takes image and video input but runs only through llama.cpp, LM Studio, or a server runtime for now. Run whichever size fits your hardware locally, then pair the Tokios connector to call it from any machine, teammate, or hosted agent through `https://api.tokios.com` — no inbound ports, no VPN.

## Run it locally with Ollama

Qwen3-VL is in the Ollama library under the slug `qwen3-vl`. Pull and run the default tag (the `8B` dense Instruct model):

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run qwen3-vl
```

For other sizes, add a tag — for example `ollama run qwen3-vl:30b-a3b` for the MoE model, or `ollama run qwen3-vl:8b-thinking` for the reasoning edition. See the [Ollama library page](https://ollama.com/library/qwen3-vl/tags) for the full tag list — this page covers connecting it, not every runtime detail.

> [!NOTE]
>
> Ollama's default context window is small (2048 tokens) — far below Qwen3-VL's native 256K. Raise `num_ctx` in your Modelfile or client request to use more of its context.

## Variants & tags

Every value below is from the Hugging Face model cards and the Ollama library. Tags and download sizes change as new quantizations ship — confirm against [ollama.com/library/qwen3-vl/tags](https://ollama.com/library/qwen3-vl/tags) before you standardize on one.

| Property | `8B` (dense) | `30B-A3B` (MoE) |
| - | - | - |
| Ollama tag | `qwen3-vl:8b` (alias `:latest`) | `qwen3-vl:30b-a3b` |
| Download size (default quant) | \~6.1 GB | \~20 GB |
| Download size (`bf16`) | \~18 GB | \~62 GB |
| Parameters | 8B dense | 30B total / **3B active** (128 experts, 8 routed) |
| Context window | 262,144 native | 262,144 native |
| Modality | Text + image + video | Text + image + video |
| Editions | Instruct and Thinking | Instruct and Thinking |
| License | Apache-2.0 | Apache-2.0 |
| Hugging Face | [`Qwen/Qwen3-VL-8B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) | [`Qwen/Qwen3-VL-30B-A3B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct) |

Sources: [ollama.com/library/qwen3-vl/tags](https://ollama.com/library/qwen3-vl/tags), [Qwen/Qwen3-VL-8B-Instruct config](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct/raw/main/config.json), and [Qwen/Qwen3-VL-30B-A3B-Instruct config](https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct/raw/main/config.json). The Ollama library also lists `2B`, `4B`, `32B`, and `235B-A22B` sizes; this table focuses on the verified local-friendly sizes.

> [!TIP]
>
> The `30B-A3B` model is mixture-of-experts — only 3B parameters are active per token, so it decodes faster than a dense model of the same total size. But all parameters still have to fit in memory. Don't size your GPU off the active-parameter count alone. See [how inference works](https://docs.tokios.com/concepts/how-inference-works.md) for why.

## Quantization & formats

Ollama's tags cover a default quant plus `-q8`, and separate `-bf16` tags for every size, each available in both `-instruct` and `-thinking` editions. Which format you can run depends on your hardware — see [Quantization formats and the hardware they need](https://docs.tokios.com/models/quantization-and-hardware.md) for the full matrix, and [what quantization is](https://docs.tokios.com/concepts/quantization.md) for how to read quant names like `Q4_K_M`.

## Hardware fit

Comfort at roughly `Q4` for a modest context length. Treat every entry as an estimate — the ground truth is the actual `GGUF` size plus the KV cache for your context. See [what size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math.

| Target | `8B` (\~6 GB) | `30B-A3B` MoE (\~20 GB) |
| - | - | - |
| RTX 3090 / 4090 (24 GB) | Comfortable, long context | Comfortable |
| RTX 5090 (32 GB, Blackwell) | Comfortable, long context | Comfortable |
| RTX 6000 Ada / A6000 (48 GB) | Comfortable, long context | Comfortable, long context |
| RTX 5080 / 4080 (16 GB) | Comfortable | Tight — reduce context |
| Apple Silicon (36 GB+ unified) | Comfortable | Comfortable |
| AMD Strix Halo (ROCm/Vulkan, ≤96 GB) | Comfortable | Comfortable |

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run qwen3-vl
> ```
>
> Leave Ollama listening on the default `http://127.0.0.1:11434`. Raise `num_ctx` to use more context, and send images in your chat request the way you would with any other Ollama vision model.

> [tab] LM Studio (GUI)
>
> Search for **Qwen3-VL** in LM Studio's model catalog, download a `GGUF` quant that fits your VRAM (start with `Q4_K_M`), then start LM Studio's local server. It exposes an OpenAI-compatible API on `http://127.0.0.1:1234/v1`, including vision input.

> [tab] vLLM (advanced)
>
> Qwen recommends vLLM (`vllm>=0.11.0`) for fast Qwen3-VL serving. The official example targets a large multi-GPU box; scale the flags down for your hardware:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve Qwen/Qwen3-VL-8B-Instruct \
>   --max-model-len 262144 \
>   --media-io-kwargs '{"video": {"num_frames": -1}}'
> ```
>
> Add `--tensor-parallel-size <N>` to shard across GPUs. Qwen's own README does not publish a `--tool-call-parser` or `--reasoning-parser` flag for Qwen3-VL — check the [vLLM multimodal serving docs](https://docs.vllm.ai/en/latest/serving/multimodal_inputs.html) and the model card before relying on structured tool calls or thinking-tag parsing in production.

> [tab] SGLang (advanced)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> python -m sglang.launch_server \
>   --model-path Qwen/Qwen3-VL-8B-Instruct \
>   --host 0.0.0.0 \
>   --port 22002
> ```
>
> Add `--tp <N>` for tensor parallelism. Flags are from the [Qwen3-VL GitHub README](https://github.com/QwenLM/Qwen3-VL).

## Capabilities

* **Vision:** Multimodal — accepts text, images, and video in the same request, with native support for long-document OCR, GUI/agentic screen understanding, and spatial grounding.
* **Long context:** 262,144 tokens natively (confirmed for the `8B` and `30B-A3B` configs); Qwen's model card describes it as expandable further with YaRN scaling on vLLM.
* **Reasoning:** Ships as two separate editions per size — **Instruct** (direct answers) and **Thinking** (reasoning-enhanced, e.g. `qwen3-vl:8b-thinking` on Ollama). Qwen's public deployment examples do not publish a dedicated reasoning-parser flag for Qwen3-VL on vLLM or SGLang.
* **Tool-calling:** The chat template on Hugging Face defines a `<tool_call>` JSON format, and Ollama applies it automatically. Qwen's official vLLM/SGLang deployment examples for Qwen3-VL don't show a `--tool-call-parser` flag — test function-calling on your runtime before shipping it.

## Connect it with Tokios

Once Qwen3-VL is running locally, pair the Tokios connector so you can reach it as `https://api.tokios.com` from Claude Code, Cline, or any other compatible client — instead of only from `localhost:11434`.

**1. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Install and run the connector on the machine running Ollama:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Point the connector at Ollama's /v1 API**

Ollama exposes an OpenAI-compatible surface at `/v1`. For a single-model setup, the console's Setup wizard fills this in automatically as `http://127.0.0.1:11434/v1`. For multiple models, set it explicitly in `tokios-connector.json` — see [Connector config](https://docs.tokios.com/reference/connector-config.md).

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the model and choose a public deployment name, for example `qwen3-vl-tunnel`. This name — not Ollama's local model id — is what clients send in the `model` field.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-vl-tunnel",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "What is in this image?"},
          {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}}
        ]
      }
    ]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3-vl-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`qwen3-vl-tunnel` above), never Ollama's local model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) and [Use Cline](https://docs.tokios.com/guides/cline.md) for full client setup.

## Caveats

* Qwen3-VL ships Instruct and Thinking as separate downloads rather than one model with a toggle — pick the tag you need (`qwen3-vl:8b` vs. `qwen3-vl:8b-thinking`) up front.
* Qwen's official README and model cards don't publish explicit `--tool-call-parser` or `--reasoning-parser` flags for Qwen3-VL on vLLM/SGLang, unlike some other Qwen3 releases. Test tool-calling and thinking-tag behavior on your runtime before relying on it in production.
* The `30B-A3B` MoE model needs all of its total parameters in memory even though only 3B are active per token — size your hardware off the full download, not the active count.
* The `32B` and `235B-A22B` variants need more hardware than most local Tokios users will have. Treat them as advanced pulls and confirm their exact files, context settings, and runtime support before planning around them.

## FAQ

> [accordion] How much VRAM does Qwen3-VL need?
>
> The `8B` dense model needs \~6.1 GB at the default quant (\~18 GB at bf16). The `30B-A3B` MoE model needs \~20 GB at the default quant (\~62 GB at bf16). Add KV cache on top for your context length — both support 262,144 tokens natively, but long contexts multiply memory use.

> [accordion] Can Qwen3-VL run on a 24 GB GPU?
>
> Yes — both the `8B` (\~6 GB) and the `30B-A3B` MoE (\~20 GB) fit a 24 GB card (RTX 3090 / 4090) at the default quant. The `8B` leaves plenty of headroom for long context; the `30B-A3B` is comfortable but tighter at very large context lengths. Even a 16 GB card (RTX 5080 / 4080) runs the `8B` comfortably.

> [accordion] Is Qwen3-VL open source?
>
> Yes — both the `8B` and `30B-A3B` are released under **Apache 2.0**, which is permissive and carries no acceptable-use policy or user-count threshold. Note that open weights and open source are not the same thing — the training data and pipeline are not released.

> [accordion] What is the difference between Thinking and Instruct variants?
>
> They ship as separate downloads, not a toggle. **Instruct** editions give direct answers. **Thinking** editions (e.g. `qwen3-vl:8b-thinking`) are reasoning-enhanced and emit step-by-step reasoning before the final answer. Pick the tag you need up front — Ollama's `-thinking` suffix selects the reasoning edition.

> [accordion] Does Qwen3-VL support video input?
>
> Yes — the full Qwen3-VL family accepts text, images, and video in the same request, with native support for long-document OCR, GUI/agentic screen understanding, and spatial grounding. This is one of the differences from the earlier Qwen2.5-VL.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> Match Qwen3-VL to your available VRAM before you pull it.

> [card] [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Which of Qwen3-VL's formats — `GGUF`, `bf16` — runs on your hardware.

> [card] [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md)
>
> See how Qwen3-VL compares to other local vision models by task.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.