> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run Phi locally and use it from anywhere

> Serve Microsoft's Phi-4 family — small models with strong reasoning for their size — with Ollama or LM Studio, then reach it via a Tokios endpoint.

> [!NOTE]
>
> Last verified: 2026-10-14

Phi-4 is Microsoft's small-model family: dense models trained on curated, reasoning-heavy synthetic data so they punch above their parameter count on math and logic. The lineup spans a 14B desktop model, a 3.8B mini model light enough for a laptop or edge box, and a 5.6B multimodal model that adds image and audio input. Run whichever fits your hardware locally, then pair the Tokios connector to call it from any machine, teammate, or hosted agent through `https://api.tokios.com` — no inbound ports, no VPN.

## Run it locally with Ollama

Phi-4 (14B) is in the Ollama library. Pull and run it:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run phi4
```

For the smaller mini model, run `ollama run phi4-mini`. See the [Ollama library page](https://ollama.com/library/phi4) for the full tag list — this page covers connecting it, not every runtime detail.

> [!NOTE]
>
> Ollama's default context window is small (2048 tokens) — well below what Phi-4 supports natively. Raise `num_ctx` in your Modelfile or client request to use more context, especially on the mini model's 128K window.

## Variants & tags

Every value below is from the model cards on Hugging Face and the Ollama library. Tags and download sizes change as new quantizations ship — confirm against [ollama.com/library/phi4](https://ollama.com/library/phi4) and [ollama.com/library/phi4-mini](https://ollama.com/library/phi4-mini) before you standardize on one.

| Property | `phi4` (14B) | `phi4-mini` (3.8B) | Phi-4-multimodal (5.6B) |
| - | - | - | - |
| Ollama tag | `phi4:14b` (alias `:latest`) | `phi4-mini:3.8b` (alias `:latest`) | Not on Ollama — Hugging Face / Transformers only |
| Download size (`Q4_K_M`) | \~9.1 GB | \~2.5 GB | — |
| Parameters | 14B dense | 3.8B dense | 5.6B (Phi-4-mini backbone + vision/speech encoders) |
| Context window | 16,384 tokens | 131,072 tokens | 131,072 tokens |
| Modality | Text | Text | Text + image + audio |
| Tool/function calling | No (chat template has no tool role) | Yes — `<\|tool\|>` system-prompt format | Yes — `<\|tool\|>` system-prompt format |
| License | MIT | MIT | MIT |
| Hugging Face | [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4) | [`microsoft/Phi-4-mini-instruct`](https://huggingface.co/microsoft/Phi-4-mini-instruct) | [`microsoft/Phi-4-multimodal-instruct`](https://huggingface.co/microsoft/Phi-4-multimodal-instruct) |

Sources: [ollama.com/library/phi4](https://ollama.com/library/phi4), [ollama.com/library/phi4-mini](https://ollama.com/library/phi4-mini), [microsoft/phi-4 model card](https://huggingface.co/microsoft/phi-4), [microsoft/Phi-4-mini-instruct model card](https://huggingface.co/microsoft/Phi-4-mini-instruct), [microsoft/Phi-4-multimodal-instruct model card](https://huggingface.co/microsoft/Phi-4-multimodal-instruct).

> [!TIP]
>
> There are also `phi4-reasoning` (14B, `<think>`-style chain-of-thought, 32K context) and `phi4-mini-reasoning` (3.8B, 128K context) tags in the Ollama library, fine-tuned from the base models for explicit reasoning traces. They're separate pulls (`ollama run phi4-reasoning`), not a flag on the base models — see [ollama.com/library/phi4-reasoning](https://ollama.com/library/phi4-reasoning) if you want the reasoning-tuned variant instead of the instruct one covered on this page.

## Quantization & formats

Ollama's default tags are `GGUF` `Q4_K_M`. The library also publishes `-q8_0` and `-fp16` for both `phi4` and `phi4-mini`. Which format you can run depends on your hardware — see [Quantization formats and the hardware they need](https://docs.tokios.com/models/quantization-and-hardware.md) for the full matrix, and [what quantization is](https://docs.tokios.com/concepts/quantization.md) for how to read the `Q4_K_M`-style names.

## Hardware fit

Comfort at roughly `Q4` for a modest context length. Treat every entry as an estimate — the ground truth is the actual `GGUF` size plus the KV cache for your context. See [what size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math.

| Target | `phi4-mini` (\~2.5 GB) | `phi4` 14B (\~9.1 GB) |
| - | - | - |
| Laptop / integrated GPU, CPU-only | Comfortable | Slow but workable |
| RTX 3060 / 4060 (8–12 GB) | Comfortable, long context | Comfortable |
| RTX 3090 / 4090 (24 GB) | Comfortable, full context | Comfortable, long context |
| RTX 5090 (32 GB, Blackwell) | Comfortable | Comfortable, full 16K+ |
| Apple Silicon (16 GB+ unified) | Comfortable | Comfortable |
| AMD Strix Halo (ROCm/Vulkan) | Comfortable | Comfortable |

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run phi4
> ```
>
> Leave Ollama listening on the default `http://127.0.0.1:11434`. Raise `num_ctx` to use more context, and swap in `phi4-mini` for a lighter footprint.

> [tab] LM Studio (GUI)
>
> Search for **Phi-4** in LM Studio's model catalog, download a `GGUF` quant that fits your VRAM (start with `Q4_K_M`), then start LM Studio's local server. It exposes an OpenAI-compatible API on `http://127.0.0.1:1234/v1`.

> [tab] Transformers (advanced)
>
> Phi-4-mini and Phi-4-multimodal are the variants worth self-hosting outside Ollama — multimodal needs `trust_remote_code` for its custom vision/audio modules:
>
> ```python theme={"theme":{"light":"github-light","dark":"css-variables"}}
> from transformers import AutoModelForCausalLM, AutoTokenizer
>
> model = AutoModelForCausalLM.from_pretrained(
>     "microsoft/Phi-4-mini-instruct",
>     device_map="cuda",
>     torch_dtype="auto",
>     trust_remote_code=True,
> )
> tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-4-mini-instruct")
> ```
>
> For Phi-4-multimodal, load with `AutoModelForCausalLM.from_pretrained("microsoft/Phi-4-multimodal-instruct", trust_remote_code=True)` and use the model card's processor for image/audio inputs — it isn't a plain chat pipeline. Flags are from each model card's usage section.

> [tab] vLLM (advanced)
>
> Phi-4-multimodal's model card documents serving it with vLLM, including its speech/vision LoRA adapters:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve microsoft/Phi-4-multimodal-instruct \
>   --dtype auto \
>   --trust-remote-code \
>   --max-model-len 131072 \
>   --enable-lora \
>   --max-lora-rank 320 \
>   --limit-mm-per-prompt audio=3,image=3 \
>   --max-loras 2
> ```
>
> Add `--tensor-parallel-size <N>` to shard across GPUs. Flags are from the model card's vLLM section.

## Capabilities

* **Reasoning:** The instruct models covered here answer directly — no `<think>` block. Microsoft ships separate `phi4-reasoning` and `phi4-mini-reasoning` checkpoints (see the tip above) that emit explicit `<think>…</think>` chain-of-thought for math-heavy tasks; pull those instead if you want that behavior.
* **Tool-calling:** Phi-4 (14B) has no tool role in its chat template — don't rely on structured function calls from it. Phi-4-mini and Phi-4-multimodal support it: wrap your tool definitions in `<|tool|>...<|/tool|>` inside the system message, per their model cards.
* **Vision and audio:** Only Phi-4-multimodal. It accepts text, image, and audio in the same prompt (`<|image_1|>`, `<|audio_1|>` placeholders) and is not available as an Ollama tag — run it via Transformers or vLLM.
* **Long context:** Phi-4 (14B) is limited to 16,384 tokens. Phi-4-mini and Phi-4-multimodal both support 131,072 tokens.

## Connect it with Tokios

Once Phi-4 is running locally, pair the Tokios connector so you can reach it as `https://api.tokios.com` from Claude Code, Cline, or any other compatible client — instead of only from `localhost:11434`.

**1. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Install and run the connector on the machine running Ollama:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Point the connector at Ollama's /v1 API**

Ollama exposes an OpenAI-compatible surface at `/v1`. For a single-model setup, the console's Setup wizard fills this in automatically as `http://127.0.0.1:11434/v1`. For multiple models, set it explicitly in `tokios-connector.json` — see [Connector config](https://docs.tokios.com/reference/connector-config.md).

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the model and choose a public deployment name, for example `phi-tunnel`. This name — not Ollama's local model id — is what clients send in the `model` field.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "phi-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Walk through solving this step by step: what is the derivative of x^3 + 2x?"}]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model phi-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`phi-tunnel` above), never Ollama's local model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) and [Use Cline](https://docs.tokios.com/guides/cline.md) for full client setup.

## Caveats

* Phi-4 (14B) tops out at a 16,384-token context window — much shorter than most current open models. If you need long context on a Phi model, use `phi4-mini` (131,072 tokens) instead.
* Phi-4 (14B) has no tool-calling support in its chat template. Reach for `phi4-mini` if your workflow needs function calls.
* Phi-4-multimodal isn't on Ollama — you'll need Transformers or vLLM (with `trust_remote_code`) to serve its vision/audio capabilities locally.
* Reasoning-tuned variants (`phi4-reasoning`, `phi4-mini-reasoning`) are separate model pulls, not a mode switch on the instruct models covered here.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> Match Phi-4 to your available VRAM before you pull it.

> [card] [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Which of Phi-4's `GGUF` formats runs on your hardware.

> [card] [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md)
>
> See how Phi-4 compares to other local models by task.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.