> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run DeepSeek-R1 locally: distills, hardware fit, and setup

> Serve a DeepSeek-R1 distill with Ollama — or the 671B R1 on a multi-GPU server — then reach it from any OpenAI- or Anthropic-compatible client through Tokios.

> [!NOTE]
>
> Last verified: 2026-10-14

DeepSeek-R1 is DeepSeek's earlier reasoning family. The way almost everyone runs it locally is a **distill** — a smaller Qwen or Llama model fine-tuned on R1's reasoning traces, so the reasoning behavior runs on a laptop-class GPU instead of a multi-GPU server. Use this page when you need that smaller hardware path. For DeepSeek's current open-weight release, coding-agent improvements, tool calling, and 1M context, start with [`DeepSeek-V4-Flash-0731`](https://docs.tokios.com/models/deepseek). Pair Tokios with whichever model you run to reach it from any machine, teammate, or hosted agent through `https://api.tokios.com` — no inbound ports, no VPN.

## Run it locally with Ollama

The `deepseek-r1` tag on Ollama ships the distills, not the full model. Pull and run the default (currently the `7b` Qwen distill):

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run deepseek-r1
```

For a specific size, name the tag directly — for example `ollama run deepseek-r1:14b`. See the [Ollama library page](https://ollama.com/library/deepseek-r1) for the full tag list.

> [!WARNING]
>
> Ollama's default context window is 2048 tokens, far below what these models support — and R1-style reasoning burns through context fast on its way to an answer. Raise `num_ctx` (via `/set parameter num_ctx <value>` in the CLI, or in your Modelfile/API request) toward the context size listed in the table below.

## Variants & tags

Every value below is from the Ollama library and each model's Hugging Face config. Tags, sizes, and defaults change as new quantizations ship — confirm against [ollama.com/library/deepseek-r1](https://ollama.com/library/deepseek-r1) before you standardize on one.

| Distill | Base model | Ollama tag (`Q4_K_M`) | Download size | Context |
| - | - | - | - | - |
| 1.5B | Qwen2.5-Math-1.5B | `deepseek-r1:1.5b` | \~1.1 GB | 128K |
| 7B | Qwen2.5-Math-7B | `deepseek-r1:7b` (default) | \~4.7 GB | 128K |
| 8B | Llama-3.1-8B | `deepseek-r1:8b` | \~4.9 GB | 128K |
| 8B (0528, Qwen3 base) | Qwen3-8B | `deepseek-r1:8b-0528-qwen3-q4_K_M` | \~5.2 GB | 128K |
| 14B | Qwen2.5-14B | `deepseek-r1:14b` | \~9.0 GB | 128K |
| 32B | Qwen2.5-32B | `deepseek-r1:32b` | \~20 GB | 128K |
| 70B | Llama-3.3-70B-Instruct | `deepseek-r1:70b` | \~43 GB | 128K |
| 671B (full R1) | — | `deepseek-r1:671b` | \~404 GB | 160K |
| 671B (R1-0528, full) | — | `deepseek-r1:671b-0528-q4_K_M` | \~404 GB | 160K |

Sources: [ollama.com/library/deepseek-r1/tags](https://ollama.com/library/deepseek-r1/tags), [deepseek-ai/DeepSeek-R1 README](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/README.md).

> [!TIP]
>
> The `671b` tags are the **full** DeepSeek-R1 (and R1-0528), not distills — Ollama just publishes them under the same `deepseek-r1` name. Everything from `1.5b` through `70b` is a distill: a Qwen or Llama model fine-tuned on R1's reasoning traces, not R1 itself shrunk down. See [how inference works](https://docs.tokios.com/concepts/how-inference-works.md) for what that distinction means for output quality.

## Quantization & formats

Ollama's default tags are `GGUF` `Q4_K_M`. Every distill size and the full `671b` also publish `-q8_0` and `-fp16` tags on the library page for less lossy (and much larger) downloads. See [Quantization formats and the hardware they need](https://docs.tokios.com/models/quantization-and-hardware.md) for what runs on your GPU, and [what quantization is](https://docs.tokios.com/concepts/quantization.md) for how to read the `Q4_K_M`-style names.

## Hardware fit

Comfort at roughly `Q4` for a modest context length. Treat every entry as an estimate — the ground truth is the actual `GGUF` size plus the KV cache for your context. See [what size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math.

| Target | 1.5B–8B (\~1–5 GB) | 14B (\~9 GB) | 32B (\~20 GB) | 70B (\~43 GB) | 671B (\~404 GB) |
| - | - | - | - | - | - |
| Laptop / 8 GB GPU | Comfortable | Tight | Won't fit | Won't fit | Won't fit |
| RTX 3090 / 4090 (24 GB) | Comfortable | Comfortable | Comfortable | Won't fit | Won't fit |
| RTX 5090 (32 GB) | Comfortable | Comfortable | Comfortable, longer context | Won't fit | Won't fit |
| RTX 6000 Ada / A6000 (48 GB) | Comfortable | Comfortable | Comfortable, long context | Tight, short context | Won't fit |
| Apple Silicon / Strix Halo (96–128 GB unified) | Comfortable | Comfortable | Comfortable | Comfortable | Won't fit |
| Multi-GPU server (8× 80 GB class) | Comfortable | Comfortable | Comfortable | Comfortable | Comfortable at `Q4` |

The full `671b` needs roughly 404 GB just for weights at `Q4_K_M` — that's multiple datacenter-class GPUs or an equivalent unified-memory cluster, not a single workstation. Everything `70b` and under is realistic on the hardware hobbyists actually own.

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run deepseek-r1:14b
> ```
>
> Leave Ollama listening on the default `http://127.0.0.1:11434`. Raise `num_ctx` to use more of the model's context window.

> [tab] LM Studio (GUI)
>
> Search for **DeepSeek R1** in LM Studio's model catalog, pick a distill size that fits your VRAM (start with `14b` or `32b` at `Q4_K_M`), and start LM Studio's local server. It exposes an OpenAI-compatible API on `http://127.0.0.1:1234/v1`.

> [tab] vLLM (advanced)
>
> For a distill:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
>   --tensor-parallel-size 2 \
>   --max-model-len 32768
> ```
>
> For the full model, shard across enough GPUs to hold \~404 GB (`Q4`) or more at higher precision. Flags are from the model card's vLLM section.

> [tab] SGLang (advanced)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> python3 -m sglang.launch_server \
>   --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
>   --trust-remote-code \
>   --tp 2
> ```
>
> Swap the model id for `deepseek-ai/DeepSeek-R1` and raise `--tp` to run the full model. Flags are from the model card's SGLang section.

## Capabilities

* **Reasoning:** R1 and every distill emit a `<think>…</think>` block before the final answer. DeepSeek's own model card recommends a temperature of `0.5–0.7` (0.6 suggested), no system prompt, and — for math — explicitly asking the model to "reason step by step" and box its final answer.
* **Native tool-calling:** **Not reliable on the distills.** Ollama's chat template for the R1 distills doesn't support tool/function-calling the way Llama 3.1/3.3 or Qwen3 do — see [ollama/ollama#8517](https://github.com/ollama/ollama/issues/8517) (closed, unresolved for the distill template). If your workload needs an agent that calls tools, pick a model built for it — see [Choose a model by task](https://docs.tokios.com/guides/model-by-task.md).
* **Context:** 128K for every distill (`1.5b` through `70b`); 160K for the full `671b` R1 and R1-0528.
* **Vision:** None of the models on this page are multimodal — DeepSeek-R1 and its distills are text-only.

## Beyond R1: use V4 Flash 0731 for the current frontier

R1 is a reasoning-tuned checkpoint built on DeepSeek's V3 architecture. The current official release is **`deepseek-ai/DeepSeek-V4-Flash-0731`**, which supersedes the V4 Flash preview. It is open-weight and MIT-licensed, but it needs a multi-device server rather than a typical single workstation. The [DeepSeek family overview](https://docs.tokios.com/models/deepseek.md) covers its 284B-target / 13B-active architecture, bundled DSpark head, tool calling, 1M context, and local hardware paths.

## Connect it with Tokios

Once a DeepSeek model is running locally, pair the Tokios connector so you can reach it as `https://api.tokios.com` from Claude Code, Cline, or any other compatible client — instead of only from `localhost:11434`.

**1. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Install and run the connector on the machine running Ollama:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Point the connector at Ollama's /v1 API**

Ollama exposes an OpenAI-compatible surface at `/v1`. For a single-model setup, the console's Setup wizard fills this in automatically as `http://127.0.0.1:11434/v1`. For multiple models, set it explicitly in `tokios-connector.json` — see [Connector config](https://docs.tokios.com/reference/connector-config.md).

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the model and choose a public deployment name, for example `deepseek-r1-tunnel`. This name — not Ollama's local model id — is what clients send in the `model` field.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Explain the halting problem."}]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model deepseek-r1-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`deepseek-r1-tunnel` above), never Ollama's local model id. Tokios turns whatever's running on `localhost` into a stable, authenticated endpoint — it doesn't change what the model can do. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) and [Use Cline](https://docs.tokios.com/guides/cline.md) for full client setup.

## Caveats

* **Tool-calling is unreliable on the R1 distills.** If you're wiring up an agent, CLI, or IDE plugin that needs the model to call tools, use a model with confirmed native tool support instead — see [Choose a model by task](https://docs.tokios.com/guides/model-by-task.md).
* **Raise the context window.** Ollama's 2048-token default cuts off R1-style reasoning output quickly; increase `num_ctx` for real use, up to 128K (distills) or 160K (full `671b`).
* **A distill is not the full model.** The `1.5b`–`70b` tags are Qwen/Llama checkpoints fine-tuned on R1's reasoning traces — they reproduce the reasoning *behavior* on light hardware, not R1's full capability. Benchmark quality drops as size drops; see the [DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1) for the distill evaluation table.
* **Licensing differs by distill.** The full R1/R1-0528 weights and the Qwen-based distills (`1.5b`, `7b`, `14b`, `32b`) are MIT. The Llama-based distills (`8b`, `70b`) carry Meta's Llama 3.1/3.3 license instead, inherited from their base model — check that license before commercial use of those two specific tags.
* **671B needs real multi-GPU hardware** — roughly 404 GB at `Q4_K_M` alone — not a typical single-workstation setup. The current V4 Flash 0731 checkpoint is smaller but still needs roughly 162–167 GB before KV cache and runtime overhead; see the [DeepSeek family overview](https://docs.tokios.com/models/deepseek.md).
* Tokios passes tool-calling requests straight through to whatever the upstream model supports — it doesn't add tool-calling capability the underlying model lacks.

> [card] [DeepSeek family overview](https://docs.tokios.com/models/deepseek.md)
>
> R1 versus V3.x versus the V4 frontier — which tier fits your hardware.

> [card] [Choose a model by task](https://docs.tokios.com/guides/model-by-task.md)
>
> Pick a model with reliable tool-calling if you're building an agent.

> [card] [Match a model to your GPU](https://docs.tokios.com/guides/model-fit.md)
>
> Compare VRAM classes across distill sizes before you pick a tag.

> [card] [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Which formats — `GGUF`, `FP16`, and beyond — run on your hardware.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.