> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run Nemotron locally and use it from anywhere

> Serve NVIDIA's Nemotron 3 family — hybrid Mamba/transformer reasoning models — with Ollama, vLLM, or SGLang, then reach it via a Tokios endpoint.

> [!NOTE]
>
> Last verified: 2026-10-14

Nemotron is NVIDIA's family of open-weight reasoning models, built on `nemotron_h` — a **hybrid Mamba-2/transformer architecture** that mixes state-space layers with a handful of attention layers instead of using attention everywhere. That hybrid design is what lets Nemotron 3 models natively support very long context (up to 1M tokens on the largest checkpoints) without the usual attention-only memory blowup. The family spans a local-friendly `Nano` tier up through multi-GPU `Super` and `Ultra` models, all tuned for agentic use: reasoning traces, tool-calling, and instruction-following are trained in, not bolted on. Run the size that fits your hardware locally, then pair the Tokios connector to call it from any machine, teammate, or hosted agent through `https://api.tokios.com` — no inbound ports, no VPN.

> [!NOTE]
>
> Nemotron 3 weights are released under the **[NVIDIA Nemotron Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/)** (Hugging Face tags it `other`), not Apache or MIT. It's an open-weights license that allows commercial use under its terms — read them before you redistribute or fine-tune on top of these weights.

## Run it locally with Ollama

Two Nemotron 3 **Nano**-tier builds are in the Ollama library — Nano is the local-friendly tier:

* **`nemotron3`** is the **Nano Omni** multimodal build (text + image input). `ollama run nemotron3` pulls the default `nemotron3:33b` (\~28 GB), served at 128K context.
* **`nemotron-3-nano`** is the **text-only** Nano reasoning model, served at its full **1M** context. `ollama run nemotron-3-nano` pulls the `30B-A3B` MoE (\~24 GB at `q4_K_M`, only \~3.5B active per token); `ollama run nemotron-3-nano:4b` is a \~2.8 GB dense model for laptops and edge boxes.

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
# Multimodal Nano Omni (text + image, 128K context)
ollama run nemotron3

# Text-only Nano, full 1M context — 30B MoE, ~3.5B active per token
ollama run nemotron-3-nano
```

The bigger tiers are on Ollama too: `nemotron-3-super:120b` (\~87 GB, multi-GPU) and `nemotron-3-ultra` (cloud-only — no downloadable local-weights tag). NVIDIA's older tuned Llama, `nemotron` (Llama-3.1-Nemotron-70B), is a separate, earlier line. See the [nemotron-3-nano](https://ollama.com/library/nemotron-3-nano) library page for the current tag list — this page covers connecting it, not every runtime detail.

> [!NOTE]
>
> Ollama's default context window is small (2048 tokens) — far below Nemotron 3 Nano's 1M. Raise `num_ctx` in your Modelfile or client request to use more of its context. The `nemotron3` (Nano Omni) tag is served at up to 128K.

## Variants & tags

Every value below is from Hugging Face model cards and the Ollama library. Tags and download sizes change as new quantizations ship — confirm against [ollama.com/library/nemotron3](https://ollama.com/library/nemotron3) before you standardize on one.

| Property | Nano Omni (`nemotron3`) | Nano text-only (`nemotron-3-nano`) | Super | Ultra |
| - | - | - | - | - |
| Ollama tag | `nemotron3:33b` (alias `:latest`) | `nemotron-3-nano:30b` (MoE); `:4b` (dense) | `nemotron-3-super:120b` | `nemotron-3-ultra:cloud` (cloud-only) |
| Download size | \~28 GB (`q4_K`) | \~24 GB (`q4_K_M`); `:4b` \~2.8 GB | \~87 GB | cloud-only on Ollama; NVFP4 on HF |
| Parameters | \~30B total (Omni build) | 30B total / **3.5B active** (128 routed + 1 shared, 6 per token); `:4b` = \~4B dense | 120B total / **12B active** | 550B total / **55B active** |
| Context window | 128K (as served) | 262,144 native, extensible to 1M | 262,144 native, extensible to 1M | up to 1M |
| Modality | Text + image | Text | Text | Text |
| Reasoning | On by default, toggleable | On by default, toggleable | On by default, toggleable | On by default, toggleable |
| License | NVIDIA Nemotron Open Model License | NVIDIA Nemotron Open Model License | NVIDIA Nemotron Open Model License | NVIDIA Nemotron Open Model License |
| Hugging Face | [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16) | [`nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) | [`nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16) | [`nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4) |

Sources: [ollama.com/library/nemotron-3-nano](https://ollama.com/library/nemotron-3-nano), [ollama.com/library/nemotron3](https://ollama.com/library/nemotron3), [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16), [NVIDIA-Nemotron-3-Super-120B-A12B-BF16 model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16), [NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4).

> [!TIP]
>
> Two Nano builds, two Ollama tags: `nemotron3` is Nano **Omni** (multimodal, 128K as served), while `nemotron-3-nano` is the **text-only** Nano with the full 1M context. Both are mixture-of-experts — see [how inference works](https://docs.tokios.com/concepts/how-inference-works.md) for why the full parameter count still has to fit in memory even though only the \~3.5B active slice runs per token. For the documented vLLM/SGLang tool-call and reasoning parsers, serve the Hugging Face checkpoint directly (see the tabs below).

## Quantization & formats

Ollama's `nemotron3` tags are `GGUF` at `q4_K`, `q8`, and `bf16`. On Hugging Face, NVIDIA also publishes **NVFP4** builds of the larger tiers — the `Ultra-550B-A55B-NVFP4` checkpoint is trained natively in NVFP4, and `Super` and Nano Omni both have NVFP4 variants alongside their `BF16` originals. NVFP4 is NVIDIA's 4-bit format built specifically for **Blackwell** Tensor Cores (`RTX 50` series and Blackwell-generation datacenter GPUs) — it won't run natively on older NVIDIA cards. See [Quantization formats and the hardware they need](https://docs.tokios.com/models/quantization-and-hardware.md) for the full format×hardware matrix, and [what quantization is](https://docs.tokios.com/concepts/quantization.md) for how to read quant names like `Q4_K_M`.

## Hardware fit

Treat every entry as an estimate — the ground truth is the actual download size plus the KV cache for your context. See [what size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md) for the sizing math.

| Target | Nano / Nano Omni (\~28-30 GB `Q4`/`bf16`) | Super 120B-A12B (multi-GPU) | Ultra 550B-A55B (multi-GPU, NVFP4) |
| - | - | - | - |
| RTX 3090 / 4090 (24 GB) | Tight — use `q4_K`, reduce context | Won't fit | Won't fit |
| RTX 5090 (32 GB, Blackwell) | Comfortable | Won't fit on one card | Won't fit on one card |
| RTX 6000 Ada / A6000 (48 GB) | Comfortable, long context | Won't fit on one card | Won't fit on one card |
| Multi-GPU workstation / server (Blackwell, 4-8x 80GB+ class) | Overkill — use Nano | Fits with tensor parallelism | NVFP4 build targets exactly this class of hardware |
| Apple Silicon / AMD Strix Halo | Nano `GGUF` builds should run; check for a community `GGUF`/`MLX` conversion | Not practical | Not practical |

Super (`nemotron-3-super:120b`, \~87 GB) and Ultra are datacenter/multi-GPU tiers — Ultra is cloud-only on Ollama. If you're running local on a single workstation, Nano (`nemotron-3-nano`) or Nano Omni (`nemotron3`) is the realistic pick. See [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md) for the NVFP4/Blackwell requirement in more detail.

## Run it — beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run nemotron3
> ```
>
> Leave Ollama listening on the default `http://127.0.0.1:11434`. This is the Nano Omni build (128K context as served); see the note above if you need the text-only Nano checkpoint with full context instead.

> [tab] vLLM (advanced, text-only Nano)
>
> The text-only Nano checkpoint needs its reasoning and tool-call parsers set explicitly, plus a custom reasoning-parser plugin file from the model repo:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/resolve/main/nano_v3_reasoning_parser.py
>
> vllm serve nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 \
>   --served-model-name model \
>   --max-num-seqs 8 \
>   --tensor-parallel-size 1 \
>   --max-model-len 262144 \
>   --trust-remote-code \
>   --enable-auto-tool-choice \
>   --tool-call-parser qwen3_coder \
>   --reasoning-parser-plugin nano_v3_reasoning_parser.py \
>   --reasoning-parser nano_v3
> ```
>
> Set `VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` and `--max-model-len 1M` to use the full 1M context instead of 256K. Flags are from the model card's vLLM section.

> [tab] SGLang (advanced, text-only Nano)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> python3 -m sglang.launch_server \
>   --model-path nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 \
>   --trust-remote-code \
>   --tp 1 \
>   --attention-backend flashinfer \
>   --tool-call-parser qwen3_coder \
>   --reasoning-parser nano_v3
> ```
>
> Flags are from the model card's SGLang section.

## Capabilities

* **Reasoning:** Nemotron 3 models are unified reasoning/non-reasoning models — they emit a reasoning trace before the final answer by default, and it's on across the whole line (Nano, Super, and Ultra all default to `enable_thinking=True`). Turn it off with `enable_thinking=False` in the chat template if you want a direct response at a slight accuracy cost on harder prompts. Budget control (`reasoning_budget`) lets you cap how many tokens the reasoning trace can use.
* **Native tool-calling:** Yes — the model cards document tool-calling support with the `qwen3_coder` tool-call parser on both vLLM and SGLang, trained in via synthetic tool-calling data during post-training. On vLLM/SGLang you must set the parser flags shown above, or tool calls won't parse; when combining tools with reasoning, also set `"chat_template_kwargs": {"enable_thinking": true, "force_nonempty_content": true}` in the request.
* **Vision:** The plain-text Nano/Super/Ultra checkpoints are text-only. The Ollama `nemotron3` tag is Nano **Omni** and exposes text + image input. Broader video/audio/image/text behavior belongs to the upstream Omni model card, not to the simple Ollama tag surface documented here.
* **Long context:** Up to 262,144 tokens natively on Nano/Super, extensible to 1M with `VLLM_ALLOW_LONG_MAX_MODEL_LEN=1`. The hybrid Mamba-2/attention architecture is what makes that context length practical — a pure-attention model this size would need far more memory for the KV cache. Ollama's `nemotron3` tag currently exposes only 128K of that range.

## Connect it with Tokios

Once Nemotron is running locally, pair the Tokios connector so you can reach it as `https://api.tokios.com` from Claude Code, Cline, or any other compatible client — instead of only from `localhost:11434`.

**1. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing** to get a one-time claim code (`TKS-XXXX-XXXX`). Install and run the connector on the machine running Ollama:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Then **approve** the device on the **Setup** or **Connectors** tab.

**2. Point the connector at Ollama's /v1 API**

Ollama exposes an OpenAI-compatible surface at `/v1`. For a single-model setup, the console's Setup wizard fills this in automatically as `http://127.0.0.1:11434/v1`. For multiple models, set it explicitly in `tokios-connector.json` — see [Connector config](https://docs.tokios.com/reference/connector-config.md).

**3. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the model and choose a public deployment name, for example `nemotron-tunnel`. This name — not Ollama's local model id — is what clients send in the `model` field.

**4. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-…` key now — it's shown only once.

### Call it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nemotron-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Walk through this incident report and flag the root cause."}]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model nemotron-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`nemotron-tunnel` above), never Ollama's local model id. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) and [Use Cline](https://docs.tokios.com/guides/cline.md) for full client setup.

## Caveats

* Nemotron emits a reasoning trace by default across the family. If a downstream client shows the raw thinking or you want lower latency, disable it with `enable_thinking=False` or set a `reasoning_budget`.
* The Ollama `nemotron3` tag is **Nano Omni** (multimodal, 128K as served). For the **text-only** Nano with the full 1M context, pull `nemotron-3-nano` instead — or serve the Hugging Face checkpoint on vLLM/SGLang for the documented tool-call and reasoning parsers.
* Tool calling requires the `qwen3_coder` parser flags on vLLM/SGLang, plus `force_nonempty_content` when combining tools with reasoning — without them, tool calls won't parse correctly.
* Every Nemotron 3 tier (Nano, Super, Ultra) is a mixture-of-experts model — size your hardware off the **total** parameter count, not the active-parameter count. Super and Ultra are multi-GPU/datacenter tiers; don't plan to run them on a single consumer card.
* Nemotron 3 uses the **[NVIDIA Nemotron Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/)**, not Apache or MIT — commercial use is allowed under its terms, but check the exact checkpoint license before redistribution or fine-tuning.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first model end to end.

> [card] [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md)
>
> Match Nemotron's Nano, Super, or Ultra tier to your available VRAM before you pull it.

> [card] [Quantization & hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> The NVFP4/Blackwell requirement behind Nemotron's Super and Ultra quantized builds.

> [card] [Which local model for which task?](https://docs.tokios.com/guides/model-by-task.md)
>
> See how Nemotron compares to other local models by task.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.