> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Run gpt-oss locally and use it from anywhere

> Serve OpenAI's open-weight gpt-oss models with Ollama, LM Studio, vLLM, or SGLang, then reach them from any OpenAI- or Anthropic-compatible client via Tokios.

> [!NOTE]
>
> Last verified: 2026-10-14

gpt-oss is OpenAI's open-weight reasoning model family. It gives you two mixture-of-experts sizes: `gpt-oss-20b` for local and lower-latency use, and `gpt-oss-120b` for workstation or server hardware. Run the model locally first, then use the Tokios connector to expose that local runtime through `https://api.tokios.com` without opening inbound ports.

## Run it locally with Ollama

Start with `20b` unless you already know your machine can hold the larger model:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run gpt-oss:20b
```

For the larger tier:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run gpt-oss:120b
```

Leave Ollama on its default loopback address, `http://127.0.0.1:11434`. Tokios will connect to Ollama's OpenAI-compatible `/v1` API later.

## Variants and tags

Tags and quantizations change over time. Confirm the exact tag on [Ollama's gpt-oss library page](https://ollama.com/library/gpt-oss) before you standardize on it.

| Variant | Ollama tag | Hugging Face repo | Parameters | Active parameters | Download size | Context | Input | License |
| - | - | - | - | - | - | - | - | - |
| gpt-oss 20B | [`gpt-oss:20b`](https://ollama.com/library/gpt-oss:20b) | [`openai/gpt-oss-20b`](https://huggingface.co/openai/gpt-oss-20b) | 21B MoE | 3.6B per token | 14 GB | 128K | Text | Apache 2.0 |
| gpt-oss 120B | [`gpt-oss:120b`](https://ollama.com/library/gpt-oss:120b) | [`openai/gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b) | 117B MoE | 5.1B per token | 65 GB | 128K | Text | Apache 2.0 |

Sources: [Ollama gpt-oss tags](https://ollama.com/library/gpt-oss/tags), [OpenAI gpt-oss announcement](https://openai.com/index/introducing-gpt-oss/), [`openai/gpt-oss-20b`](https://huggingface.co/openai/gpt-oss-20b), and [`openai/gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b).

> [!NOTE]
>
> The Ollama `gpt-oss:latest` tag currently points at the `20b` download. Pin `gpt-oss:20b` or `gpt-oss:120b` in production so the local upstream model id stays clear.

## Quantization and formats

OpenAI ships gpt-oss in `MXFP4`: the MoE weights are quantized to 4.25 bits per parameter. Ollama supports that format natively for gpt-oss, so the beginner path does not need a manual conversion step. OpenAI states that `gpt-oss-20b` can run within 16 GB of memory and `gpt-oss-120b` can fit on a single 80 GB GPU when using its native quantization.

Use the format, not just the parameter count, when you choose hardware:

| Format | Where to use it | Notes |
| - | - | - |
| `MXFP4` | Ollama, llama.cpp, vLLM | Native gpt-oss format. This is the main local path. |
| `bf16` / `FP16` | Research or high-memory serving | Much larger than the local-friendly quantized path. |
| `GGUF` | llama.cpp-based apps when a compatible build exists | Check the model card or runtime catalog before you pull a third-party conversion. |
| `AWQ` / `GPTQ` / `NVFP4` | Only if your serving repo publishes that format | Do not assume a quant exists just because another model family has it. |

See [Quantization formats and the hardware they need](https://docs.tokios.com/models/quantization-and-hardware.md) for the format matrix, and [what quantization is](https://docs.tokios.com/concepts/quantization.md) for the `Q4` / `Q8` naming basics.

## Hardware fit

Treat this as a planning guide for the native `MXFP4` / Ollama path at a modest context length. The real limit is the downloaded model size plus KV cache for your context.

| Target | `gpt-oss:20b` | `gpt-oss:120b` |
| - | - | - |
| Laptop CPU or 8 GB GPU | Won't fit comfortably | Won't fit |
| 16 GB GPU or 16 GB unified memory | Tight; keep context modest | Won't fit |
| RTX 3090 / 4090, 24 GB | Comfortable | Won't fit |
| RTX 5090, 32 GB | Comfortable, more room for context | Won't fit |
| RTX 6000 Ada / A6000, 48 GB | Comfortable | Won't fit |
| Apple Silicon or Strix Halo, 96-128 GB unified | Comfortable | Fits by capacity; expect lower bandwidth than dedicated VRAM |
| Single 80 GB datacenter GPU | Comfortable | Fits by OpenAI's target |
| Multi-GPU workstation or server | Comfortable | Comfortable |

For the sizing method, see [What size model fits your GPU?](https://docs.tokios.com/guides/model-fit.md). For the prompt lifecycle and why long context adds memory, see [KV cache and context](https://docs.tokios.com/concepts/kv-cache-and-context.md).

## Run it: beginner to advanced

> [tab] Ollama (easiest)
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> ollama run gpt-oss:20b
> ```
>
> Use `gpt-oss:120b` only on hardware with enough memory. Ollama publishes both tags and supports the model's native tool and reasoning features through its chat API.

> [tab] LM Studio (GUI)
>
> Search for **gpt-oss** in LM Studio's model catalog, choose the `20b` or `120b` build that fits your memory, and start the local server from the **Developer** tab. LM Studio's OpenAI-compatible API normally uses:
>
> ```text theme={"theme":{"light":"github-light","dark":"css-variables"}}
> http://127.0.0.1:1234/v1
> ```
>
> Use that URL as the Tokios upstream instead of Ollama's `11434` URL if LM Studio is the runtime.

> [tab] Unsloth Studio (no-code)
>
> Use Unsloth Studio when you want a GUI workflow around local open models, fine-tuning, or export. If your Studio build exposes an OpenAI-compatible inference endpoint, point Tokios at that `/v1` URL. Otherwise export or load the model through Ollama, LM Studio, or llama.cpp and connect Tokios to that runtime.

> [tab] vLLM (advanced)
>
> Hugging Face lists a direct vLLM serve command for both gpt-oss repos:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve openai/gpt-oss-20b
> ```
>
> For `120b`, swap the model id:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> vllm serve openai/gpt-oss-120b
> ```
>
> vLLM exposes an OpenAI-compatible API on `http://127.0.0.1:8000/v1` by default.

> [tab] SGLang (advanced)
>
> Hugging Face lists this SGLang launch shape:
>
> ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
> python3 -m sglang.launch_server \
>   --model-path openai/gpt-oss-20b \
>   --host 127.0.0.1 \
>   --port 30000
> ```
>
> For `120b`, use `--model-path openai/gpt-oss-120b`. Point Tokios at `http://127.0.0.1:30000/v1`.

## Capabilities

* **Reasoning:** gpt-oss supports configurable reasoning effort: `low`, `medium`, and `high`. Set the effort through the runtime's supported prompt or parameter mechanism.
* **Tool use:** gpt-oss was trained for the harmony response format and supports native function calling, web browsing, Python tool calls, and structured outputs. Use a runtime that applies the harmony format for you.
* **Context:** Both sizes support up to 128K context. Raise your runtime context setting if the default is lower.
* **Vision:** gpt-oss is text-only.
* **Fine-tuning:** OpenAI describes the family as fine-tunable. Use your runtime or training tool's current gpt-oss support before planning a fine-tune.

> [!WARNING]
>
> gpt-oss should be used with OpenAI's harmony response format. Ollama, Transformers chat templates, and other supported runtimes can apply it for you. If you change runtimes, verify harmony support before you rely on reasoning traces or tool calls.

## Connect it with Tokios

**1. Keep your local runtime running**

Start gpt-oss in Ollama, LM Studio, vLLM, SGLang, or another OpenAI-compatible runtime. Keep the runtime bound to loopback.

**2. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing**. Tokios shows a one-time **claim code** like `TKS-XXXX-XXXX`.

Run the installer on the machine that runs the model:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Already have `tokios-connector` installed? Pass the claim code directly:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
tokios-connector TKS-XXXX-XXXX
```

Back in the dashboard, **approve** the device on the **Setup** or **Connectors** tab.

**3. Point the connector at the runtime's /v1 API**

For Ollama, use:

```json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:11434/v1"
  }
}
```

For LM Studio, use `http://127.0.0.1:1234/v1`. For vLLM, use `http://127.0.0.1:8000/v1`. For SGLang, use the port you launched, such as `http://127.0.0.1:30000/v1`.

**4. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the upstream model and choose a **public name**, for example `gpt-oss-tunnel`. API clients send this deployment name, not `gpt-oss:20b` or `openai/gpt-oss-20b`, in the `model` field.

**5. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Copy the `sk-tok-...` API key when it appears.

## Use it from anywhere

```bash curl (OpenAI-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Explain why long context uses more memory."}]
  }'
```

```bash Claude Code (Anthropic-compatible) theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model gpt-oss-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name. Tokios forwards requests to your local runtime; it does not change gpt-oss's reasoning, context, or tool-calling behavior.

## Caveats

* `gpt-oss:120b` is a different hardware tier from `gpt-oss:20b`. Use `20b` first unless you have 80 GB-class GPU memory or large unified memory.
* The model is text-only. Pick a multimodal model if your client needs images.
* Tool calling depends on the runtime applying the harmony format correctly.
* Long context increases KV-cache memory. Test your real workload before promising a 128K deployment to teammates.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector and register your first deployment end to end.

> [card] [GPU-to-model fit](https://docs.tokios.com/guides/model-fit.md)
>
> Match gpt-oss sizes to your GPU or unified memory.

> [card] [Choose a model by task](https://docs.tokios.com/guides/model-by-task.md)
>
> Compare gpt-oss against other local models for chat, coding, and agent workloads.

> [card] [Quantization and hardware](https://docs.tokios.com/models/quantization-and-hardware.md)
>
> Check which formats run on which hardware.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.