> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Turn your NVIDIA DGX Spark into a private model endpoint

> Give the models on your NVIDIA DGX Spark or GB10 one OpenAI- and Anthropic-compatible endpoint that your laptop, coding agents, and teammates can reach.

Your DGX Spark already runs the model fine. The hard part is using it from somewhere else without exposing the box to the internet or hand-building a gateway. Run the Tokios connector next to your runtime and you get one authenticated endpoint at `https://api.tokios.com` that speaks both OpenAI and Anthropic request formats, so your laptop, Claude Code, and anyone you invite can reach the Spark without touching firewall rules.

> [!TIP]
>
> Ready to connect your Spark? Jump to [the quickstart](https://docs.tokios.com/quickstart.md) for the full connector setup, or skip to [register a model](https://docs.tokios.com/setup/register-model.md) if you already have a runtime listening.

## The gap after "it runs"

DGX Spark and GB10 owners typically get a model running with Ollama, llama.cpp, vLLM, or Lemonade Server, then hit a second problem: using that model from anywhere other than the Spark itself.

* **Plumbing.** Getting one OpenAI-compatible endpoint out of a multi-model setup often means wiring together LiteLLM, `llama-swap`, and the runtime itself, plus a polling daemon to keep them in sync — components some DGX Spark community projects describe as the explicit goal, and the setup as "plumbing" they'd rather skip.
* **Remote access.** The common paths to reach the Spark from another machine — binding the runtime to `0.0.0.0` (`snap set ollama host="0.0.0.0:11434"`), an SSH tunnel, or Tailscale — each carry their own fiddly setup on ARM64 / DGX OS, and a raw bound port has no per-client auth in front of it.
* **API shape.** Claude Code speaks the Anthropic Messages format; local runtimes speak OpenAI Chat Completions. Bridging the two usually means writing and maintaining a small proxy.

Tokios is the connect layer that sits on top of the runtime you already run — it does not replace Ollama, llama.cpp, vLLM, or Lemonade Server.

> [!NOTE]
>
> 2 DGX Sparks provide enough aggregate memory to shard `DeepSeek-V4-Flash-0731`, but DeepSeek-specific cluster recipes remain community-tested and require 0731 revalidation. Keep the connector on the head node's `/v1` endpoint. See [DeepSeek V4 Flash local hardware](https://docs.tokios.com/blog/deepseek-v4-flash-local-hardware.md) for the maturity labels and setup boundary.

## Can a DGX Spark cluster run Kimi K3?

No. Kimi K3's 2.8T parameters (\~50B active per token through MoE) need roughly **1.4 TB** at their native 4-bit floor, while NVIDIA's documented Spark clusters top out at 4 nodes and 512 GB — and Moonshot targets K3 at supernodes with 64 or more accelerators. Even 8 DGX Sparks (1 TB pooled) fall **376 GB short** of just the raw weight floor.

For Spark today, choose a model within NVIDIA's documented 120B single-node, 400B two-node, or 700B four-node ranges — see [DGX Spark model compatibility](https://docs.tokios.com/guides/dgx-spark-model-compatibility.md) for the full list, or [what to run instead of Kimi K3](https://docs.tokios.com/models/kimi-k3-alternatives.md) for the best pick per Spark configuration. For the complete K3 memory calculation and cluster math, see the [Kimi K3 hardware deep dive](https://docs.tokios.com/blog/kimi-k3-hardware-deep-dive.md).

## Set it up on your Spark

**1. Start your runtime on the Spark**

Run your model server as you normally would, for example:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run llama3.1
```

or

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
vllm serve meta-llama/Llama-3.1-8B-Instruct
```

**Lemonade Server** is an alternative that handles CUDA automatically — it detects your DGX Spark's GPU and selects the right backend without manual flags:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
lemonade-server serve --model your-model
```

Lemonade Server listens on port `13305` by default and exposes an OpenAI-compatible API at `/v1`. See [lemonade-server.ai](https://lemonade-server.ai/) for install instructions.

Confirm it answers on its own OpenAI-compatible `/v1` API before adding the connector.

**2. Get a claim code and pair the connector**

In the [dashboard](https://tokios.com/console), open the **Setup** tab and select **Start pairing** to get a one-time claim code shaped `TKS-XXXX-XXXX` (valid for about 15 minutes). On the Spark, run:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Tokios ships a Linux arm64 build of `tokios-connector`, which covers DGX Spark's Grace CPU. The exact systemd integration and packaging steps can vary by DGX OS version — check yours before scripting the install.

Back in the dashboard, approve the device on the **Setup** or **Connectors** tab. The connector's token is delivered over the tunnel — you never paste it into a file.

**3. Point the connector at your runtime's /v1 endpoint**

The connector's upstream is the runtime's own OpenAI-compatible API on the Spark, not a port you expose externally. Default ports: Ollama `11434`, vLLM `8000`, Lemonade Server `13305` (all with `/v1` appended). No inbound firewall rule is needed — the connector only dials out.

**4. Register a deployment**

On the **Models** tab, map your upstream model id (for example `llama3.1`) to a public deployment name such as `spark-llama`. Clients send this deployment name in the `model` field, not the upstream id.

**5. Create an API key**

On the **Keys** tab, create a key. Keys are `sk-tok-…` Bearer tokens, shown once, and can be scoped to specific deployment name patterns.

## Use it from anywhere

Once the connector is paired and a deployment is registered, point your client at Tokios instead of the Spark's IP address.

```bash Claude Code theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model spark-llama
```

```python OpenAI SDK / Codex theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="spark-llama",
    messages=[{"role": "user", "content": "Hello from my laptop"}],
)
```

> [!NOTE]
>
> Claude Code's model picker only lists `claude*` ids, so your Tokios deployment won't appear in the list — pass `--model spark-llama` explicitly.

If you run more than one model on the Spark, or split traffic across Ollama, llama.cpp, vLLM, and Lemonade Server at once, use the [connector config file](https://docs.tokios.com/reference/connector-config.md) to define multiple routes and upstreams instead of the single-upstream pairing flow. See [Model routing](https://docs.tokios.com/guides/model-routing.md) for how to register and switch between several deployments behind the same endpoint.

## Why one endpoint

* **Swap models without touching client config.** Point a deployment name at a different upstream model or backend, and every agent and script that already targets `spark-llama` keeps working.
* **Scoped keys for teammates.** Give a teammate a key limited to specific deployment patterns instead of sharing raw access to the Spark.
* **No inbound ports on the Spark.** The connector only makes outbound connections, so you don't bind the runtime to `0.0.0.0` or maintain an SSH tunnel or Tailscale node just to reach it from a laptop.

## Next steps

> [card] [Run Qwen3.6-35B on DGX Spark](https://docs.tokios.com/guides/run-qwen36-35b-a3b-on-dgx-spark.md)
>
> Step-by-step deployment guide for the sweet-spot model on Spark.

> [card] [Run DeepSeek V4.1 Flash on 2 Sparks](https://docs.tokios.com/guides/run-deepseek-v41-flash-on-2-dgx-sparks.md)
>
> Deploy the 763B MoE model on a 2-Spark cluster.

> [card] [Team access](https://docs.tokios.com/use-cases/team-access.md)
>
> Full walkthrough for giving your team API access to your local models.

> [card] [What is a local LLM?](https://docs.tokios.com/guides/what-is-local-llm.md)
>
> Complete guide to running large language models locally.

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Go from zero to a working connector, deployment, and API key.

> [card] [Install the connector](https://docs.tokios.com/setup/connector-install.md)
>
> Full connector install and pairing reference for all platforms.

> [card] [Use Claude Code](https://docs.tokios.com/guides/claude-code.md)
>
> Configure `ANTHROPIC_BASE_URL` and `ANTHROPIC_API_KEY` in detail.

> [card] [Connector config reference](https://docs.tokios.com/reference/connector-config.md)
>
> Serve multiple models from one Spark with `tokios-connector.json`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.