> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Use Tokios with local AI workstations

> Expose models on NVIDIA GB10, DGX Spark, AMD Strix Halo, Ryzen AI Max, Apple silicon Macs, and other local AI workstations through one Tokios API endpoint.

Local AI workstations are powerful, but most clients still expect a cloud-style API endpoint. Tokios lets you keep the model on your own hardware while giving your tools a stable OpenAI- or Anthropic-compatible endpoint at `https://api.tokios.com`.

Use this page if you run local models on hardware such as NVIDIA GB10 or DGX Spark systems, AMD Strix Halo or Ryzen AI Max systems, Apple silicon Macs with high unified memory, GPU workstations, mini PCs, or homelab servers.

## What Tokios adds

Tokios does not replace your local model runtime. It sits between your clients and the backend you already run on the workstation.

| You have | Tokios gives you |
| - | - |
| A model running on a local AI workstation | A public deployment name for API clients |
| Ollama, LM Studio, llama.cpp, vLLM, or another OpenAI-compatible backend | A stable endpoint at `https://api.tokios.com` |
| A machine behind NAT, a home router, or a private office network | An outbound-only connector tunnel |
| Several local models or backends | One API key and one endpoint for clients |
| Coding agents or SDK apps | OpenAI- and Anthropic-compatible request surfaces |

## Hardware this page is for

Tokios is useful when the machine has enough memory and compute to run local models, but the tools you use live somewhere else: a laptop, editor, coding agent, notebook, server process, or teammate's client.

| Hardware category | Common local AI reason | Tokios role |
| - | - | - |
| NVIDIA GB10 and DGX Spark systems | Run local AI development workloads on Grace Blackwell hardware | Give remote clients one stable API endpoint |
| AMD Strix Halo and Ryzen AI Max systems | Use high-memory desktop or mini-PC hardware for local inference | Expose the backend without opening inbound ports |
| Apple silicon Macs with high unified memory | Run local LLMs on a MacBook Pro, Mac Studio, or Mac mini | Connect Mac-hosted models to OpenAI- and Anthropic-compatible clients |
| GPU workstations and homelab servers | Serve models from a dedicated local machine | Route multiple deployments through one connector |

## Recommended setup

**1. Start the local backend**

Run your model server on the workstation. Tokios works with backends that expose an OpenAI-compatible Chat Completions API, such as Ollama, LM Studio, llama.cpp, or vLLM.

**2. Install the connector on the workstation**

Run `tokios-connector` on the same machine, or on a nearby machine that can reach the backend. The connector dials out to Tokios, so you do not need to open inbound ports.

**3. Register a deployment**

Map the backend's upstream model id to a public deployment name such as `strix-coder`, `gb10-chat`, or `local-qwen`. Clients send this deployment name in the `model` field.

**4. Point your tools at Tokios**

Use `https://api.tokios.com/v1` for OpenAI-style clients and `https://api.tokios.com` for Anthropic-style clients.

## Hardware-oriented workflows

| Workflow | Why it fits local AI hardware | Tokios page |
| - | - | - |
| Coding agents on your own machine | Keep code context local while using familiar agent clients | [Coding agents](https://docs.tokios.com/use-cases/coding-agents.md) |
| Compare models on the same workstation | Change only the deployment name and keep hardware fixed | [Test your deployment](https://docs.tokios.com/guides/benchmark-your-deployment.md) |
| Try several local runtimes | Route Ollama, LM Studio, llama.cpp, or vLLM behind one endpoint | [Backends](https://docs.tokios.com/setup/supported-backends.md) |
| Share one workstation endpoint with your own apps | Use API keys and deployment names instead of per-app backend URLs | [Compatible clients](https://docs.tokios.com/guides/compatible-clients.md) |
| Move models between machines later | Keep the public deployment name stable while changing the backend route | [Model routing](https://docs.tokios.com/guides/model-routing.md) |

## Naming deployments

Use names that describe the job, not only the hardware. That makes client configs easier to keep stable.

| Avoid | Prefer | Why |
| - | - | - |
| `gb10` | `local-coder` | The client cares about the workflow |
| `strix-halo-qwen` | `fast-draft` | You can move the deployment later |
| `ollama-model-1` | `chat-local` | The backend is an implementation detail |

> [!TIP]
>
> If you upgrade from one workstation to another, keep the same deployment name and update the connector route. Your clients can keep using the same `model` value.

## What to benchmark

Local AI workstation buyers often compare hardware by model size or raw tokens per second. For Tokios, measure the workflow you actually plan to run.

| Signal | Why it matters |
| - | - |
| Time to first token | Shows whether the setup feels interactive |
| Streaming stability | Matters for coding agents and chat UIs |
| Total task time | Captures prompt processing, generation, and client behavior |
| Model quality | Determines whether the local model can do the job |
| Backend reliability | Catches restarts, context limits, and driver/runtime issues |

Use [Test your deployment](https://docs.tokios.com/guides/benchmark-your-deployment.md) to compare deployments with the same prompt and hardware.

## Next steps

> [card] [Connect a backend](https://docs.tokios.com/setup/supported-backends.md)
>
> Configure Ollama, LM Studio, llama.cpp, or vLLM as your local upstream.

> [card] [Install the connector](https://docs.tokios.com/setup/connector-install.md)
>
> Run the connector beside your workstation model server.

> [card] [Use coding agents](https://docs.tokios.com/use-cases/coding-agents.md)
>
> Connect Claude Code, OpenAI Codex, and compatible coding clients.

> [card] [Test your deployment](https://docs.tokios.com/guides/benchmark-your-deployment.md)
>
> Compare local models and backends through the same Tokios endpoint.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.