> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Use an Xinference server from anywhere — safely

> Reach your Xinference OpenAI-compatible server from other machines, agents, or teammates without exposing its port — using an outbound-only connector.

Xinference (Xorbits Inference) serves models behind an OpenAI-compatible API on your local machine — by default that API only answers on localhost, so it isn't reachable from another machine unless you bind it wider or forward a port. Keep Xinference on localhost and run the Tokios connector next to it instead: the connector dials out to `https://api.tokios.com`, and your tools, teammates, or agents call one authenticated endpoint from anywhere.

> [!NOTE]
>
> **Advanced runtime.** Xinference is a model-serving framework aimed at more involved setups. If you just want the easiest way to serve a model, start with [Ollama](https://docs.tokios.com/guides/ollama-remote-access.md) or [LM Studio](https://docs.tokios.com/guides/lm-studio-remote-access.md).

> [!NOTE]
>
> Xinference exposes an OpenAI-compatible Chat Completions API, so it works with Tokios like any OpenAI-compatible server: the connector forwards to whatever `/v1` `BaseUrl` you set, regardless of which server implements that surface. Ollama, llama.cpp, vLLM, and LM Studio are the backends Tokios documents most closely, but anything OpenAI-compatible — including Xinference — works the same way.

## The outbound-only way

**1. Run Xinference locally, as usual**

Start the local Xinference server — for example with `xinference-local` — and launch a model through its UI or CLI. The exact startup command and flags can vary by version, so check yours.

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
xinference-local
```

Xinference commonly serves its OpenAI-compatible API on port `9997` — confirm the default port for your install. Keep it bound to `127.0.0.1` rather than `0.0.0.0`, and don't forward the port on your router.

**2. Pair the Tokios connector**

Sign in at [tokios.com/console](https://tokios.com/console), open the **Setup** tab, and click **Start pairing**. Tokios shows a one-time **claim code** like `TKS-XXXX-XXXX`, valid for about 15 minutes.

On the machine running Xinference, run the install one-liner for your OS:

```powershell Windows theme={"theme":{"light":"github-light","dark":"css-variables"}}
irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'
```

```bash macOS theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX
```

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Already have `tokios-connector` installed? Pass the code directly instead:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
tokios-connector TKS-XXXX-XXXX
```

Back in the console, **approve** the device on the **Setup** or **Connectors** tab. The connector's token is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

**3. Point the connector at Xinference's /v1 API**

For the basic single-model path, the console's Setup wizard asks for your upstream base URL — enter Xinference's OpenAI-compatible endpoint, `http://127.0.0.1:<port>/v1` (confirm the port against your install). If you're serving multiple models through a `tokios-connector.json` config, set the upstream `BaseUrl` explicitly:

```json theme={"theme":{"light":"github-light","dark":"css-variables"}}
{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:9997/v1"
  }
}
```

Confirm `9997` against your running Xinference instance before using it. See [Connector config](https://docs.tokios.com/reference/connector-config.md) for the full multi-model file format.

**4. Register a deployment**

In the [Models](https://tokios.com/console/models) tab, register the Xinference model and choose a **public name** — for example, `xinference-tunnel`. That public name, not the model id you launched in Xinference, is what clients send in the `model` field.

**5. Create an API key**

In the [Keys](https://tokios.com/console/keys) tab, click **Create key**. Optionally scope it to specific deployments with model-name patterns (comma-separated globs, `*` for all). Copy the `sk-tok-…` key now — it's shown only once.

## Use it from anywhere

Once you have a deployment name and an API key, any OpenAI- or Anthropic-compatible client can reach your Xinference model — from another machine, a teammate's laptop, or a hosted agent.

```bash curl theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "xinference-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

```python Python (OpenAI SDK) theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="xinference-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")
```

```bash Claude Code theme={"theme":{"light":"github-light","dark":"css-variables"}}
export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model xinference-tunnel
```

> [!TIP]
>
> The `model` field is always your registered **deployment** name (`xinference-tunnel` above), never the model id you launched in Xinference. See [Use Claude Code](https://docs.tokios.com/guides/claude-code.md) for persisting the environment variables across sessions.

## FAQ

> [accordion] Is Xinference officially supported?
>
> Ollama, llama.cpp, vLLM, and LM Studio are the backends Tokios documents most closely. But the connector works against any server exposing an OpenAI-compatible Chat Completions API at a `/v1` base URL, and Xinference does that, so it works as a generic OpenAI-compatible upstream the same way. Confirm behavior against your own Xinference version before relying on it in production.

> [accordion] Does streaming work?
>
> Yes, if the underlying Xinference model and server support streaming responses. Set `"stream": true` and Tokios returns standard server-sent events (SSE) in OpenAI or Anthropic format, depending on which API surface you called.

> [accordion] Can I serve more than one Xinference model?
>
> Yes. Register additional deployments in the [Models](https://tokios.com/console/models) tab, or use a `tokios-connector.json` config file to define multiple upstreams and routes for a single connector — useful if you launch more than one model in Xinference. See [Connector config](https://docs.tokios.com/reference/connector-config.md).

> [accordion] Can I share access with teammates without exposing Xinference directly?
>
> Yes. Create a separate `sk-tok-…` key per teammate in the [Keys](https://tokios.com/console/keys) tab, and scope each key to only the deployments they need. Revoking a key stops it from authenticating immediately, so you can cut off access without touching Xinference or the connector.

## Your model never leaves your machine — your endpoint works everywhere

Xinference keeps doing exactly what it does today, on localhost, with no inbound ports and no endpoints exposed beyond `/v1`. The connector only dials out. Everything past that — auth, routing, streaming — is one endpoint you can call from any machine, teammate, or agent you choose to give a key to.

> [card] [Supported backends](https://docs.tokios.com/setup/supported-backends.md)
>
> See which local runtimes Tokios explicitly tests and supports today.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create, scope, and rotate `sk-tok-…` keys for teammates and agents.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.