> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Connect AnythingLLM and Dify to your local model

> Add Tokios as a custom OpenAI-compatible provider in AnythingLLM or Dify so either app can call your local model from anywhere — no Docker workarounds.

Both AnythingLLM and Dify let you add a custom OpenAI-compatible model provider. Point that provider at `https://api.tokios.com/v1` with your `sk-tok-…` API key and your registered **deployment** name, and either app calls your local model — no `host.docker.internal`, no port forwarding, no VPN.

## Prerequisites

> [card] [Install and pair the connector](https://docs.tokios.com/setup/connector-install.md)
>
> Run `tokios-connector` next to your model and approve it in the console.

> [card] [Register a deployment](https://docs.tokios.com/setup/register-model.md)
>
> Give your model a public name, for example `gemma-tunnel` — this is the `model` value clients send.

> [card] [Create an API key](https://docs.tokios.com/setup/api-keys.md)
>
> Create an `sk-tok-…` key on the **Keys** tab, optionally scoped to this deployment.

> [card] [Check your backend is supported](https://docs.tokios.com/setup/supported-backends.md)
>
> Ollama, llama.cpp, vLLM, and LM Studio all work behind the connector.

> [!NOTE]
>
> Tokios implements `GET /v1/models` and normalizes the OpenAI API surface, so when AnythingLLM or Dify validates the provider (listing models, checking the endpoint), that check succeeds the same way it would against OpenAI itself.

## Add Tokios as a provider

> [tab] AnythingLLM
>
> **1. Open LLM provider settings**
>
> In AnythingLLM, go to **Settings** → **LLM Provider** and choose **Generic OpenAI** from the provider list. The exact menu path and provider label can vary by release — check yours.
>
> **2. Set the base URL**
>
> In **Base URL**, enter:
>
> ```text theme={"theme":{"light":"github-light","dark":"css-variables"}}
> https://api.tokios.com/v1
> ```
>
> **3. Set the API key**
>
> In **API Key**, paste your Tokios key:
>
> ```text theme={"theme":{"light":"github-light","dark":"css-variables"}}
> sk-tok-YOUR_KEY
> ```
>
> **4. Set the model name**
>
> In **Model name** (also called **Chat Model Name** in some AnythingLLM versions), enter your Tokios deployment name, for example `gemma-tunnel` — not the upstream model id your backend uses locally.
>
> **5. Set token / context window**
>
> Set the **Token context window** field to match your model's context length — the exact field label varies by version, and some versions require this value before they'll save the provider.
>
> **6. Save and test**
>
> Save the provider, then send a message in a workspace to confirm AnythingLLM reaches your model through Tokios.

> [tab] Dify
>
> **1. Open model provider settings**
>
> In Dify, go to **Settings** → **Model Provider** and add the **OpenAI-API-compatible** provider. The exact menu path and provider label can vary by release — check yours.
>
> **2. Set the model name**
>
> In **Model Name**, enter your Tokios deployment name, for example `gemma-tunnel`.
>
> **3. Set the API key**
>
> In **API Key**, paste your Tokios key:
>
> ```text theme={"theme":{"light":"github-light","dark":"css-variables"}}
> sk-tok-YOUR_KEY
> ```
>
> **4. Set the API endpoint URL**
>
> In **API endpoint URL**, enter the full `/v1` base:
>
> ```text theme={"theme":{"light":"github-light","dark":"css-variables"}}
> https://api.tokios.com/v1
> ```
>
> **5. Set the completion mode**
>
> Set **Completion mode** to match how your model responds (chat vs. completion) — the default value and exact label can vary by release, so check yours.
>
> **6. Save and test**
>
> Save the provider, then select your deployment in an app or workflow to confirm Dify reaches your model through Tokios.

## No more `host.docker.internal`

AnythingLLM and Dify both commonly run in Docker, and your model often runs on a different machine — or a different container — than the app. That split is where most self-hosted setups get stuck: `localhost` inside a container doesn't reach the host, `host.docker.internal` doesn't resolve the same way on every platform, and a model on a separate machine needs its own port forwarding or VPN entry.

Pointing AnythingLLM or Dify at `https://api.tokios.com/v1` removes that problem entirely. The endpoint is public and stable regardless of which container, host, or network the app runs on — only the connector next to your model needs outbound access, and it never opens an inbound port.

## Now that it's connected

Your model is reachable from AnythingLLM or Dify wherever they run. The same deployment and API key work from any other OpenAI- or Anthropic-compatible client too.

> [card] [Compatible clients](https://docs.tokios.com/guides/compatible-clients.md)
>
> See the full list of clients Tokios works with and which base URL each one needs.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create a separate scoped key per app or teammate, and revoke access without touching your connector.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.