> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Trace your local model calls with Langfuse

> Wrap the OpenAI client with Langfuse's drop-in import and point it at your Tokios deployment to trace every call to your local model.

Langfuse ships a drop-in replacement for the OpenAI SDK — `from langfuse.openai import openai` — so you get full call tracing by importing that instead of `openai` and pointing `base_url` at `https://api.tokios.com/v1` with an `sk-tok-…` key. Because the wrapper mirrors the OpenAI client's interface, anything you already send to Tokios (chat completions, streaming, tool calls) gets traced with no change beyond the import and the base URL.

## Prerequisites

> [card] [Langfuse account or self-host](https://langfuse.com)
>
> Sign up for Langfuse Cloud or run a self-hosted instance, and grab your Langfuse public/secret key pair.

> [card] [Install the connector](https://docs.tokios.com/setup/connector-install.md)
>
> Pair `tokios-connector` with your local backend.

> [card] [Register a deployment](https://docs.tokios.com/setup/register-model.md)
>
> Choose the public name your `model` field will use.

> [card] [Create an API key](https://docs.tokios.com/setup/api-keys.md)
>
> Generate an `sk-tok-…` key, scoped to the deployments you need.

## Trace calls

**1. Install both SDKs**

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
pip install langfuse openai
```

**2. Set your Langfuse credentials**

Langfuse's OpenAI wrapper reads its credentials from the environment:

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
export LANGFUSE_PUBLIC_KEY="pk-lf-YOUR_PUBLIC_KEY"
export LANGFUSE_SECRET_KEY="sk-lf-YOUR_SECRET_KEY"
export LANGFUSE_HOST="https://cloud.langfuse.com"
```

> [!TIP]
>
> Set `LANGFUSE_HOST` to your self-hosted instance's URL if you're not using Langfuse Cloud.

**3. Import the wrapper and call your deployment**

Import `openai` from `langfuse.openai` instead of the plain `openai` package, then use it exactly like the standard OpenAI client, with `base_url` pointed at Tokios and `model` set to your registered deployment name:

```python langfuse_tokios.py theme={"theme":{"light":"github-light","dark":"css-variables"}}
from langfuse.openai import openai

client = openai.OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="gemma-tunnel",
    messages=[{"role": "user", "content": "Summarize this code: print('hello')"}],
)

print(response.choices[0].message.content)
```

Every call through this client is traced in Langfuse — no `@observe` decorator or manual span required for the call itself.

> [!NOTE]
>
> `model` is always the Tokios **deployment** name you registered in the [Models](https://tokios.com/console/models) tab, never your backend's local model id, and the base URL must include `/v1`.

## Cost tracking for local models

Langfuse calculates cost from a model's per-token price, which it knows for hosted providers like OpenAI and Anthropic. A Tokios deployment fronts your own local model, so Langfuse has no built-in price for it — traces still capture prompts, completions, latency, and token counts, but cost may show as zero or missing until you tell Langfuse how to price that model.

> [!WARNING]
>
> To fix this, define a custom model in Langfuse's model settings that matches on the deployment name you send as `model`, and set a price per token — including a \$0 price if you just want usage tracked without a dollar cost. The exact steps can change between Langfuse releases, so check Langfuse's own model-pricing documentation for the current flow before relying on cost figures.

## Now trace it from anywhere

The tracing setup above works the same whether the client calling Tokios sits on your laptop or on a teammate's machine — the connector still only dials out from your local backend. Once a teammate or a deployed agent has a scoped `sk-tok-…` key, their calls through the same `langfuse.openai` wrapper land in the same Langfuse project.

> [card] [Compatible clients](https://docs.tokios.com/guides/compatible-clients.md)
>
> See the full list of OpenAI- and Anthropic-compatible clients Tokios works with.

> [card] [API keys](https://docs.tokios.com/setup/api-keys.md)
>
> Create a separate scoped key per teammate or agent.

> [card] [Request logging](https://docs.tokios.com/setup/request-logging.md)
>
> Only need to read what was sent? Turn on request logging in the dashboard instead of running a tracing stack.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.