> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Tunnels, VPNs, and LLM streaming: what breaks and why

> Cloudflare Tunnel, ngrok, and Tailscale have documented SSE buffering, timeout, and bandwidth limits — Tokios is purpose-built for streaming LLM APIs instead.

A local LLM API needs two things most general-purpose tunnels don't optimize for: **token streaming** (Server-Sent Events that flush as each token is generated, not buffered into one response) and **long-lived connections** (a generation can legitimately run past a proxy's default timeout). Cloudflare Tunnel, ngrok, and Tailscale are solid tools for the traffic they're designed around — but each documents constraints that surface specifically on this workload. Tokios's connector is outbound-only like a tunnel, but built around streaming LLM traffic from the start: no read/write timeout to configure around, and keys scoped to your models instead of raw ports.

## The two things that break

### Streaming gets buffered

An LLM completion streamed over SSE sends partial tokens as soon as they're generated — the client expects a steady drip, not one chunk at the end. Some reverse proxies buffer response bodies by default to optimize throughput for typical web traffic, which is the wrong behavior for SSE: the client sits waiting, then gets the whole response at once (or times out first).

![Top bar shows a buffered response that stays empty then fills all at once; bottom bar shows a streamed response filling steadily token by token.](https://mintcdn.com/tokios/4cPOecaYHMdZ-9wS/images/diagrams/sse-buffering.svg?fit=max\&auto=format\&n=4cPOecaYHMdZ-9wS\&q=85\&s=63191d46e4d0a8bb92b003fab362f845)

Buffered: the client waits, then the whole answer dumps at once. Streamed: tokens flush as they're generated.

### Requests run longer than the timeout allows

A large model on modest hardware can take well past the ordinary web-request budget to finish a long generation. Any proxy or tunnel with a fixed read/write timeout will cut the connection before the model is done, and the client sees a gateway error instead of a completion — even though the model was still working.

## Comparison

| Approach | What it solves | Documented limitation |
| - | - | - |
| Port forwarding | Direct inbound access to a home server | Requires router access and a public IP; structurally impossible behind carrier-grade NAT (CGNAT), which many residential ISPs use and which you cannot reconfigure yourself — a common reason a tunnel is needed at all |
| Cloudflare Tunnel | Exposes a local server publicly without opening inbound ports | Per Cloudflare's own documentation, proxied HTTP traffic is buffered by default unless the origin sends `Content-Type: text/event-stream`; the proxy read timeout is 120 seconds and write timeout 30 seconds on non-Enterprise plans, so long generations can end in a 524 error |
| ngrok (free plan) | Quick, temporary public URL for a local server | Per ngrok's own documentation, the free plan caps usage at 1 GB of data transfer per month, 20,000 HTTP requests per month, and 3 endpoints, assigns one auto-generated dev domain, and shows an interstitial browser-warning page that visitors must click through (or callers must send a bypass header) |
| Tailscale (tailnet) | Private device-to-device access across NAT and CGNAT for devices you control | Per Tailscale's own documentation, every caller must join your tailnet — it's built for your own devices, not for arbitrary public API clients |
| Tailscale Funnel | Tailscale's feature for exposing a tailnet service on a public URL | Per Tailscale's own documentation, Funnel listens only on ports `443`, `8443`, or `10000`, serves only `*.ts.net` names, and has "non-configurable bandwidth limits" |
| Tokios | Outbound-only connector purpose-built for streaming LLM APIs | SSE streaming is correct on every API surface, there are no inbound ports to open, and access is scoped with `sk-tok-…` keys instead of raw network reachability. No ceiling on request duration or throughput is claimed here |

> [!NOTE]
>
> Every limit above is the vendor's own documented behavior for its general-purpose use case, not a failure — Cloudflare Tunnel, ngrok, and Tailscale all do what they're designed to do well. The friction shows up specifically when the workload is a streaming LLM API with generations that can run long.

## Where Tokios fits

Tokios isn't a replacement for Tailscale or Cloudflare Tunnel in general. Keep using Tailscale for SSH into a home server, or Cloudflare Tunnel for a low-traffic web app — those are exactly the jobs they're built for. Tokios is specifically the connect layer for one path: getting a coding agent or API client to a **streaming LLM endpoint** running on your own hardware, without buffering SSE or racing a proxy timeout.

The `tokios-connector` dials out over a single outbound WebSocket, the same direction-flipping idea behind a tunnel — see [How it works](https://docs.tokios.com/how-it-works.md) for the full request flow. From there, every request authenticates with an `sk-tok-…` API key scoped to specific deployments, rather than a raw port or hostname being reachable at all.

## Next steps

> [card] [Quickstart](https://docs.tokios.com/quickstart.md)
>
> Pair a connector, register a model, and send your first streaming request.

> [card] [How Tokios routes requests](https://docs.tokios.com/how-it-works.md)
>
> See the outbound-only architecture and request flow in detail.

> [!NOTE]
>
> For a detailed breakdown of Cloudflare's SSE buffering behavior and the \~100–120s proxy timeout on non-Enterprise plans, see [Tokios vs Cloudflare Tunnel](https://docs.tokios.com/guides/tokios-vs-cloudflare-tunnel.md) — it covers the specific failure modes and when the edge is (and isn't) the right fit for streaming inference.

> [!TIP]
>
> Serving Ollama specifically? See [Ollama remote access](https://docs.tokios.com/guides/ollama-remote-access.md) for backend-specific setup notes.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.