Quick start

Start with one application. Changing its gateway URL and API key is enough to monitor supported LLM calls, including calls made by its schedulers. Add feature labels later if you want to distinguish the work inside that application.

1. Create a connection

1. Open Settings → LLM providers. Add your real provider API key and check the available models. This key stays on the server.
2. Open Applications → Application LLM calls and create an LLM connection for your application, for example AgentTrade. Use separate connections for separate applications or environments.
3. Copy the Dispatcher connection key and the gateway URL shown in setup. Save the key in your application's secret environment, not in source code.

Setting	What to use
Base URL	Your copied gateway URL, ending in /llm
API key	The Dispatcher connection key, not your OpenAI/Anthropic provider key
Model	Your application's existing model, provided it is configured and accessible through the gateway

The connection name identifies the application; it is not a model name. Model restrictions are optional. If you set a connection's default provider/model, Dispatcher overrides the model requested by your app. Leave that unset to keep the app's own model choice.

2. Change the URL and key in your app

Choose the example matching your existing SDK. Replace the example values below with your copied URL, connection key and configured model. LLM_MODEL is just an example environment variable: keep your existing model configuration if you already have one.

OpenAI-compatible SDK

Set these variables in your application's environment, then restart the process:

OPENAI_BASE_URL=https://your-dispatcher-host/llm
OPENAI_API_KEY=your-dispatcher-connection-key
LLM_MODEL=your-configured-model

import os
from openai import OpenAI

client = OpenAI()  # Reads OPENAI_BASE_URL and OPENAI_API_KEY.
response = client.chat.completions.create(
    model=os.environ["LLM_MODEL"],
    messages=[{"role": "user", "content": "Reply with OK."}],
)
print(response.choices[0].message.content)

Native Anthropic SDK

Use the same gateway URL and the connection key authorized for this application:

ANTHROPIC_BASE_URL=https://your-dispatcher-host/llm
ANTHROPIC_API_KEY=your-dispatcher-connection-key
LLM_MODEL=your-configured-Claude-model

import os
from anthropic import Anthropic

client = Anthropic()  # Reads ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY.
response = client.messages.create(
    model=os.environ["LLM_MODEL"],
    max_tokens=64,
    messages=[{"role": "user", "content": "Reply with OK."}],
)
print(response.content)

No Dispatcher SDK is required for these LLM calls. Both SDKs use /llm; existing OpenAI setups using /llm/v1 still work. Do not append /v1 to the Anthropic base URL. Grok, Gemini and DeepSeek models can also use the OpenAI-compatible route when configured on the gateway. Their native SDK protocols are not exposed by this release.

Check it: send one request, then open Overview and filter by your application. Check its provider/model, tokens and estimated cost. These examples call the real provider and may incur a small charge. A key identifies the application automatically; it cannot identify each end user or the purpose of a call by itself.

3. Optional: distinguish features inside an application

For example, AgentTrade might use scheduler.classify for a background job and analysis.summary for a user query. You only need one label: feature. You do not need to repeat the application name or add project or api_id.

Add this single keyword argument inside your existing OpenAI call:

extra_body={"metadata": {"dispatcher_context": {"feature": "scheduler.classify"}}},

For an existing native Anthropic call, use this version instead:

extra_body={"litellm_metadata": {"dispatcher_context": {"feature": "scheduler.classify"}}},

These lines belong inside client.chat.completions.create(...) or client.messages.create(...); they are not standalone scripts. If the call already has an extra_body, merge this entry into it. For Anthropic, use litellm_metadata, not its provider-specific metadata field.

Read a service's feature from an environment variable

DISPATCHER_FEATURE=scheduler.classify

In your application's shared LLM helper, read the variable:

feature = os.environ["DISPATCHER_FEATURE"]

Then replace the literal "scheduler.classify" in the appropriate keyword above with feature. Setting the environment variable alone does not send a label: your code must read it and attach it to each call. A per-call feature can replace this service default when one service performs several kinds of work. Calls without a feature still appear under the application.

Or use your existing YAML configuration

Example llm.yaml for an OpenAI-compatible client:

llm:
  base_url_env: OPENAI_BASE_URL
  api_key_env: OPENAI_API_KEY
  model: your-configured-model
usage:
  feature: scheduler.classify

Your application loads this file; Dispatcher does not read your local YAML. Using PyYAML (pip install pyyaml), a complete example is:

import os
import yaml
from openai import OpenAI

with open("llm.yaml", encoding="utf-8") as file:
    config = yaml.safe_load(file)

# A service environment value takes precedence over this example's YAML default.
feature = os.getenv("DISPATCHER_FEATURE") or config["usage"]["feature"]
client = OpenAI(
    base_url=os.environ[config["llm"]["base_url_env"]],
    api_key=os.environ[config["llm"]["api_key_env"]],
)
response = client.chat.completions.create(
    model=config["llm"]["model"],
    messages=[{"role": "user", "content": "Reply with OK."}],
    extra_body={"metadata": {"dispatcher_context": {"feature": feature}}},
)

For Anthropic, use the ANTHROPIC_* variable names in YAML and the native SDK call above with litellm_metadata. Keep your existing response limit.

Check it: make a new call and open Overview → Application usage. When feature labels are present, the initial view groups by Application + Feature. Use Group by on the left to select one or more dimensions; both charts update immediately. You can combine dimensions, for example Feature + Provider. Expand Recent activity to see the Feature column. Open a request's Details for the aligned application, token, cost and attribution fields. Older calls cannot acquire labels retroactively. Use short, stable labels (at most 100 characters), without secrets or message text. Optional user_id and session_id support finer attribution; a reported user ID is not proof of that person's identity.

4. Optional: protect sensitive input (PII)

PII means personally identifiable information, such as an email address. Protection is configured per application/environment; you do not add another key or SDK to the app.

1. Open Data protection, select the application/environment, expand Protection services and select Check services. The installation administrator must first configure Presidio and OPA; GLiNER is an optional additional detector.
2. Add an EMAIL_ADDRESS → Mask rule.
3. Open Test draft policy, enter the synthetic text Contact alice@example.com and select Test policy. Expect Contact <EMAIL_ADDRESS>.
4. Select Enable data protection and Save policy to activate it. Draft testing alone does not activate protection and does not call an LLM.
5. Send a new text-only request through the gateway and review the protection decision history. All connection keys for this application/environment share the saved policy.

Action	Effect
Observe	Record detected entity types/counts; send the original text
Mask	Replace matched values before sending the request to the model
Block	Stop the request before it reaches the model

Presidio detects/masks values; OPA evaluates the application rules; optional GLiNER extends detection. If any rule blocks, the whole request is blocked. Protection checks supported input text, including system text and streaming requests. It does not scan model responses, direct provider calls or collector account snapshots. Protected apps currently reject images/files, tool calls/definitions, embeddings and server-stored conversation context. Protection-service failures also block protected requests.

Decision history stores actions and entity types/counts, not prompts or matched values. Test representative data before rollout: detection is not a guarantee that every sensitive value will be found. Ask your installation administrator about protection-service configuration and decision-history retention.

Connect Codex, Claude Code and other supported accounts

1. In Applications → CLI account usage, create a separate named connection for each developer’s device and download its JSON configuration and the installer for that device's OS.
2. Install one collector on the device and import the configuration. It contains the backend URL and sync key; treat the file as a secret. One collector/key handles multiple supported tools on the same device.
3. In the collector's Tools tab, enable Codex and Claude Code, then start collection. After installing another supported CLI, use Refresh tools and enable its reader. New readers are not automatically enabled. Stop collection before changing tools.
4. Check Applications → CLI account usage → Device tools for the latest reported status. Refresh status reads the latest report; enable/disable changes happen in the collector on the device. Upgrade older collectors to 0.3.3 for running/stopped status.

In Overview → CLI account usage, each device shows its name, who registered it, last heartbeat and status. Search by device, developer or account. This section refreshes automatically every 30 seconds and has its own search; application chart filters do not hide CLI account snapshots.

• Running: Collector is sending heartbeats, even between account checks.
• Stopped: Collector reported an intentional stop or normal window close.
• Offline / not reporting: no heartbeat for two minutes by default (sleep, network loss or a terminated process). This does not prove the app was intentionally stopped.
• Not connected yet: a named device key exists but has never reported.
• Status unavailable: an older collector sent tool information but no heartbeat.

In Collector, ▶ Running stays visible on both tabs. Start collection is disabled while busy; the red ■ Stop button ends collection. Minimizing keeps it running; closing stops it. Last reported account usage remains visible, with its measurement time and a stale indicator when old. A running collector does not guarantee every provider account can be read: check Tools for sign-in or permission problems.

Use one device key per machine, shared by its Codex, Claude and other enabled readers. Do not reuse one key across developers’ devices: reports would share the same identity. Workspace owners/admins see all active registered devices; other members need the appropriate application visibility grant. “Registered by” is the key creator, which may be the administrator; it is not proof of the person currently using that machine.

You keep using your existing CLIs. Collection reports account usage/limits, not individual messages or a complete session/request ledger. Availability depends on the OS, supported reader and local sign-in. Install on each participating device; one installation cannot discover every employee's CLI. The macOS preview is not Developer ID signed or notarized; if macOS blocks it, follow System Settings → Privacy & Security → Open Anyway for the downloaded installer. Follow Apple’s instructions if the installer is blocked.

Review the results

• Filter application request history by application, provider, model or person. Feature labels separate scheduled and interactive work within the same application.
• Compare trends over 24H, 7D, 30D or all history, with spending versus budget.
• Each chart has its own Measure: Requests, Known cost or Reported tokens. For cost, select one currency; currencies are never added together. Reported tokens are input plus output, without adding reasoning/cache tokens a second time.
• Use Compare · right axis on the trend to compare another measure, for example tokens on the left and cost on the right. Solid lines use the left axis; dashed lines use the right. The two scales are independent. Choose None to remove comparison.
• Click › at the pie's top right, or ‹ at the trend's top left, to expand that chart across the available row and hide the other. Click again to restore both.
• Measure, currency, comparison and chart-layout choices are remembered for your user and workspace in this browser. Changing report filters does not reset them. Grouping is selected in the left panel; there is no duplicate drag/drop strip above the charts.
• Application usage and CLI account usage have separate expandable headers, both open initially. Usage data table and Recent activity start closed below the charts. The table pages through every group matching the report filters, including any combined as Other groups in the charts. Cost currencies/evidence remain separate.
• Reported LLM tokens show input, output, cache read/write and reasoning together on desktop. Reasoning includes a reporting count: zero is different from Not reported, and not every provider/model returns a separate reasoning measurement.
• In Recent activity, choose 20, 50 or 100 rows per page. The default display window is 7 days; change it or leave days blank for all history within the report filters. This hides older rows from the list; it does not delete database records or change chart totals. Preferences are saved per user/workspace in that browser.
• Edit a provider's balance checkpoint in the header. Remaining balance subtracts recorded spending and may be negative; it is not a live provider-balance sync.

Gateway calls measure LLM work that actually runs. To detect a scheduler that never starts, or measure ordinary APIs, add job/API reporting.