Hermes Runtype OTel

What Hermes Runtype OTel does

Each Hermes turn becomes one trace made of an invoke_agent Hermes span for the whole turn, a chat span for each model request with model, token usage and finish reason, and an execute_tool span for each tool call. The plugin only observes Hermes through plugin hooks, changes no model calls or tools, depends on nothing beyond the Python standard library, and works in both the Hermes CLI and the gateway.

By default it sends identifiers, model names, token usage, timings and status, but no message or tool content. Setting RUNTYPE_OTEL_CAPTURE_CONTENT to true adds prompts, responses and tool payloads, limited to the last 16 messages per request and 4,096 characters per value, with best-effort removal of API keys and bearer tokens. Export is one request per turn with a 2-second timeout, and errors never fail the turn. Turn status is end_turn, max_turns, cancelled or error. Traces cover a single turn and exclude auxiliary model calls and subagent internals.

Key features

• One OpenTelemetry trace per Hermes turn with agent, chat and tool spans
• Token usage, model name, finish reason and timing on each model request
• Content capture is off by default and optional
• Standard-library only, with no extra dependencies
• Works in both the Hermes CLI and the gateway

When to use it

• Inspecting what an agent did on a specific turn, including every tool call
• Turning a real Hermes run into an eval case in Runtype
• Tracking token usage and latency per model request

Who it is for: Teams that run Hermes Agent and want observability and evaluation data in Runtype.

How it fits with Hermes Agent

It is a Hermes Agent plugin (tagged hermes-plugin) that observes runs through Hermes plugin hooks and does not alter model calls or tools.

How to install Hermes Runtype OTel

These commands are copied from the project's README. Check the repository for the latest steps before you run them.

hermes plugins install runtypelabs/hermes-runtype-otel --enable

Requirements: A Runtype agent ID (RUNTYPE_AGENT_ID) and a Runtype API key with the TELEMETRY:WRITE permission (RUNTYPE_OTEL_API_KEY)

Note: It sends traces to the Runtype service and stays inactive until RUNTYPE_AGENT_ID and RUNTYPE_OTEL_API_KEY are set.

FAQ