State of the art of telemetry and agent observability, and which event sources feed heartbeat.
Researched 2026-09-18 in session 4f0a68f9-1435-4150-bb9e-84567dc7db02 (harness-exploration-status). Heartbeat itself: Heartbeat. The ACP frontend for other agents: ACP.
The OpenTelemetry (OTel) span vocabulary for generative AI: invoke_agent → chat → execute_tool, with attributes gen_ai.request.model, gen_ai.usage.input_tokens/output_tokens, gen_ai.response.finish_reasons.
| Part | Status |
|---|---|
gen_ai.client spans (one per LLM round trip) | Stable since early 2026. |
Agent spans (invoke_agent, execute_tool) | Experimental. |
| Conventions | Moved to a dedicated repo, v1.42.0 (June 2026); no 1.0 committed (OTel blog). |
Shadow the attribute names in heartbeat's event schema; do not pin them.
This setup's state.
| Feature | State |
|---|---|
| OTel logs and metrics | Wired to the peek control port: CLAUDE_CODE_ENABLE_TELEMETRY=1, OTLP endpoint 127.0.0.1:{{PEEK_CONTROL_PORT}}/otlp. OTEL_LOG_TOOL_DETAILS=1 tags real skill, agent and MCP names on cost and token metrics. |
| Distributed traces | NOT enabled: OTEL_TRACES_EXPORTER=otlp and CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 are unset. Traces would add the interaction → llm_request → tool → hook span tree, with W3C traceparent propagated into subagents, Bash and MCP. |
/skill-doctor | Built in, v2.1.252+: per-skill context cost, 7-day tokens, invocation counts. Blind to routine and headless sessions: it marked all smine-* skills "never invoked" though they run nightly. Own telemetry attributes them correctly; the planned per-skill execution-time and token estimation exceeds it. |
Transcript JSONL, hook events and the existing telemetry are event-complete as heartbeat's feed. Traces add only precise timing and automatic cross-process correlation, both reconstructible from the transcript tree peek already walks: an optional later addition if correlation gets painful.
The Claude Code native frontend.
| Role | Sources |
|---|---|
| Sensors | Transcript JSONL (lossless, per session); hook events (PreToolUse, PostToolUse, Stop, SubagentStart/Stop, PreModelSwitch, TaskCreated/TaskCompleted, WorktreeCreate, TeammateIdle); OTel logs and metrics via peek. The hook surface grows release by release, so the Claude Code changelog is a dependency feed. |
| Actuators | PreToolUse permissionDecision: deny / updatedInput; Stop/SubagentStop exit-2 continue; PreModelSwitch exit-2 block. Hooks are per-session, stateless and in-process. The engine (cross-session run state, skill-model awareness, Z3 judgment) is heartbeat's own layer; no first-party counterpart exists. |
The transcript JSONL format is unversioned and undocumented; pin it with the same discipline as peek versions.