feat(streaming): emit OTel metrics for ttft, tps, token counts

[x]

Metrics

Eight OTel metrics in agentex.lib.core.observability. Histograms are streaming-only (need per-chunk visibility); counters are provider-agnostic and fire from RunHooks.on_llm_end.

Metric	Type	Source
agentex.llm.ttft	histogram (ms)	streaming model — request → first content token
agentex.llm.ttat	histogram (ms)	streaming model — request → first answering token (excludes reasoning chunks)
agentex.llm.tps	histogram (tok/s)	streaming model — over generation window (first_token_at→last_token_at)
agentex.llm.requests	counter (with status)	LLMMetricsHooks
agentex.llm.input_tokens	counter	LLMMetricsHooks
agentex.llm.output_tokens	counter	LLMMetricsHooks
agentex.llm.cached_input_tokens	counter	LLMMetricsHooks
agentex.llm.reasoning_tokens	counter	LLMMetricsHooks

status values: success / rate_limit / server_error / client_error / timeout / network_error / other_error.

Why both layers

Hooks fire on every Runner.run / Runner.run_streamed call — async-Temporal and sync ACP both pick them up. TemporalStreamingHooks inherits LLMMetricsHooks so existing async users get the counters automatically. Sync ACP opts in: Runner.run_streamed(..., hooks=LLMMetricsHooks()).

The streaming model owns ttft/ttat/tps because hooks can't see individual chunks.

Cardinality

model on histograms. model × status on the request counter. No task-id, no per-call labels.

Robustness

on_llm_end reaches into nested optional fields some providers don't populate (litellm/Anthropic). Defensive structure:

• requests emits outside the usage-extraction try — success counter fires even if the response shape is unusual.
• Token-counter block has its own try/except. ... or 0 handles None.
• record_llm_failure (used in error path) has its own try/except — exporter failures never shadow the original LLM exception.

Tests cover: missing .usage, missing input_tokens_details, all-None token values.

Tested

• 20 unit tests in core/observability/tests/. All passing.
• Local sync ACP agent hitting SGP /v5/ (litellm-equivalent) via OpenAIChatCompletionsModel — 3 messages, no crashes.
• aimi-scale -k 2 eval pinned to this commit — 2/2 examples, metrics visible in Mimir.

Notes

stream_start_perf is bookmarked before responses.create() so ttft/ttat include network RTT.

If upstream OpenAI Agents SDK ever adds metrics hooks on Model, this is a clean swap-out.

Greptile Summary

• Adds eight OTel metrics (ttft, ttat, tps histograms; requests, input_tokens, output_tokens, cached_input_tokens, reasoning_tokens counters) via a lazy singleton (get_llm_metrics()), wired into LLMMetricsHooks (new RunHooks subclass) and TemporalStreamingModel for streaming-only histograms. TemporalStreamingHooks now inherits LLMMetricsHooks so existing Temporal users pick up counters automatically.
• stream_start_perf is correctly placed before responses.create() in the updated code, and instruments are lazily created so a MeterProvider configured after import still binds.
• Several robustness concerns flagged in the prior review remain open: the token-counter block shares a single try/except causing silent cascade drops, the str(agent.model) fallback can produce memory-address cardinality when agent.model is a Model instance, and the streaming-metrics emit block is unwrapped — an OTel exporter exception there would be swallowed by the outer except handler.

Confidence Score: 4/5

Safe to merge with awareness that several P1s from the prior review round are still open; no new blocking issues introduced.

The prior review surfaced multiple P1 findings (single try/except cascade drop, str(agent.model) cardinality, unwrapped OTel emit block). Those conversations remain unresolved in the thread history, holding the score at 4.

src/agentex/lib/core/observability/llm_metrics_hooks.py (token counter cascade + cardinality), src/agentex/lib/core/temporal/plugins/openai_agents/models/temporal_streaming_model.py (unwrapped OTel emit and record_llm_failure ordering).

Important Files Changed

Filename	Overview
src/agentex/lib/core/observability/llm_metrics.py	New file — defines lazy LLMMetrics singleton with 8 OTel instruments and classify_status helper. Clean design, lazy init avoids no-op provider binding, bounded cardinality.
src/agentex/lib/core/observability/llm_metrics_hooks.py	New RunHooks adapter — token counters share a single try/except block so a failure on cached_tokens silently drops reasoning_tokens; str(agent.model) can produce memory-address cardinality when model is a Model instance (both flagged in prior review).
src/agentex/lib/core/observability/tests/test_llm_metrics_hooks.py	Good coverage of normal, zero-token, and failure cases; missing positive assertion for output_tokens in test_missing_token_details_skips_those_counters.
src/agentex/lib/core/temporal/plugins/openai_agents/hooks/hooks.py	TemporalStreamingHooks now inherits LLMMetricsHooks instead of RunHooks directly — clean change that adds metric emission to existing Temporal streaming users.
src/agentex/lib/core/temporal/plugins/openai_agents/models/temporal_streaming_model.py	Adds ttft/ttat/tps emission after streaming loop; stream_start_perf correctly placed before responses.create(). Prior-review issues remain: unwrapped OTel emit block, record_llm_failure ordering.

Sequence Diagram

sequenceDiagram
    participant SDK as OpenAI Agents SDK
    participant TSM as TemporalStreamingModel
    participant OTel as OTel (get_llm_metrics)
    participant Hooks as LLMMetricsHooks.on_llm_end

    SDK->>TSM: get_response()
    TSM->>TSM: "stream_start_perf = perf_counter()"
    TSM->>TSM: await responses.create()
    loop Each stream event
        TSM->>TSM: Record first_token_at / last_token_at / first_answer_at
    end
    TSM->>OTel: ttft_ms.record()
    TSM->>OTel: ttat_ms.record()
    TSM->>OTel: tps.record()
    TSM-->>SDK: ModelResponse
    SDK->>Hooks: on_llm_end(agent, response)
    Hooks->>OTel: "requests.add(1, status=success)"
    Hooks->>OTel: input_tokens.add()
    Hooks->>OTel: output_tokens.add()
    Hooks->>OTel: cached_input_tokens.add()
    Hooks->>OTel: reasoning_tokens.add()
    note over TSM,OTel: Error path
    TSM-xTSM: Exception in streaming loop
    TSM->>OTel: record_llm_failure(model, exc)
    TSM-->>SDK: raise typed LLM exception

Loading

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
src/agentex/lib/core/observability/tests/test_llm_metrics_hooks.py:165-170
The test documents that `input_tokens` is emitted before the failing `input_tokens_details` access, but doesn't assert that `output_tokens` is also emitted. Since `m.output_tokens.add(...)` runs at line 45 of `llm_metrics_hooks.py` — before the AttributeError on `input_tokens_details` — it should be emitted here too. Without the assertion this behaviour is unverified, and a future reordering of those lines could silently break `output_tokens` reporting.

```suggestion
        # input_tokens.add and output_tokens.add fire before the nested attribute access.
        m.input_tokens.add.assert_called_once_with(100, {"model": "gpt-5"})
        m.output_tokens.add.assert_called_once_with(50, {"model": "gpt-5"})
        # cached_input_tokens / reasoning_tokens skipped — the AttributeError
        # bailed before they could be called.
        m.cached_input_tokens.add.assert_not_called()
        m.reasoning_tokens.add.assert_not_called()
```

_{Reviews (8): Last reviewed commit: "fix: add @override on LLMMetricsHooks.on..." | Re-trigger Greptile}