#llm-ops
-
LLM Cost & Latency Observability with OpenTelemetry
Implement LLM cost and latency observability with OpenTelemetry: token accounting, versioned price estimates, streaming timing, and request metrics.
-
Online Evaluation: Closing the Eval-Prod Gap
Offline eval scores are green and production is worse. The gap is structural, not measurement error, and online evaluation is how you instrument it.
-
End-to-End Tracing for LLM Apps: Span Design Guide
Implement end-to-end tracing for LLM applications with request spans, context propagation, retrieval evidence, and sampling checks for incident analysis.