This is a working ML observability setup for a classifier: log every prediction keyed by request ID and model version, compute output-score PSI per version against a pinned training reference, push it to Prometheus, and alert on drift and on a stale drift job. If you build one thing first, build the prediction log.
The setup exists because models fail without paging anyone. The pod returns HTTP 200 at normal p99 latency while the model is confidently wrong for a slice of traffic nobody watches; Rabanser, Günnemann and Lipton put it as ML systems that “tend to fail silently.” Labels that would prove it arrive weeks later, if ever.
What ML observability is, and the serving, data, output and quality layers it spans, is defined in ML observability: architecture and signals; this post assumes that model and builds the output layer. Where checking ends and investigating begins is covered in ML observability vs monitoring, and LLM and RAG systems in how to monitor LLMs in production.
The metric that matters: prediction drift per model version
If you alert on one ML-specific signal, make it drift in the model’s own output distribution, computed per model version against a pinned reference. For a classifier, that is the predicted-probability histogram. For a regressor, the prediction itself.
The usual statistic is the Population Stability Index: PSI = Σ (aᵢ − eᵢ) · ln(aᵢ / eᵢ), where eᵢ and aᵢ are the shares of reference and current-window predictions in bin i. Bin edges come from reference quantiles and are frozen with the reference. Evidently’s drift configuration docs list psi with a default threshold of 0.1, flagging drift at or above it. Treat that as a starting point and tune it against your own history.
Why it beats the two obvious alternatives:
- Realized accuracy is what you actually care about, and it is gated by label latency. A fraud model whose chargebacks resolve weeks later cannot alert on accuracy until the losses are booked.
- Drift on every input feature fires one alarm per feature that moved, most of which the model barely weights. The score distribution only moves when an input shift reaches the decision.
The research agrees: in the Rabanser et al. experiments, two-sample testing on pre-trained classifier outputs, used for dimensionality reduction, performed best. Your production model can play that role for free.
Once probabilities are calibrated, add estimated performance. NannyML’s CBPE turns confidence scores into an expected confusion matrix, with stated assumptions: well-calibrated probabilities, no shift into unseen regions of the input space, and no concept drift. The statistics behind drift tests are compared in model drift detection.
Wiring it up
Three pieces: emit at serve time, compute on a schedule, alert in Prometheus.
Emit. A low-cardinality histogram feeds the live chart; the full record goes to the prediction log for the label join and the drift job. In production, make the sink an async buffer, not a write in the request path.
import json
import time
import uuid
from prometheus_client import Counter, Histogram
PRED_SCORE = Histogram(
"model_prediction_score",
"Predicted probability of the positive class",
["model_name", "model_version"],
buckets=[i / 10 for i in range(1, 11)],
)
FEATURE_NULLS = Counter(
"model_feature_null_total",
"Null values seen per input feature",
["model_name", "feature"],
)
def predict(model, features: dict, sink) -> float:
request_id = str(uuid.uuid4())
for name, value in features.items():
if value is None:
FEATURE_NULLS.labels(model.name, name).inc()
score = float(model.score(features))
PRED_SCORE.labels(model.name, model.version).observe(score)
sink.write(json.dumps({
"request_id": request_id,
"ts": time.time(),
"model_name": model.name,
"model_version": model.version,
"features": features,
"score": score,
}) + "\n")
return score
Compute. An hourly job reads the window from the log, computes PSI against the frozen reference, and pushes three gauges. A batch job’s outcome is the use case the Pushgateway guidance describes.
import numpy as np
from prometheus_client import CollectorRegistry, Gauge, push_to_gateway
def psi(reference: np.ndarray, current: np.ndarray, bins: int = 10) -> float:
edges = np.unique(np.quantile(reference, np.linspace(0, 1, bins + 1)))
edges[0], edges[-1] = -np.inf, np.inf
e = np.histogram(reference, edges)[0] / len(reference)
a = np.histogram(current, edges)[0] / len(current)
e, a = np.clip(e, 1e-6, None), np.clip(a, 1e-6, None)
return float(np.sum((a - e) * np.log(a / e)))
def run(model_name: str, model_version: str, reference_scores, current_scores):
registry = CollectorRegistry()
Gauge("model_prediction_psi", "Score PSI vs pinned reference",
registry=registry).set(psi(reference_scores, current_scores))
Gauge("model_drift_window_rows", "Rows in the current window",
registry=registry).set(len(current_scores))
Gauge("model_drift_job_last_success_unixtime", "Last successful drift run",
registry=registry).set_to_current_time()
push_to_gateway(
"pushgateway:9091",
job="model-drift",
registry=registry,
grouping_key={"model_name": model_name, "model_version": model_version},
)
For humans, Evidently’s ValueDrift(column="score", method="psi") renders the same statistic as an HTML report.
Alert. Scrape the Pushgateway with honor_labels: true so grouping-key labels survive, then write Prometheus alerting rules. Thresholds are placeholders to tune.
groups:
- name: ml-observability
rules:
- alert: ModelPredictionDrift
expr: model_prediction_psi >= 0.1 and model_drift_window_rows > 1000
for: 2h
labels:
severity: ticket
annotations:
summary: "Score drift on {{ $labels.model_name }} {{ $labels.model_version }}"
- alert: DriftJobStale
expr: time() - model_drift_job_last_success_unixtime > 3 * 3600
labels:
severity: page
- alert: FeatureNullRateSpike
expr: |
sum by (model_name, feature) (rate(model_feature_null_total[15m]))
/ on (model_name) group_left
sum by (model_name) (rate(model_prediction_score_count[15m])) > 0.05
for: 15m
labels:
severity: page
Drift goes to a ticket queue: it says behaviour changed, not that users were hurt. A dead job and a null spike are unambiguous breakage, so they page. Tiering is covered in alerting for ML model drift.
What you’ll see
Healthy. The score histogram keeps its shape per version. PSI sits low and noisy, with a daily or weekly rhythm. Null counters hover near zero.
Step change on the new version only, aligned with a deploy. The model changed. Compare canary and baseline model_version series side by side; this is why version belongs in every record.
Step change on every version, no deploy. Upstream data. Check null-rate and freshness panels before touching the model.
Slow ramp over weeks. Genuine population shift. Check realized performance on the labeled slice before retraining.
A perfectly flat PSI line. The job died. The Pushgateway “never forgets series pushed to it,” so the last value reads as stability forever. That is what DriftJobStale catches.
Panel layout is in ML model monitoring dashboard; triage order is in debugging model accuracy drops in production.
Caveats
Binning and volume false alarms. PSI depends on bin edges and sample size, and quiet overnight windows are noisy. The minimum-rows guard and for: duration exist for this. A reference that misses a full weekly cycle will fire on weekday traffic every week.
Output drift is blind to concept drift. If the input-to-outcome relationship changes while inputs and scores stay put, PSI stays flat. Only labels or scored evaluations catch that; CBPE assumes it away.
Cardinality blowup. The Prometheus naming guide warns that every label combination is a new time series and that unbounded values such as user IDs do not belong in labels. Request IDs and raw feature values go in the log. model_version is fine only if old versions are retired.
Sampling cost. Full feature vectors at high QPS add up. Sample routine traffic, but keep every record for canary versions and for the slice that will receive labels.
Feedback loops and biased labels. The model decides which labels exist: a declined loan never produces a repayment outcome. Sculley et al. list hidden feedback loops among ML’s system-level risks. Join labels by request ID, never by time window, and treat accuracy on a self-selected slice as biased.
All of this runs on open-source parts; the full assembly is in the open-source ML observability stack.
FAQ
Can you monitor a model without ground-truth labels?
Yes, partially. Prediction-score drift, input data quality and feature drift need no labels (SentryML’s ML monitoring taxonomy groups these signals), and confidence-based estimators such as NannyML’s CBPE approximate accuracy from calibrated probabilities. What you cannot catch without labels or scored evaluations is concept drift, where the relationship between inputs and outcomes changes while the inputs themselves look unchanged.
Which open-source tools are used for ML observability?
A common open-source stack is Prometheus and Grafana for serving metrics and alerts, Evidently or NannyML for drift and estimated performance, MLflow for linking a deployed model version to its training run, and OpenTelemetry with Arize Phoenix for tracing LLM and RAG pipelines. Commercial platforms such as Arize and Fiddler package the same layers.
Related across the network
- Model Monitoring in Production: Metrics and Triggers — sentryml.com
- Population Stability Index (PSI): Formula and Thresholds — mlmonitoring.report