Evaluating and Monitoring Models

Two kinds of checking

Offline evaluation is checking a model on past examples with known labels, before any real user sees its answers. The check is fast, safe and repeatable. Its weakness is that the past is not the future.

An online experiment is giving the new model to a small share of live traffic and comparing real outcomes against the current model. The evidence is real. The risk is real too, which is why the share starts small.

A model earns release in two stages. It must pass the past-data check first. Then it must hold up on live traffic.

Offline evaluation in depth

The check uses examples the model never trained on. Say the model answers 900 of 1,000 held-back examples correctly.

900 / 1,000 x 100 = 90, so the model scores 90 percent on past data.

One average score hides weak groups. Scores are also cut by category, by user group and by item age. A model that fails one group badly is not ready, whatever its average says.

Doing well on new examples, not just memorised ones, is the point of the held-back set. A model that only repeats its training data fails here. That honesty is what makes the check worth running.

Online experiments

The worked experiment runs on a feed with 1,000,000,000 reads a day. The test arm takes 1 percent of those reads.

1,000,000,000 / 100 = 10,000,000 reads in the test arm.

The current model keeps the rest. 1,000,000,000 - 10,000,000 = 990,000,000 reads.

Safety limits are set before the experiment starts. If harm measures cross a stated line, the experiment stops and traffic returns to the current model. One change runs at a time, so the cause of any shift is clear.

Monitoring live models

Monitoring watches three streams. The inputs show whether incoming data still looks like training data. The predictions show whether the mix of answers is shifting. The outcomes show real quality, once labels arrive.

Outcomes often arrive late. A purchase, a complaint or a repayment may take days to be known. Input and prediction watches give the early warning while outcomes catch up.

Latency and cost are watched beside quality. A model that grows slower or dearer is failing the system even when its answers stay good.

Model drift

Model drift is the slow loss of quality that follows when live data changes away from the data the model learned from. The model does not break. The world moves.

Behaviour changes with seasons, prices and fashion. New items arrive with no history. Each change makes the training data a worse portrait of the present.

The response is retraining on newer data, then the same two-stage check as any release. Drift is expected and budgeted for. A design with no drift plan has a quality plan that expires.

Feedback loops

A feedback loop is a cycle where a model's own predictions shape the examples it later learns from. The model then trains on a world partly of its own making.

Items the model shows gather clicks and purchases. Items it never shows gather nothing, however good they are. The next training set then over-represents the shown items, and the model grows more certain of its own past choices.

The countermeasure is deliberate variety. A small share of traffic sees less certain items, gathering honest evidence about them. The evidence keeps the training data from narrowing to the model's habits.

Evaluation and trade-offs

Past-data checks are safe and fast, and they cannot see live behaviour. Live experiments see real behaviour, and they expose real users to a model that may be worse. The two together cover each other's blind spots.

A single average score is never the whole verdict. Group-level scores, harm measures and cost all sit beside it. A release decision reads the whole panel.

Evaluation is a permanent part of the system, not a gate passed once. Every retrain, every data change and every season reopens the question of whether the model still earns its place.

The worked experiment