Monitoring the System in Production

The characteristic failure here is silence: the system keeps running, produces plausible output, and stops being right. Nothing alerts.

What to instrument

Detection counts per camera per hour, which is the cheapest signal and catches most gross failures.

Confidence score distribution. Scores drifting toward the threshold mean the model is less certain than it was.

Alert rate, trended, in both directions.

Image statistics: brightness, contrast, sharpness. A dirty lens shows here before it shows in detections.

Frame delivery: dropped frames, camera offline, latency.

Override and correction rate from reviewers, which is the closest thing to a live accuracy signal.

Processing time per frame, which drifting upward means the system is falling behind.

What each failure looks like

Camera offline: counts drop to zero. Obvious, and only if someone is watching.

Lens dirty or misaligned: sharpness falls, counts drop gradually, confidence falls.

Scene changed: counts step-change without a corresponding change in activity.

Model degraded after an update: confidence distribution shifts across all cameras at once, on a date.

Threshold changed by someone: alert rate steps, with no other signal moving.

Being worked around: detections fall while the underlying event rate, measured another way, does not.

Alerting on the monitoring

Alert on change, not on level, because every camera has its own baseline.

Camera offline beyond a short period.

Counts departing from that camera's normal range for the hour.

Confidence distribution shifting.

Processing falling behind.

Not on individual detections, which is the system's normal output and which produces the noise that makes people ignore the channel.

The outcome measure

The check that catches everything the above misses.

Track the real-world outcome the system exists to affect: incidents, defects escaping, complaints, whatever it is.

Detections falling with the outcome measure flat is the gaming or degradation signal, and it is the only way to catch a system that has quietly stopped working while producing a reassuring chart.

Report the two together, always.

Versioning and change control

Record which model, which threshold, which camera configuration, from which date.

Every change, including threshold adjustments made informally.

Without this you cannot attribute a change in behaviour to anything, and you cannot answer what the system was doing on the day of an incident.

Treat threshold changes as changes, with a record and a reason, not as tuning.

Ownership

A named person, with allocated time, or the monitoring dashboards join the alerts in being ignored.

A monthly look at the trends and a quarterly evaluation against ground truth.

Reported to whoever relies on the system, who are currently assuming it works at the number they were given at installation.

A system nobody owns is providing false assurance, which is worse than no system because decisions are being made on the assumption it works.

The commissioning baseline

Everything the monitoring compares against, captured once, at installation.

Detection counts per camera per hour, by day of week.

Confidence score distribution.

Image statistics: brightness, contrast, sharpness.

A reference frame per camera.

The evaluation result, with conditions.

The configuration: model version, threshold, zones.

Without a baseline, drift is invisible, because there is nothing to compare against and every value looks normal.

The false assurance failure

The worst outcome in this field and the quietest.

A degraded system keeps producing plausible output.

Nobody notices, because there is no alert for being wrong.

Decisions continue on the assumption that something is watching.

Which is worse than no system, because the absence of a control would have been managed.

Report the evaluation result to whoever relies on it, including when it is worse than at installation, because they are currently assuming the installation number still holds.