Performance Drifts

Vision systems are not installed and finished. The world in front of the camera changes, and performance falls without any alert.

What causes it

The camera moves. Vibration, cleaning, someone knocking it. A few degrees changes the geometry the system was tuned for.

The lens degrades. Dust, spiderwebs, condensation, scratches, UV yellowing.

The scene changes. New racking, repainted walls, a new machine, different signage, seasonal displays.

Lighting changes. Replaced fittings, a new window blind, a tree that grew, the sun angle in December.

The population changes. Different uniforms, new vehicle models, different packaging.

The process changes, so the thing being detected no longer looks the way it did.

Detecting it without ground truth

Collecting ground truth continuously is impractical. These proxies are cheap and catch most drift.

Detection rate over time. A steady count that suddenly halves is not a change in the world.

Confidence score distribution. Scores clustering nearer the threshold indicate the model is less certain than it was.

Alert rate per camera, trended. Both directions matter.

Image statistics: mean brightness, contrast, sharpness. A blurred lens shows up here before it shows up in detections.

Override and correction rate, where operators can mark an alert wrong.

Alert on change, not on level, since every camera has its own baseline.

Periodic ground truth

The proxies detect change; only ground truth measures accuracy.

Take a sample: a defined period, reviewed by a person recording what actually happened, compared against system output.

Quarterly for anything consequential, annually at minimum.

Cover the hard conditions, not a convenient sunny afternoon.

Record the result so the trend across evaluations exists.

The seasonal problem

Sun angle, daylight hours and weather change the input substantially across a year.

A system evaluated in June has not been evaluated for December.

Establish per-season baselines rather than one, or the annual variation will read as drift and the real drift will hide inside it.

Maintenance that actually matters

Clean the lenses, on a schedule. This is the highest-return maintenance action and it is nobody's job by default.

Check camera alignment against a reference image taken at installation.

Re-run the evaluation after any scene change — a refit, new equipment, changed lighting.

Version the model and record what is deployed where, or you cannot attribute a change in behaviour to anything.

The organisational failure

Systems are commissioned by a project and then owned by nobody.

Six months later performance has fallen, nobody has measured it, and the alerts are being ignored — which means the system is providing false assurance rather than a control.

Assign an owner at commissioning, with the periodic evaluation as a scheduled task rather than an intention.

Report the evaluation result to whoever relies on the system, because they are the ones currently assuming it works.

The reference image

The cheapest drift check available and it takes a minute per camera.

Capture a reference frame at commissioning, in defined lighting, and store it.

Compare periodically: has the view moved, has the scene changed, is the image softer?

A few degrees of rotation changes the geometry the detection was tuned for, and it is invisible in the alert stream.

Automate the comparison where possible, alerting on structural change.

Re-baseline deliberately after any intentional change, with a record, so the next comparison is meaningful.

Cleaning the lenses

The highest-return maintenance action in this field and nobody's job by default.

Dust, spiderwebs, condensation and film degrade the image gradually, which is why nobody notices.

Sharpness statistics catch it before detections fall enough to be visible.

Put it on a schedule with a named owner, alongside the reference-frame comparison.

Record when each camera was last cleaned.

A quarter of the performance complaints in a mature deployment are answered by a cloth, which is unglamorous and true.