What It Cannot Tell You
Questions routinely asked of a vision system that it cannot answer, and what to use for each instead.
Reference
A system that answers everything asked of it is answering some things badly. Defending the boundary is what keeps it trusted on the rest.
Why someone did something
Why not: the footage shows an action, not a reason.
Use instead: asking them. The account frequently explains something the video makes look inexplicable.
What someone was feeling or intending
Why not: contested science, no ground truth, and prohibited in several contexts.
Use instead: nothing from video. The underlying concern is usually a management or a safety question with its own answer.
Whether an action was authorised
Why not: the system sees the action, not the permission.
Use instead: the access or work-authorisation record, joined by time.
What happened outside the frame
Why not: the field of view is a boundary, and absence of detection is not absence of event.
Use instead: the mapped blind zones, stated honestly, and other evidence.
Who a person is, from a system that does not identify
Why not: and this is the good news. A well-designed counting system genuinely cannot answer it.
Use instead: access control with a credential the person presents, if identity is genuinely needed.
Whether the system is still accurate
Why not: detection counts and confidence scores are proxies. Only ground truth measures accuracy.
Use instead: periodic evaluation against an independent record.
Whether an alert is correct
Why not: a detection is a classification with a threshold behind it, and a probability is not a fact.
Use instead: human review with the surrounding context, before any consequence.
How to decline
Say what the system does support, which is usually adjacent and substantial.
Say what would answer the question and roughly what it would cost.
Give a range where a range is honest.
Write the limitation into the report, not the covering email, because the report is what circulates and the caveat is dropped by the second retelling.
Do not produce a confident answer you cannot support. In this field the confident wrong answer results in someone being approached, accused or refused, which is why the boundary matters more here than in most measurement disciplines.
What it does support
Whether a defined condition was present, in view, at a time.
How many, where, for how long, in aggregate.
Whether an object meets a specification.
Where events cluster by location and hour.
A searchable record of what a camera saw.
Five things answered reliably is a system worth having, and defending the boundary is what keeps those five trusted.
Writing the caveat where it survives
A limitation stated in conversation is dropped by the second retelling.
Put it in the report, in the section people read, not in the covering email.
Attach it to the number itself: the threshold, the conditions, the date of evaluation, the sample size.
State what was not tested.
Repeat it in any summary, because summaries circulate and originals do not.
In this field a confident wrong answer results in someone being approached or refused, which is why the caveat is not a formality.
Refusing to produce a number
Sometimes the right answer is that the question cannot be answered from this system.
Say what it does support, which is usually adjacent.
Say what would answer the question and roughly what it would cost.
Offer the alternative source: access records, process data, asking people.
Do not produce a weak figure with a caveat, because in this field the figure results in someone being approached and the caveat does not travel with it.
A refusal that is explained is accepted; a number that turns out to be wrong is remembered.
More in this section