Reading a Vendor Claim

The vocabulary in this sector is unusually loose. Translating it is most of the evaluation.

"99 percent accurate"

Usually means: accuracy on a curated test set at an unstated threshold, possibly the training distribution.

Ask: recall and precision separately, at what threshold, on what set, with what composition.

And: false alarms per camera per day in a deployment like ours.

"AI-powered"

Means a learned model is involved somewhere. It says nothing about capability.

Ask: what specifically is learned, and what is rule-based? For many measurement tasks a rule is better, and a learned model where a measurement would do adds opacity for no benefit.

"Privacy-preserving"

Sometimes means edge processing with frames discarded, which is genuine.

Sometimes means faces are blurred in the interface while templates are computed and stored, which is not.

Ask: what is computed, what is transmitted, what is stored, and is a biometric template created at any point?

"Anonymised"

Rarely accurate. A tracking record without a name is usually still personal data if the person can be identified by other means.

Ask: what specifically is removed, at what point, and can a person be re-identified from what remains?

"Demographic analytics"

Means biometric categorisation — inferring age and sex from appearance.

Ask why it is needed, and note the prohibition on inferring sensitive attributes, the questionable accuracy, and that the aggregate question it answers can usually be answered by counting.

"Behavioural analytics"

Covers everything from dwell time to intent inference.

Ask: which specific behaviours, defined how? If the answer includes suspicion, intent or emotion, apply the boundary section.

"Real-time alerting"

Means the system can emit an alert quickly. It says nothing about whether anyone can act on the volume.

Ask: the alert rate at our camera count, and do the arithmetic in the room.

"Trained on millions of images"

Says nothing about relevance. Millions of images of a different environment does not help with yours.

Ask: trained on what, and how does it perform on our footage in a pilot?

"Proven in deployment"

Ask for a reference site with comparable conditions, and ask them directly about the false alarm rate and whether anyone still reads the alerts.

Reference calls are the most informative twenty minutes available and are frequently skipped.

"Compliant"

Compliance is a property of a deployment, not of a product. A vendor cannot make your deployment lawful.

Ask: which risk tier under the applicable regulation, what documentation is provided, and what obligations remain with us? A provider who cannot answer has not done the work.

The general test

Any claim without a number, a threshold and a test set is a description of a hope.

Any claim you cannot check on your own site during a pilot should be treated as untested rather than as false — and untested is enough reason to structure the pilot around checking it.

Getting it in writing

The difference between a sales answer and a commitment.

Ask the questions by email, not in a meeting.

Ask for the answers to be attached to the contract.

Where a claim matters — no template computed, no training on customer data, processing in a stated location — it belongs in a term rather than a slide.

A vendor happy to say something and unwilling to write it has answered a different question than the one you asked.

Keep the correspondence with the deployment file.

The reference call

Twenty minutes with an existing customer, and it is the most informative step available.

Speak to the operator, not the sponsor.

Ask the false alarm rate per shift, and whether anyone still reads the alerts.

Ask what camera work was needed beyond the original quote.

Ask what the vendor was like when something broke.

Ask what they would do differently.

Ask for a site with comparable conditions, and note what it means if none can be offered.