Skip to content

All notes  /  Building

Buying One

The questions that separate a product you can assess from one you cannot, and the answers that should end a conversation.

Procedure

Most of the risk in this field is accepted at procurement, in a room where nobody asks the questions that would surface it.

Performance questions

What are the recall and precision at the recommended threshold, on what test set?

What is the composition of that test set?

Has subgroup performance been measured where people are involved, and can we see it?

What is the false alarm rate per camera per day in a deployment like ours?

Can we run a pilot on our cameras with our own ground truth?

A refusal on the last is an answer.

Data questions

What is transmitted from the camera: frames, crops, templates, or derived events?

What is stored, where, and for how long, by default?

Is our footage used to train your models? Frequently yes, frequently buried, and it is a separate purpose.

Who are the subprocessors and where do they operate?

What happens to our data at termination, and in what timescale?

Can we get the answers in the contract rather than in an email?

Capability questions

The ones that reveal what the product actually is.

Which features involve identifying individuals, and can those be disabled at the device rather than in a settings page?

Does it do demographic estimation or emotion inference? Check rather than ask, because these are frequently present and enabled.

Can it re-identify a person across days? If yes, it is an identification system regardless of how it is sold.

What is the classification under the AI Act, and can you show the documentation?

A provider who cannot say which risk tier their product falls into has not done the work, and that is a finding about the provider.

Operational questions

What is the alert volume at our camera count? Do the arithmetic together.

How are models updated, and what happens to our tuned thresholds?

How do we know when performance degrades?

What monitoring does the product expose — confidence distributions, detection counts, health?

What does support look like when a camera stops detecting and nobody knows why?

What should end the conversation

No subgroup performance figures and no willingness to measure them, for a people-facing application.

A demonstration instead of a pilot.

Identification features that cannot be disabled.

An unwillingness to state where data is processed.

Training on customer footage as a non-negotiable term.

Accuracy claims without a stated threshold or test set.

Costing it properly

Licence, per camera per year.

Camera work, which is usually the largest line and never in the quote.

Network and storage.

Operator time on alerts, at the measured rate, permanently.

Evaluation: the ground truth collection, quarterly.

Assessment and legal work, for anything people-facing.

Decommissioning.

The total is frequently several times the licence, which is worth knowing before rather than after.

The contract terms worth insisting on

Six terms that are easier to obtain before signing than after.

Processing location and subprocessors, named.

No use of customer footage for model training, or an explicit opt-out.

Deletion at termination within a stated period, confirmed in writing.

Export of detection history in a usable format.

Notification before feature changes that alter what the system does.

Documentation supporting the regulatory classification.

A vendor unwilling to commit to these in writing has told you how the relationship will go.

The question that ends conversations

One question that reveals whether a vendor has done the work.

"Which risk tier does your product fall into under the applicable regulation, and can you show the documentation?"

A provider unable to answer has not assessed their own product, which means you cannot rely on anything else they claim about compliance.

Ask it early, because it saves the rest of the evaluation.

Note that their answer does not make your deployment compliant — compliance is a property of a deployment — but their inability to answer tells you what kind of supplier this is.