Skip to content

All notes  /  The boundary

What Not to Build

A short list to rule out at design time, with the reason for each and what to offer instead. Cheaper to refuse than to remove.

Reference

Ruling something out before it exists costs a conversation. Removing it afterwards requires admitting it should not have been built, which organisations avoid by keeping it.

Emotion, engagement, intent or deception inference

Why not: contested science, no ground truth for intent, uneven error distribution, and prohibited in workplace and educational contexts in the EU.

Instead: detect observable conditions. Ask people about their experience.

Biometric categorisation by sensitive characteristic

Inferring race, ethnicity, religion, political opinion, sexual orientation or health from appearance.

Why not: specifically prohibited in the EU, unsound in principle, and the errors are the harm.

Instead: nothing. There is no legitimate operational question this answers.

Untargeted database building

Scraping faces from the internet or from CCTV to build or expand a recognition database.

Why not: explicitly prohibited under the EU AI Act, and the subject of enforcement action elsewhere.

Instead: enrol people who have chosen to enrol, for a stated purpose.

Covert analysis

Why not: notice is a legal requirement with narrow, procedural exceptions that a standing capability does not meet.

Instead: notify. The deterrent value depends on people knowing.

Individual productivity scoring from video

Time at a workstation, activity level, movement counts attributed to a named person.

Why not: it measures the workstation and the process rather than the person, it is trivially adapted to, and it destroys the cooperation the safety applications depend on.

Instead: aggregate measures of the process. This is the same boundary that applies to every monitoring technology.

Analysis in welfare and private areas

Bathrooms, changing rooms, medical rooms, break areas, prayer rooms.

Why not: no operational question requires it, and in many jurisdictions it is unlawful outright.

Instead: exclude at the camera configuration level so no data exists, and publish the exclusion list.

Automatic consequences from a detection

A detection triggering discipline, refusal of service or an accusation without a human decision.

Why not: every system has an error rate that falls unevenly, and automated decisions with significant effects carry specific legal requirements including human involvement and a route to contest.

Instead: human review at the point of consequence, always, with the reviewer able to see and override.

Gait, iris or other biometrics as a workaround

Proposed when facial recognition is refused, on the basis that it is not a face.

Why not: it is still biometric identification, the legal analysis is the same, and the accuracy is generally worse.

Instead: recognise the request for what it is and apply the same test.

The design-time test

What decision does this inform? If none, drop it.

Could detection or counting inform the same decision? If yes, do that.

Would you be comfortable explaining this to the people it applies to, in detail? If not, that discomfort is the answer.

Apply all three before the proof of concept, because a working prototype creates its own momentum and the conversation gets harder.

Refusing at the prototype stage

The moment when refusal is still cheap.

A working prototype creates momentum that a proposal does not: someone has seen it, it exists, and stopping it becomes a loss rather than a decision.

Which means the boundary questions belong before the proof of concept, not after.

Ask the three tests at the point the idea is described: what decision, could a lower tier do it, would you explain it to the people affected.

A week of refusal at proposal stage costs a conversation. The same refusal after a demonstration costs a fight.

Recording what was ruled out

The document that answers the same proposal when it returns in two years.

What was proposed.

What was ruled out, and the specific reason — legal, technical, or both.

What was built instead.

Who decided, and when.

Findable, so the second identical proposal is answered by reference rather than relitigated from scratch.

Reviewed annually, because the law changes in this field faster than most, and it has been changing toward more restriction rather than less.