Lesson 6 — Recap and frequent questions
The five lessons in five sentences
- Bias enters at six points — historical, representation, measurement, aggregation, deployment and feedback loops — and removing sensitive attributes fails because proxies reconstruct them and because you then cannot measure whether discrimination occurs.
- The main fairness definitions are mathematically incompatible when base rates differ, so the criterion must be chosen consciously by whoever is accountable rather than optimised by a library.
- Explainability methods approximate a model's behaviour rather than describing its reasoning, and for consequential decisions an interpretable model plus a real contestability route beats attribution plots.
- Consent does not survive machine learning at scale, models memorise and leak, inference reveals attributes never collected, and anonymisation is risk reduction rather than elimination.
- Accountability fails by diffusion, human oversight only counts when the reviewer has authority, time and information, and regulation tiers its obligations by what happens to a person when the system is wrong.
The one question underneath all of it
Every framework in existence is an elaboration of that question. Answer it honestly and the appropriate level of rigour follows.
A review checklist for any system
| Question | What a good answer looks like |
|---|---|
| What decision does this make? | Specific, with the acting person or system named |
| Who is harmed if it is wrong, and how badly? | Named groups and concrete consequences, not "users" |
| Who is accountable for that outcome? | A person or role, identified before deployment |
| What is the performance per affected group? | Numbers per group, not an aggregate |
| Which fairness criterion applies, and who chose it? | A recorded decision with reasoning |
| What proxy is the label, really? | An honest account of what was measured |
| Can an affected person contest it? | A route to an empowered human, with a timeframe |
| What was it validated for? | A documented boundary of intended use |
| How would you know it stopped working? | Specific monitoring with thresholds |
| What would make you switch it off? | An agreed condition, written before launch |
Twelve frequent questions
1. Is not AI more objective than humans?
It is more consistent, which is different. A model applies the same rule to everyone, including when the rule is discriminatory — and it does so at a scale and speed no individual could, with an appearance of neutrality that makes it harder to challenge. Consistency is valuable where the rule is right and amplifies harm where it is not.
2. Can we just remove ethnicity and sex from the model?
No. Proxies such as postcode, name and education reconstruct them, and once you stop recording group membership you can no longer measure whether your system discriminates. You need the attribute available to the auditing process and unavailable to the model.
3. Which fairness metric should we use?
Whichever matches what each error costs each group in your specific context. Equal opportunity where missing qualified people is the main harm, equalised odds where both error types are serious, calibration where humans use the scores. It is a value judgement, it belongs to whoever is accountable, and it must be written down.
4. Is a human in the loop enough?
Only when that human can override without penalty, has time to think about each case, and sees the inputs, the uncertainty and the known failure modes. Otherwise it adds the appearance of accountability without the substance, and automation bias means people defer to the machine precisely when they are busiest.
5. Can complex models be explained well enough for a regulator?
Attribution methods can produce a defensible account of which factors drove a specific decision, and they are approximations that disagree with each other and can be gamed. For high-stakes decisions an interpretable model is usually the safer and often barely less accurate choice, and the burden of proof should fall on adopting opacity.
6. What if our training data is all we have and it is biased?
Then say so, measure the resulting gaps by group, and decide whether to deploy with mitigations, deploy in a narrower scope, or not deploy. Those are the options. Deploying while asserting the data is fine is the one choice that is clearly wrong.
7. Does GDPR ban automated decisions?
No. It restricts solely automated decisions with significant effects, requiring a lawful basis and generally a right to human involvement and to meaningful information about the logic. Automated support for a human decision faces a much lower bar than automated decision-making.
8. Can someone ask us to delete their data from a trained model?
They can ask, and honouring it properly means retraining, which is expensive and rarely done. Machine unlearning research is active and not a general solution. This is an unresolved gap between what the law envisages and what the technology supports, and pretending otherwise is worse than acknowledging it.
9. Is using public web data for training legal?
Genuinely unsettled and actively litigated on both copyright and data protection grounds, with rulings going different ways on different facts. Anyone stating this confidently in either direction is overreaching. Read your provider's indemnity terms.
10. Should some systems not be built?
Yes. Some uses are prohibited outright by regulation. Beyond those, a system whose error profile falls hardest on people least able to contest it is a design problem rather than a bug awaiting a fix, and declining to build it is competent practice rather than obstruction.
11. How do we start doing this properly without slowing everything down?
Three things, cheap and high-return: name an owner for every model, report performance by relevant subgroup rather than in aggregate, and write a one-page card covering purpose, data, limitations and intended-use boundary. That catches a large share of real problems and costs hours rather than weeks.
12. What if I raise a concern and nobody listens?
Put it in writing, specifically and factually, addressed to whoever owns the outcome — a documented concern changes what happens after an incident and sometimes before one. Escalate to risk, legal or data protection functions, whose mandate covers exactly this. Sector regulators and professional bodies exist where internal routes fail.
Vocabulary you can now use precisely
| Term | Meaning in one line |
|---|---|
| Historical bias | Data accurately records past decisions that were unjust |
| Representation bias | A group is under-represented, so performance for it is worse |
| Measurement bias | The recorded label is a loaded proxy for what you care about |
| Aggregation bias | One model applied to groups whose relationships differ |
| Deployment bias | Using a system beyond what it was validated for |
| Proxy variable | A feature that reconstructs a sensitive attribute you removed |
| Fairness through unawareness | Deleting sensitive attributes; ineffective and prevents measurement |
| Demographic parity | Equal selection rates across groups |
| Equal opportunity | Equal true positive rates across groups |
| Equalised odds | Equal true and false positive rates across groups |
| Calibration | A score means the same thing in every group |
| Feature attribution | Assigning a prediction to input contributions; an approximation |
| Counterfactual explanation | What would have needed to differ for another outcome |
| Interpretable model | One whose behaviour can be read directly, by construction |
| Contestability | An affected person's practical ability to challenge a decision |
| Automation bias | Deferring to a machine recommendation against one's own judgement |
| Membership inference | Determining whether a record was in the training data |
| Differential privacy | A mathematical bound on any individual record's influence |
You have finished the free path
Twelve courses, from what artificial intelligence is to what it owes the people it affects. You can now read a claim about an AI system and know which questions would test it, which is the skill that outlasts any specific technology.
Validate this course: take the 5-question quiz →
Review the whole path: all free courses
Go further: the premium catalogue covers forty courses with hands-on projects, notebooks and datasets, each with a verifiable certificate after a 40-question examination. Included in every paid plan.