Skip to main content

Module 9 — Personal data protection

The AI Act (module 8) regulates the AI system. The General Data Protection Regulation — Regulation (EU) 2016/679, universally called the GDPR — regulates the personal data that flows through it. The two frameworks coexist: an AI system that complies with the AI Act still needs a GDPR justification for every byte of personal data it touches, and vice versa. This module goes through the GDPR principles that shape a machine learning project, ends on Article 22 (the article that governs automated decisions and is the one every credit scorer must answer to), and mentions the equivalent frames for readers outside the Union.

Six principles that discipline the data​

Article 5 GDPR states six principles that every processing operation must satisfy.

Lawfulness, fairness and transparency. The processing must rest on one of the six lawful bases in Article 6 (consent, contract, legal obligation, vital interests, public interest, legitimate interests), and it must be documented in a way the data subject can understand.

Purpose limitation. Data collected for one purpose cannot be reused for an incompatible purpose. If the credit dataset was collected to decide loan applications, using it to train a marketing model requires a new legal basis and, in practice, a new consent.

Data minimization. Collect and process only what is necessary for the stated purpose. This principle is the one machine learning teams most often violate, because "more features usually helps accuracy". If the extra features are not necessary, they cannot be processed.

Accuracy. Personal data must be accurate and kept up to date. Data subjects have a right to correction (Article 16), and the credit scorer must have a procedure for pushing corrected records back into training pipelines.

Storage limitation. Data must not be kept longer than necessary. The retention period must be defined per data category and enforced by automation.

Integrity and confidentiality. Appropriate technical and organizational measures — encryption, access control, audit logs, incident response.

A seventh, transversal principle — accountability (Article 5(2)) — requires the controller to be able to demonstrate compliance. Documentation, not a good-faith declaration.

The lawful basis for the credit scorer​

For processing credit applications, the standard lawful basis is contract necessity (Article 6(1)(b)): the processing is necessary to enter into or perform the loan contract requested by the applicant. Consent is not the appropriate basis here, because refusing consent would prevent the very service being requested — the "freely given" condition would be doubtful. For the retention of application data beyond the immediate decision, and for the training of future models on that data, a distinct basis is needed, usually the bank's legitimate interests (Article 6(1)(f)) subject to a balancing test that documents why the interest is not overridden by the applicant's rights.

Any use of special categories of data (Article 9: ethnicity, health, biometrics, sexual orientation, political opinions, religion, trade union membership, genetic and biometric data) requires one of the ten specific derogations. Special-category data is almost never present as a labeled column in a credit dataset, but its proxies (module 2) may be. If the audit reveals that ZIP code plus first name predicts ethnicity with high accuracy, the special-category regime is arguably triggered — a question for the data protection officer, not the data scientist.

Data subject rights​

Chapter III of the GDPR grants eight rights:

  • Information (Articles 13-14): the data subject is told about the processing at collection time.
  • Access (Article 15): they can obtain a copy of their data.
  • Rectification (Article 16): they can have inaccurate data corrected.
  • Erasure (Article 17): "right to be forgotten", subject to exceptions.
  • Restriction (Article 18): they can require processing to be paused.
  • Portability (Article 20): they can obtain their data in a machine-readable format.
  • Objection (Article 21): they can object to processing based on legitimate interests.
  • Not to be subject to a solely automated decision (Article 22): the topic of the next section.

Every one of these rights must have a technical implementation. "Right to erasure" is a joke if the training set is copied to five clusters and only one gets cleared. The audit records the pipeline that propagates each request and the maximum delay from request to full effect.

Article 22 and automated decisions​

Article 22 is the article that most directly touches machine learning:

The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.

Three conditions must all be met for Article 22 to bite: the decision is solely automated, it produces legal or similarly significant effects, and it is based on the automated processing. A credit decision at 30 000 euros clearly ticks the second and the third; whether the first applies depends on whether a human meaningfully intervenes.

Three carve-outs exist (Article 22(2)): necessity for the contract, authorization by Union or Member State law with suitable safeguards, or the data subject's explicit consent. In the two most-used cases (contract, consent), the controller must at minimum implement suitable measures including the right to obtain human intervention, to express a point of view, and to contest the decision. Recital 71 adds a "right to an explanation" though its exact scope is still litigated.

The design consequence is that a credit scorer must never be the sole decision-maker on a significant application without: (i) an explicit contract-necessity or consent basis, (ii) a clear notification of the automated nature, (iii) the module 7 recourse procedure. Fail on any one and the decision is illegal under Article 22.

Data protection impact assessment​

Article 35 requires a DPIA before starting a processing likely to result in a high risk. Credit scoring almost always qualifies. A DPIA describes the processing, assesses its necessity and proportionality, identifies the risks to data subjects, and details the measures to address them. The audit dossier of module 10 embeds the DPIA. The DPIA is not one-off: substantial changes to the model, the data or the scale trigger a new one.

Beyond the Union​

Readers outside the Union work under other frames that overlap heavily with the GDPR: the UK GDPR; Brazil's LGPD; Canada's PIPEDA and its Quebec Law 25; California's CCPA/CPRA; Colorado's CPA and other US state laws; Australia's Privacy Act reforms; South Africa's POPIA; the Arab world's varying but increasingly convergent regimes (Saudi Arabia's PDPL, the UAE's Federal Personal Data Protection Law). None of them make the credit scorer easier; several add specifics (right to explanation, algorithmic impact assessment). The audit checks the applicable frame at the deployment location, not at the training location.

Summary​

  • The GDPR frames every byte of personal data: six principles (lawfulness, purpose, minimization, accuracy, storage limitation, integrity) plus accountability.
  • The credit scorer's lawful basis is contract necessity for the decision itself and legitimate interests (documented) for training-set retention.
  • Every data subject right must have an implementation that propagates through the whole pipeline, not just the primary store.
  • Article 22 forbids solely automated significant decisions unless a carve-out applies and unless human intervention, expression and contestation are guaranteed.

Next module: the full audit report on the credit scorer, stitching together everything from modules 2 through 9.