Module 8 — The life cycle of an AI project
The public image of an AI project is modeling: an expert in front of learning curves. The reality of a project that succeeds looks different: most of the effort happens before and after the model. This module walks through the seven stages, with the decision thresholds that keep you from getting bogged down.
Where the effort actually goes
Field surveys have converged for years: modeling represents 10 to 20% of the time of a completed project. The rest: understanding the problem, obtaining and cleaning the data, building the infrastructure, deploying, monitoring. A project plan that allocates most of its time to algorithm selection is a warning sign in itself.
Stage 1 — Framing: the decision before the model
A project is framed with four questions, in this order.
What decision will the system inform or automate? "Predicting churn" is not an end; "calling the 50 highest-risk customers every week" is. Without a downstream decision, the best prediction produces no value.
What is an improvement worth? Price the gap between the current situation and one where the prediction is, say, 20% better. If that number is small, stop here — the most profitable project is sometimes the one you do not do.
What does each type of error cost? This pricing (module 7) drives the metrics, the threshold, and the level of human oversight required.
Is there a simple solution? A business rule, a sort, a moving average settles a surprising share of "AI" requests. This question annoys people and saves quarters.
Stage 2 — Data: the honest inventory
Three checks before any modeling. Availability: does the data exist, with enough history depth, with the necessary labels? Legality: do you have the right to use it for this purpose (consent, purpose limitation, personal data)? Representativeness: does it cover the cases the system will face — or only a convenient subset?
Then the cleaning: duplicates, missing values, unit inconsistencies, contradictory labels. It is thankless and it is decisive: no algorithm recovers from wrong data.
Stage 3 — The baseline: the simplest possible model
Before any ambitious model, build a baseline: the trivial rule (majority class, yesterday's value), then a simple model (logistic regression, small tree). Three services rendered: a point of comparison that gives meaning to the numbers, an end-to-end validation of the data pipeline, and — regularly — the discovery that the baseline meets the need.
Stage 4 — Iteration: improve what limits
The modeling work proper, run in short loops: analyze where the model goes wrong (which segments, which kinds of cases), form a hypothesis (missing feature, insufficient data on a segment, model too simple), test it, measure on the validation set. Error analysis is the compass; stacking complexity without a diagnosis is the classic drift.
Stop when the marginal improvement no longer pays for the effort — the threshold was set during framing, which avoids the endless chase for one more point of performance.
Stage 5 — Final evaluation: once
Measurement on the test set (module 5), completed with checks the global metric does not show: performance per segment (is the model good everywhere, or excellent on average and failing on one group?), behavior on edge cases, robustness to degraded inputs. The system's fairness plays out here as much as its quality.
Stage 6 — Deployment: gradual, reversible
A model in production is software: versioned, tested, supervised, with a rollback plan. Gradual rollout is the norm: shadow mode (the model predicts without acting, you compare against the existing process), then a fraction of the traffic, then generalization if the indicators hold. Turning a notebook into a reliable service — reproducibility, latency, scaling — is a craft of its own: that is MLOps, to which a course in this track is devoted.
Stage 7 — Monitoring: the model ages
A model degrades without anyone touching it, because the world moves: behaviors, prices, products, frauds evolve. That is drift. Three families of signals are watched: the distribution of inputs (does today's data still look like the training data?), the distribution of predictions, and — as soon as true answers arrive — actual performance. Thresholds trigger an alert, then retraining: the cycle loops back to stage 2.
"How will we know, six months from now, that the model has degraded — and what happens then?" If the plan does not answer this question, the project is not production-ready, whatever the model's quality today.
Key takeaways
- Modeling weighs 10–20% of the effort; data and post-deployment make up the rest.
- Framing in four questions: which decision, what value, what error costs, is there a simple solution.
- Always a trivial baseline before the ambitious model; it sets the scale and validates the pipeline.
- Gradual, reversible deployment; drift monitoring with thresholds and a retraining plan — an unmonitored model degrades in silence.
Next module: what documented successes and famous failures teach — the patterns that repeat on both sides.