Module 7 — Cold start: new users and new items

Every recommender the previous modules built shares one blind spot: it cannot say anything useful about a user with no interactions, nor about an item that has none. The two problems have different fixes and different consequences, and both must be solved before your system ships — because in production, new users and new items are not corner cases, they are the majority of your daily traffic.

Two problems, not one

The new-user cold start is a data problem on the demand side. A learner registers, opens the home page, and there is no history to model them on. Every collaborative signal about that user is empty. What we know: the sign-up context (source, country, language), the very first click if any, and whatever they told us during onboarding.

The new-item cold start is a data problem on the supply side. A course is added to the catalog. Zero enrollments, zero ratings. Collaborative filtering will never surface it, and it will languish invisible until it has accumulated enough interactions to be scored — a chicken-and-egg trap that can persist for months.

The fixes differ. Content-based methods (module 4) address item cold start naturally, because the description exists on day one. For user cold start, we need to acquire information from the user: either passively (context) or actively (a short questionnaire).

Item cold start: content is the answer

Any content-based scorer solves it out of the box. Compute the item embedding from title, description and tags; add it to the ANN index; the item is now retrievable on day zero. The two-tower and LightFM approaches (modules 5 and 6) inherit this property because their item towers consume features, not identities alone.

The only real question is how aggressively to promote new items while they are cold. Three patterns work.

• Ceiling: cap the fraction of the recommended list that new items can occupy (say, 20 %). Simple, protects against the noise a new item introduces to a mature user's list, and easy to A/B test.
• Boost: multiply new-item scores by a small factor (say, 1.2) for their first 30 days on the catalog, then decay to 1.0. This gets them in front of eyeballs faster.
• Editorial slot: reserve one slot in the top-10 for "newly added" items, chosen by editorial rules or by content match to the user's known interests.

The trap is over-boosting: an unpromoted new item that turns out to be low-quality quietly disappears, an over-promoted one wastes real estate on many users and depresses your CTR for weeks. Boost modestly, measure the effect specifically on new-item metrics, and decay.

User cold start: context first, then a questionnaire

Even before the user does anything, we know:

• Country and language: derived from IP or the browser. On our platform, a French-speaking user gets French-language courses recommended by default, which alone doubles the useful surface of the home page.
• Referral source: a user who arrived from a "Docker certification path" landing page tells us something about intent.
• Time and device: a mobile signup on Sunday evening looks different from a laptop signup at 10 AM on Monday, and the two populations have measurably different completion patterns.

That is enough for a first meaningful recommendation, and it costs nothing. On top of it, an onboarding questionnaire produces the fastest cold-start jump we have measured:

Welcome — pick three topics you want to develop:
[ ] Data engineering   [ ] Machine learning   [ ] Cloud
[ ] Cybersecurity      [ ] Frontend           [ ] DevOps
[ ] Product management [ ] UX design          [ ] Databases

Optional: what is your current level?
( ) Beginner  ( ) Intermediate  ( ) Advanced

Three checkboxes plus one radio is a 15-second effort for the user and it changes the recommender's first list from "here is what everyone gets" to "here are courses in your three declared topics at your level". On our platform this simple questionnaire lifts first-week course completions by roughly a factor of two for new users.

Do not overdo it. A ten-question form scares users and its later answers become less accurate (they click through to finish it). Three checkboxes and one radio is the sweet spot.

Popularity as the safe default

Below all these methods sits the last-resort baseline: most popular. It is what you serve when everything else has zero signal, and it is worth doing well.

Two adjustments make it usable:

• Popularity by segment, not global: the most popular course among French-speaking beginners is not the same as the global bestseller. Segment popularity is very close to a proper recommendation when the segments are meaningful.
• Time-decayed popularity: what people enrolled in this month, not what people enrolled in in the last five years. A 30-day exponential decay gets you most of the way.

Global most-popular alone is worse than segmented popularity by 30 to 50 % on cold-user CTR, in our experience. Never ship "top-10 all-time" as your default; ship "top-10 in the user's segment this month".

Exploration: intentional randomness, controlled

There is a deeper problem underneath cold start: your recommender only ever learns about items it recommends. If a new course never appears on any recommendation list, it never accumulates interactions, and it stays cold forever. The recommender is optimizing its short-term reward at the price of its long-term data quality.

The fix is explicit exploration: reserve a small budget of the recommendation slots for items the model is not confident about. Two clean forms:

• $\varepsilon$-greedy: with probability $\varepsilon = 0.05$, pick a slot at random from a pool of under-served items instead of the model's top pick. Cheap, easy to reason about, correct in expectation.
• Upper-confidence bound: score each item by "estimated relevance plus a bonus proportional to $1/\sqrt{n_i}$ where $n_i$ is the number of times it has been shown". Items shown rarely get a temporary boost; items shown often revert to their measured performance.

Both cost you a fraction of a percent of CTR today and pay you back in weeks by producing a healthier data distribution tomorrow. If you never explore, you are training on a self-selected sample of what you already recommend, and every module we built silently degrades over time.

import numpy as np

def epsilon_greedy(scores: np.ndarray, epsilon: float = 0.05, k: int = 10):
    ranked = np.argsort(-scores)
    top = list(ranked[:k])
    for j in range(k):
        if np.random.random() < epsilon:
            # Replace slot j with a random underexposed item
            candidate = int(np.random.choice(ranked[k:k + 200]))
            if candidate not in top:
                top[j] = candidate
    return top

Cold-start metrics are not the usual metrics

Recall@10 and NDCG@10 (module 8) aggregate over all users. In a cold-start diagnosis they hide the problem, because 90 % of your traffic is warm users where the recommender does well. You need to slice.

• Recall@10 on users with fewer than 3 interactions: measures the new-user pipeline.
• Coverage in top-10 recommendations for items less than 30 days old: measures the new-item pipeline.
• Time to first meaningful recommendation for a new user: how many home-page visits until they get a personalized (non-popularity) suggestion.

Log these three metrics from day one. On our platform, a change that improved global recall@10 by 3 points once turned out to have dropped new-user recall by 40 % — a real production regression that the average metric erased.

Cold start is a joint product decision, not just a modeling one

The best cold-start recommender is often the one that changes the product: a mandatory onboarding step, a "your first three courses are free" bundle, a signup form that captures three topics. These change the input the recommender receives, and they beat any modeling improvement by an order of magnitude. Talk to the product team before adding complexity to the model.

Summary

• New items are handled by content-based methods and feature-enriched factorization (LightFM, two-tower); the design decision is how aggressively to boost them, with modest boosts and decay winning.
• New users are handled by context (country, language, referral) plus a very short onboarding questionnaire; three checkboxes and one radio is the sweet spot.
• Segmented, time-decayed popularity is the last-resort baseline and should always be built well; global most-popular is a bug hidden in a metric.
• Controlled exploration ($\varepsilon$-greedy or UCB) pays a small CTR cost today for the long-term health of your data, and cold-start metrics must be sliced (new users, new items) or you will not see regressions.

Next module: how to measure a ranked list — recall, NDCG, coverage, diversity — and why RMSE has no place in that measurement.