Module 9 — Implicit feedback and position bias

Every module until now has assumed the training signal was a rating or a clean positive interaction. Real recommenders are trained mostly on clicks, dwell time, enrollments and completions — signals that are cheap, plentiful and systematically distorted. This module is about knowing the distortions and doing something about them, because a model that trains naively on clicks slowly learns to recommend the top-of-page items regardless of quality.

Clicks are not preferences

A user clicks an item on your home page. What did that click mean?

• The user saw it and found it interesting.
• The user saw it, hesitated, and clicked because it was in slot 1.
• The user did not really see the ones below and clicked the first thing that caught their eye.
• The user meant to click something else and mis-tapped.
• A bot clicked it.

The first case is what we want to model. The others are noise. The problem is that we cannot tell them apart from the click log alone; we can only correct for their known statistical fingerprints.

The absence of a click is worse. A non-click on item $i$ can mean:

• The user saw it and decided against it.
• The user did not scroll far enough to see it.
• The user was on a mobile screen where only three items fit above the fold.

Treating a non-click as a hard negative is what causes the most spectacular training failures in production. The user did not tell you they disliked what they never saw.

Confidence weighting: what implicit ALS does

The classical fix, introduced in Hu, Koren and Volinsky's 2008 paper on implicit ALS, is to treat every user-item pair as a positive with a confidence proportional to how strong the signal is:

$$c_{ui} = 1 + \alpha \cdot r_{ui}$$

where $r_{ui}$ counts interactions (minutes watched, purchases, etc.). Missing entries have $r_{ui} = 0$ and confidence 1 — a weak positive at the mean, not a strong negative. Observed positives have confidence proportional to their strength.

The library implicit implements this directly (module 3 showed it). The pattern generalizes: whenever you have a graded signal, feed it as a confidence weight rather than as a hard label. A 30-second view is not a negative; it is a weak positive.

For our online learning catalog, the confidence rule we use is:

$$c_{ui} = 1 + \log(1 + \text{minutes watched}) + 3 \cdot \mathbb{1}[\text{enrolled}] + 5 \cdot \mathbb{1}[\text{completed}]$$

A completed course has $c \approx 9$; a viewed-only course $c \approx 1$; anything in between grades continuously.

Position bias: the click is a function of the slot, not just the item

Here is the observation that ruins naive click training. When you show a ranked list, users click slot 1 far more often than slot 10, and slot 10 far more often than slot 50 — regardless of the item shown. This is a property of human attention, not of the items.

The consequence: if your training data comes from a page where the recommender put popular items in slot 1, popular items collect disproportionate clicks. A new model trained on that data will learn that "high-popularity item" is a strong feature — because that is exactly what predicts a click in the log. The feedback loop is closed: your recommender learns to be its own past.

The empirical fingerprint on our platform: the item in slot 1 gets clicked about 6 % of the time on average; the item in slot 10, about 1 %. That factor of six is almost entirely position, not quality.

Inverse propensity scoring, in one paragraph

The clean fix, from causal inference, is to reweight each observed click by the inverse of its propensity — the probability that the item was seen at all. If item $i$ was shown in slot $p$ and slot $p$ has a known observation probability $\pi_p$, weight that click by $1 / \pi_p$. Clicks on low-position items now count more, correcting for the fact that they were seen less.

The training loss becomes:

$$\mathcal{L} = \sum_{(u, i, p) \in \text{clicks}} \frac{-\log P(i \mid u)}{\pi_p} + (\text{regularization}).$$

You need estimates of $\pi_p$. Two ways to get them:

• Randomization experiment: on a small fraction of traffic, shuffle the top-$k$ list randomly. From the resulting clicks by slot, you can back out an empirical $\hat{\pi}_p$ curve. Expensive (the user sees a worse list), but the ground truth.
• Model-based: fit a click model — the position-based model of Chapelle and Zhang, for instance — that decomposes click probability into $\pi_p \cdot \gamma_{ui}$ (a slot term and a relevance term). The literature is substantial; pyClick implements the common models.

On our catalog, the model-based estimate is enough. Using it as a training weight moved cold-item recall@10 by three points and coverage by five, at no cost to warm-user relevance.

Popularity bias, its cousin

Position bias and popularity bias produce very similar symptoms — the recommender learns to over-recommend the head of the catalog — but they arise from different mechanisms. Position bias comes from where items were shown. Popularity bias comes from the fact that popular items are shown to more users to start with, appear more often as random negatives in your training, and dominate the softmax normalization of module 6.

The fix is symmetric to position bias: reweight training positives by the inverse of item popularity, capped so that a single ultra-rare item does not blow up:

$$w_i = \min\left(\frac{1}{\text{freq}(i)^{0.5}}, w_{\max}\right).$$

The square-root exponent softens the correction (a full $1/\text{freq}$ makes rare items too loud and breaks training stability). The cap $w_{\max}$ prevents a single view of an obscure course from dominating a batch.

Reading learning curves for implicit models

When training on implicit feedback with in-batch negatives, the loss curve is not particularly informative — it is a sampled softmax whose absolute value depends on batch size and negative distribution. Trust the held-out ranking metrics instead, and read them by slice:

• Recall@10 on head users vs tail users. A curve that keeps improving on head users while plateauing on tail users is a model overfitting to the loud population.
• Coverage over training epochs. A model whose coverage drops from 60 % to 30 % across training is collapsing onto the head of the catalog — usually a symptom of missing popularity correction.
• Cold-item recall (items less than 30 days old). Should not go to zero; if it does, your negative sampling is starving the item tower of new-item positives.

Three curves, not one. A loss curve alone hides all three of these regressions.

def evaluate_by_slice(model, test_by_user, item_features, item_ages):
    from collections import defaultdict
    buckets = defaultdict(list)
    for u, rel in test_by_user.items():
        head = interaction_count(u) >= 50
        reco = model.recommend(u, k=10)
        r10 = len(set(reco) & rel) / max(len(rel), 1)
        buckets["head" if head else "tail"].append(r10)
        new_hit = any(item_ages.get(i, 999) < 30 for i in reco if i in rel)
        buckets["cold_item"].append(int(new_hit))
    return {k: sum(v) / max(len(v), 1) for k, v in buckets.items()}

The delayed-feedback complication

On our platform, an enrollment happens minutes after a click; a completion happens weeks after. If your training loop uses "completed" as the positive label, most of your recent positives are missing at training time — they will not materialize until the user finishes the course. Naive training on this data mislabels many warm positives as negatives.

Two mitigations:

• Multi-signal training: use enrollment as the primary positive (fast, dense) and completion as a secondary re-weighting signal, applied when it becomes available.
• Delayed-conversion models: at training time, mark recent examples with an uncertainty flag and downweight them; when a completion arrives later, upweight the original pair. The literature calls this the "delayed feedback" problem.

Most systems ignore this and pay for it in slow drift; naming it is half the fight.

Train on what the user saw, evaluate on what they did

The training set should include an explicit "impression" log: for each user, the list of items they were shown at each time step. Positives are the ones they clicked; the rest are candidate negatives, weighted by position. Without impressions, you cannot separate "user did not want this" from "user did not see this", and every module of this course silently degrades.

Summary

• Clicks and views are noisy positives, not preferences; missing interactions are not hard negatives — treating them as such wrecks training.
• Confidence weighting (implicit ALS, weighted BPR) turns graded signals into training weights and is the first correction to make.
• Position bias and popularity bias produce the same symptom (head-of-catalog domination) via different mechanisms; inverse propensity scoring on positions and a mild popularity re-weighting correct them.
• Read learning curves by slice (head vs tail users, cold-item recall, coverage over training) — a single loss curve hides the exact regressions you must avoid.

Next module: pulling it all together into a project — an evaluated recommender on the catalog, from data split to metrics table to an A/B plan.