Module 2 — User-based and item-based collaborative filtering

The oldest family of recommenders rests on one sentence: users who agreed in the past will agree in the future. No content, no metadata, no descriptions — only who consumed what. It is worth writing carefully because most modern methods (factorization, two-tower) are, at heart, a smarter version of the same idea, and because on a real catalog the neighborhood approach still surfaces failure modes that fancier models inherit and hide.

We continue with the online learning catalog of module 1: 40 000 learners, 500 courses, a rating matrix filled at about 0.45 %.

Two symmetric formulations

Given the user-item matrix $R$, two formulations use the same tool — nearest neighbors — from opposite ends.

User-based CF predicts $\hat{R}_{ui}$ by looking at users who resemble $u$ and averaging their opinions on item $i$:

$$\hat{R}_{ui} = \bar{r}_u + \frac{\sum_{v \in N_k(u,i)} \text{sim}(u,v)\,(R_{vi} - \bar{r}_v)}{\sum_{v \in N_k(u,i)} |\text{sim}(u,v)|}$$

where $N_k(u,i)$ is the top-$k$ users who both resemble $u$ and have rated $i$, and $\bar{r}_u$ is user $u$'s mean rating (subtracted to normalize away chronic optimists and pessimists).

Item-based CF predicts $\hat{R}_{ui}$ by looking at items that resemble $i$ among those $u$ has already rated:

$$\hat{R}_{ui} = \frac{\sum_{j \in N_k(i,u)} \text{sim}(i,j)\, R_{uj}}{\sum_{j \in N_k(i,u)} |\text{sim}(i,j)|}$$

The item-item variant, popularized by Amazon in the early 2000s, generally works better on real catalogs. The reason is data volume: on a platform where new users appear every day, a user's neighborhood is unstable and needs recomputing constantly, while item similarities move slowly. Between the two, prefer item-based unless you have a specific reason.

Which similarity, and why it matters more than it looks

Two similarity measures dominate.

Cosine between two vectors $\mathbf{a}$ and $\mathbf{b}$ is

$$\cos(\mathbf{a},\mathbf{b}) = \frac{\mathbf{a}\cdot\mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|}.$$

Applied to two item columns, it measures how aligned their rating vectors are, ignoring their magnitudes. A course rated by 500 users looks as similar to another course as a course rated by 5, provided the direction matches.

Adjusted cosine (or Pearson correlation on centered rows) first subtracts each user's mean before computing cosine on the residuals. This removes the "everyone rates high" effect and consistently outperforms plain cosine on explicit ratings. On implicit data — enrollments, views — you skip the centering because there is nothing to center.

The choice of similarity moves the recall@10 of a real system by several points, so it deserves attention. The bigger trap, however, is co-support.

The co-support trap and shrinkage

Two courses that share exactly two raters can have cos = 1.0. Two courses that share 400 raters and agree on 380 of them can have cos = 0.94. Which one is more "similar"? The second, obviously — the first is a coin flip pretending to be a signal.

The fix, folklore in the field, is shrinkage: dampen similarities computed from few co-raters. A common formula is

$$\text{sim}^{\star}(i,j) = \frac{|C_{ij}|}{|C_{ij}| + \lambda}\; \text{sim}(i,j)$$

where $|C_{ij}|$ is the number of users who rated both items and $\lambda$ a positive constant (25 is a reasonable starting point on our catalog). With $|C_{ij}| = 2$ and $\lambda = 25$, the raw similarity is multiplied by $2/27 \approx 0.07$. With $|C_{ij}| = 400$, by $400/425 \approx 0.94$. The order of neighbors is now sensible.

import numpy as np
from scipy.sparse import csr_matrix
from sklearn.preprocessing import normalize

R = csr_matrix(np.load("catalog/ratings_dense.npy"))   # users x items, zeros = missing
mask = (R != 0).astype(np.float32)                     # 1 where a rating exists

# Center rows on the user mean (adjusted cosine on items)
user_sum = np.asarray(R.sum(axis=1)).ravel()
user_cnt = np.asarray(mask.sum(axis=1)).ravel()
user_mean = np.divide(user_sum, np.maximum(user_cnt, 1))
Rc = R.toarray() - user_mean[:, None] * mask.toarray()
Rc = csr_matrix(Rc)

# Item-item cosine on the centered matrix
Rn = normalize(Rc.T, norm="l2", axis=1)               # each item is a normalized column
raw_sim = (Rn @ Rn.T).toarray()                       # items x items

# Shrinkage by number of co-raters
co = (mask.T @ mask).toarray()                        # co-rating counts
lam = 25.0
sim = (co / (co + lam)) * raw_sim
np.fill_diagonal(sim, 0.0)
print("mean shrinkage factor:", (co / (co + lam)).mean())

That single change — shrinkage — moved recall@10 from 8.3 % to 12.7 % on our catalog. It is the cheapest improvement in the module and the one most consistently forgotten.

Predicting for a user

Given sim, predicting a user's top-$k$ list is a sparse dot product between the user's rating row (already centered) and the item-item similarity matrix, restricted to items the user has not yet rated.

def top_k_for_user(u: int, k: int = 10) -> list[int]:
    user_row = Rc.getrow(u).toarray().ravel()          # 1 x items
    scores = user_row @ sim                            # 1 x items
    scores[user_row != 0] = -np.inf                    # do not re-recommend known items
    return np.argsort(-scores)[:k].tolist()

The bookkeeping is easy; two design decisions are less so. First, always exclude items the user has already consumed from the ranked output (a recommender that suggests the course they just finished is a bug you will ship if you forget). Second, decide what "known" means: is a page view enough to blacklist the course, or only an enrollment? On our platform, enrollment is the right threshold — many users browse without ever enrolling, and blocking on views hides half the catalog after a week of browsing.

Compute cost, and why item-based scales farther

The item-item similarity matrix has $|I|^2$ cells. With 500 items that is 250 000 cells — trivial. With 50 000 items it is 2.5 billion — a headache. Two techniques scale it.

• Neighborhood truncation: keep only the top-$k$ neighbors per item (say $k = 50$). The matrix becomes sparse and both storage and prediction time drop by orders of magnitude.
• Approximate nearest neighbors: build an ANN index (FAISS, HNSW) over item embeddings. Module 6 uses this heavily; for now, note that exact sim @ user_row becomes infeasible past a few hundred thousand items.

User-based, by contrast, has $|U|^2$ cells — for us, 40 000 users means 1.6 billion. Even a truncated user-user matrix requires re-computation as users appear. In practice, on a growing platform, item-based is the only neighborhood method that survives.

When neighborhood CF still wins

On a catalog under 5 000 items with a stable user base and a strong reason to explain recommendations ("people who took A also took B, C, D"), an item-item CF with shrinkage remains a very strong, transparent baseline. It is also a mandatory reference point when you build the fancier models of modules 3 through 6: if your two-tower model does not beat item-item on recall@10, something is wrong.

The failure mode: extreme sparsity on the user side

Recall from module 1 that the median user on our catalog has rated two courses. For such a user, neighborhood CF has almost nothing to work with. The similarity between them and everyone else rests on two entries; the shrinkage sends most of it to zero; the prediction defaults to the population mean.

That is not a bug of neighborhood CF, it is a natural consequence of the data. Modules 4 and 7 attack the same problem from two angles: content-based methods can produce recommendations from zero interactions, and cold-start heuristics fold in a short questionnaire. Neighborhood CF alone should be reserved for the top decile of active users — the ones who have rated dozens of courses.

Summary

• Neighborhood CF predicts by averaging opinions of similar users or similar items; item-based usually wins in practice for stability and scale.
• The similarity choice matters, and shrinkage by co-support is the most cost-effective correction to a raw cosine on real data.
• Always exclude already-consumed items from the ranked output, and pick "consumed" to match the actual funnel of your product.
• Neighborhood CF fails silently on low-activity users and on catalogs beyond a few hundred thousand items; both are handled by later modules, not by tuning $k$.

Next module: replace the sparse similarity table with a small dense set of latent factors — matrix factorization, the workhorse of the last twenty years.