Module 4 — Baseline models you should never skip

Before fitting anything sophisticated, we build four boringly simple forecasts and measure their error. This is not a formality: on many real series, one of the four is impossible to beat by a wide enough margin to justify the complexity of what comes next. Skipping this step is the fastest way to write a MASE numerator whose denominator is missing — and to be genuinely surprised, three months into production, that a two-line rule outperforms your Transformer.

We will fill the first row of the results table that runs through the rest of the course.

The four baselines

Each baseline is a rule so simple you could implement it in one line of pandas. Their goal is not to be right; it is to give any other model something concrete to beat.

Naïve forecast. Tomorrow equals today.

$$\hat{y}_{t+h} = y_t$$

This works when the series is close to a random walk. On daily website traffic between two weekdays it is often surprisingly good; on weekly-seasonal pharmacy data it is disastrous because it forecasts Saturday's value for the whole month starting Saturday.

Seasonal naïve forecast. Predict the same day-of-week (or day-of-year, month-of-year) as the last one you saw.

$$\hat{y}_{t+h} = y_{t + h - m \cdot \lceil h / m \rceil}$$

For daily data with weekly seasonality, $m = 7$ and this rule copies "the same weekday last week". On strongly weekly-seasonal series it is a very hard baseline to beat — often within 5 to 10 % of what a good complex model achieves.

Moving average. Predict the average of the last $k$ observations.

$$\hat{y}_{t+h} = \frac{1}{k}\sum_{i=1}^{k} y_{t - i + 1}$$

Robust to noise, terrible on any seasonal signal because it smears the seasonality into a flat line. It is the right baseline when the series has essentially no structure and you want a stable target.

Drift. Fit a straight line through the first and last training points and extrapolate.

$$\hat{y}_{t+h} = y_t + h \cdot \frac{y_t - y_1}{t - 1}$$

Captures a linear trend without any regression machinery. Interesting for macroeconomic series with clean trend, mostly a straw man for our pharmacy data.

Implementing them

The four rules fit in a small Python module we will reuse across the course.

import numpy as np
import pandas as pd

def naive(series: pd.Series, h: int) -> np.ndarray:
    return np.repeat(series.iloc[-1], h)

def seasonal_naive(series: pd.Series, h: int, m: int) -> np.ndarray:
    last = series.iloc[-m:].to_numpy()
    repeat = int(np.ceil(h / m))
    return np.tile(last, repeat)[:h]

def moving_average(series: pd.Series, h: int, k: int) -> np.ndarray:
    return np.repeat(series.iloc[-k:].mean(), h)

def drift(series: pd.Series, h: int) -> np.ndarray:
    slope = (series.iloc[-1] - series.iloc[0]) / (len(series) - 1)
    return series.iloc[-1] + slope * np.arange(1, h + 1)

Note that seasonal_naive returns a 28-day forecast that repeats the last 7 observed days four times. That is intentional: we are honest about the fact that this rule cannot learn anything from the horizon. Its error at day 28 is not different in principle from its error at day 1.

Measuring them on our pharmacy data

We use a single held-out block of 28 days at the end of the series (module 9 will replace this with a rolling backtest).

df = pd.read_parquet("pharmacy_daily.parquet")
series = df.set_index("date")["units"].asfreq("D")

train, test = series.iloc[:-28], series.iloc[-28:]

forecasts = {
    "naive": naive(train, 28),
    "seasonal_naive_7": seasonal_naive(train, 28, m=7),
    "moving_avg_28": moving_average(train, 28, k=28),
    "drift": drift(train, 28),
}

from sklearn.metrics import mean_absolute_error
for name, f in forecasts.items():
    print(f"{name}: MAE = {mean_absolute_error(test, f):.2f}")

On our pharmacy data, the MAEs land approximately as follows:

Model	MAE (units/day)
naive	24.6
seasonal_naive_7	9.8
moving_avg_28	15.2
drift	25.1

That is the first row of our results table, and it already tells a story. The best baseline is the seasonal naïve at 9.8 units per day. Any model we build in the following modules will be scored against that number, not against the plain naïve.

Why baselines matter more than they seem

Three reasons make baselines non-negotiable.

They calibrate the reader's intuition. Reporting "our model gets MAE 8.5" is meaningless without knowing that seasonal-naïve gets 9.8 and that random-guessing would score 30. The seasoned reader immediately places the 8.5 in the right band; the junior reader takes it at face value. Baselines force us to publish both numbers together.

They provide the denominator of MASE. Module 9 will define MASE (Mean Absolute Scaled Error), a metric whose denominator is exactly the MAE of a naïve forecast on the training set. Without a baseline, MASE is undefined. A MASE below 1 means "better than naïve"; a MASE around 1 means "your model is equivalent to copying yesterday, and yesterday costs nothing".

They protect against wasted engineering. Many time series, especially in operations and finance, have an intrinsic ceiling that even a Transformer will not exceed by more than a few percent. Discovering that a two-line seasonal naïve is 5 % worse than your best LSTM after two weeks of engineering is a career-defining moment.

Baselines against complex models: the empirical record

The M-competitions — the biggest recurring benchmark in forecasting — have shown for decades that simple methods win as often as complex ones. In the M3 competition, a plain exponential smoothing beat most machine-learning entries. The M4 competition (2018) was won by a hybrid method whose main innovation was combining an ES model with an LSTM. The M5 (2020), on Walmart data, was won by LightGBM with carefully engineered features — not by a deep model.

The point is not that complex models are bad. It is that they earn their place on a specific class of series, mostly ones with many related series to learn from, external regressors, and long histories. On isolated series with a strong weekly cycle, seasonal-naïve is often within 10 % of the state of the art.

A baseline you did not measure is a baseline you cannot beat

"We tried seasonal naïve, it's obviously worse" is not an answer if the number is not written down. Compute it, put it in the table, and update the table every time you propose a new model. Any comparison that does not name a baseline should not be trusted.

What we learned from the first row

Reading the pharmacy table above, three observations follow.

The weekly seasonality dominates the signal. Seasonal-naïve is by far the best, and it captures nothing else than "same weekday last week". Anything a model adds beyond that will be measured against 9.8, and modest gains (say 8.5) are still gains — but they must be shown, not asserted.

The plain naïve is disastrous. On a series with strong seasonality, copying the last value picks whichever weekday the split happens to end on. It is a straw-man baseline; the honest baseline is seasonal-naïve.

The drift baseline is meaningless here because our series has no material trend at the 28-day horizon. Drift is only interesting on series where the trend accounts for a large share of the variance, which is not our case.

Summary

• Four baselines to try on every forecasting project: naïve, seasonal-naïve, moving average, drift.
• Fit in one line each; the seasonal-naïve is often the hardest to beat on strongly cyclic series.
• The results table opens with these numbers, and every later model is scored against them; without baselines, later metrics are unreadable.
• Baselines are also the denominator of MASE, the scale-free metric we will use in module 9 to compare models across stores and horizons.

Next module: ARIMA and SARIMA — the first family of statistical models we will fit, using the ACF and PACF reading from module 3.