Module 3 — Stationarity, differencing, autocorrelation

The classical statistical models we meet in module 5 (ARIMA, SARIMA) do not accept the raw pharmacy series. They expect a stationary input — a series whose mean and variance do not drift with time — and they let us reach it through differencing. This module explains what stationarity means, how to test for it, when to difference and by how much, and how to read the two plots (ACF and PACF) that will let us pick model orders in a principled way.

What stationarity means

A time series is weakly stationary if three properties hold:

1. Its mean does not change with time.
2. Its variance does not change with time.
3. Its autocovariance at lag $k$ depends only on $k$, not on $t$.

Note what stationarity does not require: it does not require the series to be flat or predictable. Random noise around zero is stationary. Our pharmacy series is not — it has a trend, and its weekly and yearly seasonal patterns violate condition 1.

The reason to care is theoretical: the estimators inside ARIMA (autoregressive and moving-average coefficients) are only unbiased on a stationary process. Feed them a trending series and they will "learn" the trend as a coefficient close to 1, then extrapolate it into an implausibly straight line for the next month.

The ADF test

The Augmented Dickey-Fuller test formalizes the question. Under its null hypothesis, the series contains a unit root — an integrating process, meaning it needs differencing. A p-value below 0.05 rejects the null and lets us treat the series as stationary.

import pandas as pd
from statsmodels.tsa.stattools import adfuller

df = pd.read_parquet("pharmacy_daily.parquet")
series = df.set_index("date")["units"].asfreq("D")

stat, pvalue, _, _, crit, _ = adfuller(series.dropna(), autolag="AIC")
print(f"ADF statistic: {stat:.3f}, p-value: {pvalue:.4f}")
print("critical values:", crit)

On the pharmacy series without any differencing, we get p = 0.31 — the null cannot be rejected, the series is non-stationary. That is expected, and it is exactly what differencing will fix.

Two important caveats. First, the ADF test has low power on series with strong seasonality: it can report "stationary" even when a weekly cycle screams from the plot. Always look at the plot in parallel. Second, "stationary by ADF" does not mean "ready for ARIMA": if seasonality is present, seasonal differencing (see below) is also required.

Simple differencing

The first differencing operator is:

$$\Delta y_t = y_t - y_{t-1}$$

It removes a linear trend. If our pharmacy series were growing by 0.1 units per day on average, $\Delta y_t$ would be a stationary series with mean around 0.1.

In pandas:

diff1 = series.diff().dropna()
adfuller(diff1)[1]  # p-value

On our series, p drops to 0.02 after one differencing. Good — but the plot still shows the same weekly cycle. Simple differencing did nothing to seasonality; that is not what it is for.

Seasonal differencing

Seasonal differencing removes a periodic pattern of period $s$:

$$\Delta_s y_t = y_t - y_{t-s}$$

For a weekly cycle, $\Delta_7 y_t = y_t - y_{t-7}$. This subtracts "the value on the same weekday one week ago" and cancels the weekly seasonal.

diff_seasonal = series.diff(7).dropna()
adfuller(diff_seasonal)[1]

For our pharmacy series, applying both a lag-1 and a lag-7 difference leaves a residual that passes ADF and has no visible cycle. In SARIMA notation (module 5), this corresponds to d=1 (order of simple differencing) and D=1 (order of seasonal differencing) with s=7.

The order of differencing is how many times you apply the operator. Rarely more than $d = 1$ or $D = 1$; over-differencing injects negative autocorrelation and inflates the variance of residuals. pmdarima.auto_arima can pick this automatically, but knowing what it decides matters for reading the model.

The ACF: which past matters directly and indirectly

The autocorrelation function at lag $k$ is the correlation between $y_t$ and $y_{t-k}$:

$$\rho_k = \frac{\text{Cov}(y_t, y_{t-k})}{\text{Var}(y_t)}$$

Plotting ACF at lags 0 to, say, 40 tells us how far back the series remembers.

import matplotlib.pyplot as plt
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf

fig, (a, b) = plt.subplots(1, 2, figsize=(10, 3))
plot_acf(diff_seasonal, lags=40, ax=a); a.set_title("ACF")
plot_pacf(diff_seasonal, lags=40, ax=b); b.set_title("PACF")
plt.tight_layout()

On the seasonally differenced pharmacy series, the ACF drops off quickly after lag 1 and has a single spike at lag 7 — the residual weekly memory the model still has to capture. That single spike at the seasonal lag is the signature of a seasonal moving-average term of order 1, and we will name it $Q = 1$ with $s = 7$ in module 5.

The PACF: direct dependence at each lag

The partial autocorrelation function at lag $k$ measures the correlation between $y_t$ and $y_{t-k}$ after removing the effect of all shorter lags. It answers "does $y_{t-k}$ add information beyond $y_{t-1}, \dots, y_{t-k+1}$?".

Reading rules of thumb we will lean on in module 5:

Pattern on stationary series	Implication
PACF cuts off at lag $p$, ACF tails off	AR($p$)
ACF cuts off at lag $q$, PACF tails off	MA($q$)
Both tail off	ARMA($p$, $q$), choose orders by AIC

Seasonal versions use the same rules at lag $s$ instead of lag 1:

Pattern at lag $s$	Implication
PACF spike at $s$, ACF tail from $s$	Seasonal AR ($P$)
ACF spike at $s$, PACF tail from $s$	Seasonal MA ($Q$)

On our pharmacy data, PACF shows spikes at lags 1 and 2, and ACF a spike at lag 7 with the tail after. That reads as ARIMA(2, 1, 0)(0, 1, 1)$_7$ — a lookup we would then confirm with AIC.

The Ljung-Box test on residuals

After fitting any model, we test whether its residuals are white noise. The Ljung-Box test aggregates autocorrelations across many lags into a single p-value. A p-value below 0.05 says there is autocorrelation left in the residuals: the model is missing something.

from statsmodels.stats.diagnostic import acorr_ljungbox
acorr_ljungbox(residuals, lags=[10, 20, 30], return_df=True)

This test will come back in module 5 as our final quality gate on ARIMA fits.

Differencing is not stationarity by itself

A colleague reports "I differenced once, ADF passes, we're good". Ask: does the plot still show a weekly cycle? If it does, they need seasonal differencing too. Passing ADF is necessary; it is not sufficient when seasonality is present.

Summary

• Stationarity means constant mean, constant variance, and autocovariance depending only on the lag; ARIMA requires it.
• The ADF test rejects the unit-root null when p < 0.05; low power on seasonal series, always double-check with a plot.
• Simple differencing removes trend; seasonal differencing at lag $s$ removes a period-$s$ cycle. Rarely need more than one of each.
• ACF and PACF are the two plots we read to choose AR and MA orders; the same rules apply at lag 1 (non-seasonal) and lag $s$ (seasonal).

Next module: baselines. Before a single ARIMA is fit, we build the naïve, seasonal-naïve, moving-average and drift forecasts that any real model must beat.