Module 1 — What makes a sequence different from a table of features

Every dataset seen so far in this specialisation could be shuffled row by row without losing anything. Course 04 explicitly recommended shuffle=True before splitting; course 05 relied on it for clustering; even the image classifier of courses 08 and 09 could reorder its batches at will. Sequences are the first family of problems where shuffling destroys the signal. This module lays the ground for the entire course by explaining why, and how the shift from a table to a sequence changes every step of the pipeline — feature construction, splitting, batching and evaluation.

The red thread of the course is the hourly electricity consumption of a small office building over one year: 8 760 numbers between roughly 3 kW at night and 45 kW during peak hours, with a strong daily cycle, a weaker weekly cycle (offices consume less on weekends) and a slow seasonal drift with heating and cooling loads. Everything in this module — the naive tabular attempt, the sliding window, the leaky split — is shown on that series.

Order is a variable, and it is invisible to a dense network

A dense network reads a fixed-size vector of features. Feed it the electricity series as x = [x_1, x_2, ..., x_T] and it sees $T$ independent inputs; it has no notion that $x_{t+1}$ came after $x_t$. If you shuffle the rows of a supervised table built from consecutive hours, the network learns the same weights: the loss is invariant to the permutation of the rows.

That is a disaster for a series with autocorrelation. The single most predictive feature for the electricity at 3 p.m. Tuesday is the electricity at 2 p.m. Tuesday, not the yearly average. A model that forgets which row precedes which cannot exploit that.

There is a workaround, and it is worth knowing precisely because it fails in a specific way. Build the tabular version by hand: for each target hour $t$, add columns for x_{t-1}, x_{t-2}, ..., x_{t-24}. Now a random forest or a gradient boosting model reads the last 24 hours as features and often does quite well — sometimes it even beats a simple RNN on this exact problem. The catch appears when the length of the useful past changes with the situation: a Monday morning needs the previous Friday, a public holiday needs the last comparable holiday. A fixed-width window cannot express that, but a recurrent network with a hidden state can.

The sliding window: from a series to supervised examples

The standard way to turn a series into examples for a recurrent network is the sliding window. Choose a lookback $L$ (how many past hours the model sees) and a horizon $H$ (how many future hours to predict). Slide over the series, one step at a time, and every position gives one example.

import numpy as np

def make_windows(series: np.ndarray, lookback: int, horizon: int):
    x, y = [], []
    for t in range(len(series) - lookback - horizon + 1):
        x.append(series[t : t + lookback])
        y.append(series[t + lookback : t + lookback + horizon])
    return np.array(x), np.array(y)

series = np.loadtxt("electricity_hourly.csv")   # shape (8760,)
x, y = make_windows(series, lookback=168, horizon=24)
print(x.shape, y.shape)   # (8569, 168) and (8569, 24)

A lookback of 168 hours (one week) captures both the daily and weekly cycles; a horizon of 24 hours matches the operational question "what will consumption be tomorrow?". The example count $N - L - H + 1$ is huge — 8 569 examples from a single year — which is why RNNs work well on this kind of data even with modest storage.

Where temporal leakage hides

A tabular split with train_test_split(shuffle=True) cannot be used on a sliding-window dataset. It draws examples uniformly, which means a training example centred on July 4th sits next to a test example centred on July 5th. The two windows share 167 of their 168 hours. The network sees the future during training in every practical sense, and its test error becomes meaningless.

The correct split is a cut in time: the last two months, say, are the test set; the two months before are validation; everything earlier is training. No window crosses the boundaries.

n = len(series)
train_end = int(0.7 * n)
val_end   = int(0.85 * n)

train = series[:train_end]
val   = series[train_end - 168 : val_end]     # keep enough lookback
test  = series[val_end - 168 :]

x_train, y_train = make_windows(train, 168, 24)
x_val,   y_val   = make_windows(val,   168, 24)
x_test,  y_test  = make_windows(test,  168, 24)

The extra -168 on validation and test starts is deliberate: without it the first test window would need 168 hours from the training set, or it would be discarded and the test set would shrink silently.

A single shuffled split ruins a whole recurrent project

The most common way to overestimate a sequence model is to accept the default shuffle=True of the split function. Test accuracy climbs to numbers no one can reproduce in production, and the mistake is only found weeks later when the model fails on truly unseen days.

Normalisation must not see the future either

The same leakage happens with any statistic computed on the whole series before splitting. Scaling the electricity to zero mean and unit variance across the entire year, then splitting, tells the training set the mean of the summer months it has not seen yet. The gap is small on stationary series and enormous on drifting ones.

The rule: fit the scaler on the training portion only, then apply it to validation and test. Module 9 formalises this as part of the batching pipeline; here it is enough to keep the reflex.

mu, sigma = train.mean(), train.std()
train_s = (train - mu) / sigma
val_s   = (val   - mu) / sigma
test_s  = (test  - mu) / sigma

Baselines that a fancy model must beat

Before writing the first LSTM layer, always compute a naive forecast: for a 24-hour horizon, predict that tomorrow will look exactly like today. On the electricity series that baseline is surprisingly hard to beat because the daily cycle dominates the signal. A recurrent network that does not beat it by a clear margin is not learning anything the series does not already say.

Write the baseline as one line before writing the model

y_pred = x[:, -horizon:] copies the last 24 hours of the lookback as the prediction. If a 300 000-parameter LSTM cannot beat that on your data, your problem is not the architecture — it is the data or the metric.

In summary

• A sequence is a table where order is a signal; a dense network sees each row as independent and cannot exploit autocorrelation unless the past is unrolled into columns.
• The sliding window turns a series into supervised examples characterised by a lookback and a horizon; one year of hourly data yields thousands of examples, which is why RNNs work with modest datasets.
• Temporal leakage is the silent killer of forecasting projects: shuffled splits, whole-series normalisation and overlapping windows all leak future information into training.
• Always compute a naive baseline — for a periodic series, "tomorrow equals today" — before judging any recurrent model.

Next module: the recurrent neuron itself, whose hidden state is exactly the mechanism a dense network lacks to carry information from one time step to the next.