Lag Features, Rolling Windows, and Cyclical Encoding: Feature Engineering for Time-Series Models

Why time-series models need engineered features at all

A recurrent neural network like an LSTM can take a raw sequence of values and learn temporal structure internally, but a tree-based model such as XGBoost or a plain regression model has no built-in notion of "before" and "after" — it sees each row as an independent set of feature values with no memory of what came earlier. To use these fast, well-understood, easily-interpreted model families on time-series problems, the temporal structure has to be made explicit by turning it into ordinary columns: values from the past, summary statistics over recent windows, and encodings of where in a repeating cycle (hour of day, day of week) each observation falls. This is what feature engineering for time series is: converting sequence structure into a flat feature vector.

Lag features: giving the model direct access to the past

A lag feature is simply the value of the target (or another relevant variable) at some earlier point in time, attached as a new column to the current row — for instance, a column called lag_1h holding the value from one hour before, and lag_24h holding the value from exactly one day before. These are computed with a straightforward shift operation: df['lag_1h'] = df['value'].shift(1). When data covers multiple independent entities, such as separate sensors or stations, the shift has to be applied within each group separately (grouping by an ID column before shifting), otherwise a lag value could leak across entity boundaries and mix one station's history into another's row.

Choosing which lags to include is itself a modelling decision informed by the data's known periodicity: a series with strong daily and weekly seasonality typically benefits from lags at one hour, twenty-four hours, and one hundred sixty-eight hours (a week), because those are the points in the past most likely to resemble the current moment. Lags that are too short add little beyond what adjacent lags already capture, and lags chosen without reference to the data's actual cycle length often add noise rather than signal.

Rolling statistics: smoothing out noise into trend

Where a lag feature captures a single past value, a rolling (or moving-window) statistic summarises a whole span of past values into one number — a rolling mean, standard deviation, minimum, or maximum computed over, say, the trailing seven days. In pandas this is a one-liner: df['value'].rolling(window=168, min_periods=1).mean() for an hourly series with a weekly window. Rolling means smooth over short-term noise to reveal the underlying trend the model can key on; rolling standard deviations capture recent volatility, which itself can be predictive — a sensor with rising variance may be more likely to fail soon, independent of its raw mean level.

As with lag features, rolling statistics computed across multiple entities must respect group boundaries — a rolling mean for one station should never incorporate another station's readings — which in pandas means combining groupby with transform so the rolling calculation runs independently within each group while still returning a column aligned to the original row order.

Cyclical encoding: telling the model that 23:00 is close to 00:00

Calendar features like hour-of-day or month-of-year are naturally cyclical — hour 23 is followed by hour 0, not hour 24 — but represented as plain integers, a model has no way to know that 23 and 0 are adjacent; it just sees a large numeric gap between them. Cyclical encoding fixes this by mapping the value onto a circle using sine and cosine: df['hour_sin'] = np.sin(2 * np.pi * df['hour'] / 24) and the matching cosine column. Together, the sine and cosine columns preserve the true circular distance between any two hours — 23 and 0 end up close together in this two-dimensional encoding, exactly as they should be — something a single raw integer column cannot represent no matter how a tree-based model tries to split on it.

The same technique applies to day-of-week, day-of-year, or month, wherever the underlying quantity wraps around rather than running in a straight line. It is a small addition to a feature pipeline, but its absence is a common, easy-to-miss source of models that underperform specifically around cycle boundaries — midnight, year-end — without an obvious reason why.

Avoiding data leakage while engineering these features

Every one of these feature types carries a real risk of data leakage — accidentally letting information from the future influence a feature used to predict the past, which inflates validation performance in a way that will not hold up once the model is deployed and genuinely cannot see the future. The core discipline is to always split data by time before engineering any feature that touches a rolling window or statistic fit to the data, such as a scaler: fit any such transformation only on the training period, and apply it unchanged to the validation and test periods, never the reverse. Random train/test splits, which are standard practice for most tabular problems, are specifically the wrong choice for time series, because they let future rows end up in the training set and past rows in the test set, silently leaking information backward in time.

Rolling and lag features need particular care at the boundary between training and evaluation periods: a rolling mean computed at the very start of a test period will, by construction, reach back into training data for most of its window, which is usually fine since that data genuinely was available at that point in time — the leakage risk instead comes from computing any feature using data that would not yet have existed at the moment a real prediction was being made. Documenting exactly what each engineered feature means and how it was derived, and checking for suspiciously high correlation between engineered features and the target, are both practical checks that catch leakage before it reaches production.