Time Series Feature Engineering

Time series data is ubiquitous in fields like finance, healthcare, and IoT, but raw temporal observations often lack the structure needed for machine learning models to uncover patterns. Feature engineering transforms this sequential data into informative predictors, bridging the gap between time domain complexity and model interpretability. Mastering these techniques is essential for building accurate forecasts, anomaly detectors, and predictive maintenance systems.

Foundational Feature Extraction Techniques

The simplest yet most powerful features you can create derive directly from the time index and past values. Lag features are created by shifting the time series by a specific number of time steps, allowing a model to reference past behavior. For instance, in predicting tomorrow's stock price, you might use the price from one, two, and seven days ago as features. This explicitly provides the model with historical context. Creating multiple lags—a lagged feature set—helps capture short-term and long-term dependencies.

Building on lags, rolling window statistics (or sliding window aggregates) compute summaries over a fixed, moving period. For a window size of $k$, at each time point $t$, you calculate metrics like the mean, standard deviation, minimum, or maximum of observations from $t-k+1$ to $t$. This smooths noise and highlights local trends. For example, a 7-day rolling average of daily sales reveals the underlying trend by dampening daily fluctuations. Crucially, you must avoid data leakage by ensuring the rolling window only uses past information relative to the prediction point.

In contrast, expanding window aggregates use all data from the start of the series up to the current time point, calculating cumulative statistics like the running mean or cumulative sum. These features capture the entire history's evolution and are useful for modeling growing totals or long-term baseline shifts. If you are tracking total users on a platform, the expanding sum of daily sign-ups directly provides that cumulative count as a feature. Both rolling and expanding windows turn a single point-in-time observation into a contextualized summary of recent or overall behavior.

Capturing Seasonality and Temporal Context

Time series patterns are often tied to the calendar. Calendar-based features extract elements from timestamps, such as hour of the day, day of the week, month, quarter, or indicators for weekends and holidays. In electricity load forecasting, the hour of day is a critical predictor due to daily usage cycles. These features are simple to compute and allow models to learn recurring schedules without complex transformations.

For more nuanced, periodic patterns, Fourier features are a mathematical method to capture seasonality. They decompose a seasonal pattern into a sum of sine and cosine waves. You create features like $\sin (2\pi ft)$ and $\cos (2\pi ft)$, where $f$ is a frequency (e.g., 1/24 for a daily cycle in hourly data). By including multiple harmonic frequencies, you can approximate complex, multi-period seasonality. This is particularly effective for long or multiple seasonal periods, like capturing both daily and weekly cycles in sensor data. Fourier features provide a compact, continuous representation that models can easily interpret.

Advanced Signal Processing Methods

When seasonality is not perfectly periodic or contains transient patterns, more sophisticated tools are needed. Wavelet transforms provide a multi-resolution analysis by decomposing a signal into components that localize both in time and frequency. Unlike Fourier analysis which assumes stationarity, wavelets can identify how frequencies change over time—ideal for analyzing irregular events or shifts in trend. Conceptually, you apply a set of wavelet functions at different scales (frequencies) and positions (times) to the series, generating coefficients that serve as features. These coefficients can reveal, for instance, a sudden spike in volatility in financial data or a short-duration anomaly in machine vibration signals.

While wavelet transforms offer deep insights, they generate a large number of coefficients. Feature selection becomes essential to avoid the curse of dimensionality. In practice, you might use statistical measures like variance or entropy on the wavelet coefficients to create a smaller set of informative features for your model. This approach allows you to retain the multi-scale information without overwhelming the learning algorithm.

Automation and Robust Feature Pipelines

Manually crafting features can be time-consuming. Automated feature generation with libraries like tsfresh (Time Series Feature Extraction on basis of Scalable Hypothesis tests) addresses this by systematically calculating hundreds of features—from simple statistics to complex measures—for each time series. tsfresh evaluates features like linear trend, entropy, or continuous wavelet transform coefficients, and can optionally perform feature selection based on statistical significance to the target variable. This is invaluable for exploratory analysis or when dealing with many parallel time series, such as sensor readings from multiple machines.

Real-world data is often messy. Effective feature pipelines must include strategies for handling missing timestamps and irregular time series. Irregular series have non-uniform time intervals between observations. Common tactics include:

• Resampling: Aggregating or interpolating data to a regular frequency (e.g., converting irregular transaction logs to hourly snapshots).
• Forward-filling or interpolation: For missing timestamps in a regular series, using the last known value or estimated values to maintain sequence integrity.
• Including time delta features: For irregular data, adding the time elapsed since the previous observation as a feature can provide crucial context about the sampling process.

Your pipeline should document the chosen imputation or resampling method, as it introduces assumptions that affect downstream features. For example, rolling windows on resampled data may create artifacts if not handled carefully.

Common Pitfalls

1. Data Leakage with Temporal Windows: Using future information in rolling or expanding window calculations is a critical error. Always ensure that for a prediction at time $t$, your window only uses data up to $t-1$. Implement rigorous time-based cross-validation to catch this.
2. Ignoring the Time Index as a Feature Source: Failing to extract calendar-based features leaves easy predictive power on the table. A model struggling with weekly seasonality might significantly improve simply by adding a "day of week" feature.
3. Over-Engineering with High-Dimensional Features: Techniques like wavelets or automated tools (tsfresh) can generate thousands of features, leading to overfitting. Always pair automated generation with robust feature selection, dimensionality reduction, or regularization in your model.
4. Ad-Hoc Handling of Missing Data: Dropping rows with missing timestamps or using a simple global mean for imputation can distort temporal dependencies. Choose a time-aware method like forward-fill or seasonal interpolation that respects the data's sequential nature.

Summary

• Lag features, rolling statistics, and expanding aggregates form the foundation, providing models with direct access to past values and localized summaries of series behavior.
• Calendar and Fourier features explicitly encode seasonal and cyclical patterns, transforming timestamps into powerful predictors for recurring events.
• Wavelet transforms enable multi-resolution analysis, capturing how frequencies in a signal change over time, which is vital for non-stationary or event-driven series.
• Automated feature generation (e.g., with tsfresh) systematizes exploration and creation, while robust pipeline strategies for missing data and irregular sampling ensure features are built on consistent, leakage-free data.
• Always validate feature engineering steps temporally to prevent data leakage and overfitting, ensuring your features generalize to future time periods.