Hierarchical Time Series Reconciliation

Creating accurate forecasts for business units, product categories, or regional sales is challenging on its own. The complexity multiplies when you need these individual forecasts to roll up correctly into a coherent company-wide prediction. Hierarchical Time Series Reconciliation is the statistical process that ensures forecasts across different levels of a predefined hierarchy sum consistently, transforming a collection of potentially conflicting numbers into a single, reliable plan for decision-makers.

The Core Problem: Forecast Coherence in Hierarchies

A hierarchical time series is a collection of time series arranged in a tree-like aggregation structure. A classic business example is a geographic hierarchy: Total Company Sales = North America Sales + Europe Sales + Asia Sales, and each region further breaks down into country-level data. The fundamental challenge is that if you forecast each series independently—using powerful models like ARIMA or Exponential Smoothing—the forecasts will not sum correctly across the hierarchy. The bottom-level forecasts won't add up to the top-level forecast, and vice-versa, leading to confusion and poor planning. Reconciliation enforces this aggregation consistency, making the forecast "add up" both literally and figuratively for stakeholders.

Foundational Reconciliation Methods

There are three primary approaches to achieve coherence, each with different trade-offs regarding where forecast bias is introduced.

The bottom-up approach is the most intuitive method. You generate forecasts only for the series at the most disaggregated level (the "leaves" of the tree, like individual products or stores). Forecasts for all higher levels (categories, regions, total company) are derived by simply summing these base-level forecasts. This method preserves patterns at the granular level but can perform poorly if the base-level series are noisy or sparse. It places all forecasting effort and potential error at the bottom.

Conversely, the top-down approach starts by forecasting only the top-level, most aggregated series (e.g., total company revenue). These forecasts are then disaggregated down the hierarchy using historical proportions, or "allocation factors." For example, if North America historically contributes 50% of sales, its forecast is set to 50% of the total forecast. This method benefits from the relative stability and stronger signal of the top-level data but can miss shifts in the composition of the hierarchy, such as a fast-growing product line.

Optimal Reconciliation and Minimum Trace Reconciliation

Recognizing the limitations of simple top-down or bottom-up methods, modern optimal reconciliation provides a framework to combine forecasts from all levels of the hierarchy in a statistically efficient way. The core idea is to generate independent, possibly inaccurate, forecasts for every series in the hierarchy (called "base forecasts") and then adjust them minimally so they become coherent.

The mechanics rely on representing the hierarchy with a summing matrix $S$. If we have $m$ bottom-level series and $n$ total series in the hierarchy, $S$ is an $n\times m$ matrix of zeros and ones that defines how bottom series aggregate. All series in the hierarchy can be represented as $y_t=S{b}_t$, where $b_t$ is the vector of bottom-level series.

The optimal reconciled forecast ${\tilde{y}}_h$ is derived from the base forecasts ${\hat{y}}_h$ using a reconciliation matrix $P$: $${\tilde{y}}_h=SP{\hat{y}}_h.$$

The choice of $P$ defines the method. Minimum trace reconciliation, a leading optimal method, selects $P$ to minimize the sum of the variances of the reconciliation errors (the "trace" of the error covariance matrix). In essence, it finds the most likely coherent forecasts given the independent base forecasts and their estimated uncertainties. The formula for the reconciliation matrix under minimum trace is: $$P=(S^{\top }{W}_h^{-1}S{)}^{-1}{S}^{\top }{W}_h^{-1},$$ where $W_h$ is an estimate of the covariance matrix of the base forecast errors. This method optimally weights information from all levels of the hierarchy, often outperforming both top-down and bottom-up approaches in terms of overall forecast accuracy across different hierarchy levels.

Implementation and System Building

In practice, you implement these methods using specialized libraries. In R, the hts package provides a comprehensive suite for creating hierarchies and applying top-down, bottom-up, and minimum trace reconciliation. Its successor, the hierarchicalforecast package in Python and R, offers a more flexible, modern interface, integrating with popular forecasting libraries like statsmodels and scikit-learn and providing more advanced reconciliation options.

Building reconciled forecasting systems for business planning involves a standardized workflow:

1. Hierarchy Definition: Clearly define the aggregation structure (geographic, product, temporal).
2. Base Forecast Generation: Use appropriate models to create independent forecasts for all series. This step is model-agnostic.
3. Reconciliation: Apply a chosen reconciliation method (e.g., min_trace from hierarchicalforecast) to the base forecasts.
4. Evaluation and Deployment: Assess forecast accuracy at key levels (often both root and leaf levels are important) and integrate the coherent forecasts into planning systems.

Common Pitfalls

Ignoring the hierarchy entirely. Using only independent forecasts for each series you care about guarantees incoherence, leading to operational conflicts where department plans don't match divisional or corporate targets. Reconciliation is not an optional polishing step; it's a requirement for usable forecasts in a hierarchical organization.

Applying only simple top-down or bottom-up methods without evaluation. Each has significant weaknesses. Blindly using bottom-up with very sparse low-level data (e.g., new product sales) will produce terrible high-level forecasts. Assuming static historical proportions in a top-down method will miss market share shifts. Optimal reconciliation methods were developed precisely to mitigate these issues.

Failing to measure accuracy at multiple levels. A common mistake is tuning models and reconciliation only for the top-level forecast. A system might achieve great total company accuracy by wildly misallocating forecasts to business units, making the plans useless for managers. You must evaluate and potentially weight accuracy metrics across the hierarchy.

Mishandling temporal hierarchies. Hierarchies aren't just spatial. You can have a temporal hierarchy where monthly forecasts must sum to quarterly, which sum to annual forecasts. The same reconciliation principles apply, often leading to significant accuracy gains by incorporating multiple time aggregations.

Summary

• Hierarchical Time Series Reconciliation enforces aggregation consistency, ensuring forecasts for parts (like regions) correctly sum to forecasts for wholes (like the total company).
• Simple bottom-up (forecast leaves and sum) and top-down (forecast total and disaggregate) methods have clear biases; optimal reconciliation methods like minimum trace combine information from all levels statistically to minimize overall forecast error.
• Reconciliation requires generating independent "base forecasts" for all series first, then adjusting them using a structure like a summing matrix to make them coherent.
• Implement these techniques using dedicated libraries like hts or hierarchicalforecast, which are essential for building reconciled forecasting systems that produce reliable, actionable plans for business.
• Always evaluate forecast accuracy at multiple levels of the hierarchy and avoid the pitfall of seeking coherence at the expense of accuracy at the operational levels that drive business execution.