Synthetic Control Method

When a single state, country, or region implements a new policy or experiences a major event, how can we isolate its true impact? Traditional statistical methods often struggle with this "small n" problem. The Synthetic Control Method provides a powerful, intuitive solution by constructing a data-driven counterfactual—a "what-if" scenario that shows what would have happened to the treated unit in the absence of the intervention. It moves beyond simple comparisons by creating a tailored control group from a weighted combination of untreated units, making it the gold standard for evaluating policies when only one unit is treated.

The Core Idea: Building a Doppelgänger

At its heart, the synthetic control method is about creating a synthetic version of the treated unit. Imagine you want to study the economic impact of a tax reform in one specific country. You can't clone that country and run a parallel universe experiment. Instead, you find other similar countries that did not implement the reform. The key insight is that no single country may be a perfect match, but a weighted combination of several countries might be.

This weighted combination is your synthetic control. It is constructed to mirror the pre-treatment characteristics and outcome trajectory of the treated unit as closely as possible. The treatment effect is then estimated by comparing the post-treatment outcome of the actual unit to the outcome of its synthetic doppelgänger. The validity of the method hinges on the assumption that this synthetic control provides a reliable counterfactual path, an assumption rigorously tested through validation exercises.

Step 1: Defining the Donor Pool and Predictors

Your first task is to select the donor pool. This is the set of potential control units from which you will construct your synthetic counterpart. The choice is critical. Units in the donor pool should be similar to the treated unit in terms of the underlying processes that drive the outcome variable. For example, if studying a U.S. state-level policy, the donor pool would typically be other U.S. states that did not enact the policy. You must exclude any unit that experienced a similar shock or intervention to maintain a clean comparison.

Next, you select a set of predictor variables. These are characteristics measured in the pre-treatment period that are believed to predict the outcome of interest. Good predictors include:

• Lagged values of the outcome variable (e.g., GDP for the 5 years before the reform).
• Other covariates that influence the outcome (e.g., demographic composition, industrial makeup).

The method will find weights for the donor pool units that best replicate these predictor values for the treated unit.

Step 2: The Optimization Engine: Finding the Weights

This is where the mathematical machinery comes in. The algorithm finds a vector of weights, $w=(w_1,w_2,...,w_J)$, for the $J$ units in the donor pool. These weights are non-negative and sum to one, ensuring the synthetic control is a convex combination of the donor units (preventing extrapolation far beyond the observed data).

The weights are chosen to minimize the difference between the pre-treatment characteristics of the treated unit and the synthetic control. Formally, let $X_1$ be a $(k\times 1)$ vector of pre-treatment predictors for the treated unit, and $X_0$ be a $(k\times J)$ matrix of the same predictors for the donor pool units. The algorithm solves:

$$\underset{w}{\min }||X_1-X_{0}w||$$

subject to $w_j\ge 0$ and ${\sum }_{j=1}^J{w}_j=1$. The $||.||$ represents a measure of distance, often the root mean squared prediction error (RMSPE) over the pre-treatment outcome path. A good pre-treatment fit—where the synthetic control's trajectory closely overlaps with the treated unit's actual history—is the primary basis for credibility.

Step 3: Validation and Inference: Placebo and Permutation Tests

With the synthetic control built, you observe the post-treatment gap. But is the observed difference due to the treatment or just noise? Standard statistical inference doesn't apply because we have only one treated unit. The synthetic control method employs placebo tests (or permutation tests) to assess significance.

You run a series of in-space placebo tests. This involves applying the synthetic control method to every unit in the donor pool as if it were treated (even though it wasn't). You pretend each control unit enacted the policy at the same time and estimate a "placebo" effect for it. This generates a distribution of estimated effects for units where the true effect should be zero.

You then compare the effect estimated for your actual treated unit to this distribution of placebo effects. If the treated unit's effect is large relative to the placebo distribution (e.g., it lies in the extreme tail), it provides evidence that the observed effect is unlikely due to chance. A common visual check is to plot all the placebo post-treatment gaps alongside the real one; the real gap should stand out as an outlier.

Applications and a Classic Example

This method is uniquely suited for policy evaluation where randomization is impossible and only one or a few units are treated. Classic applications include:

• Evaluating the economic impact of German reunification on West Germany (the seminal study).
• Assessing the effect of California's Proposition 99 (a tobacco control program) on cigarette sales.
• Measuring the impact of a new corporate tax law in a specific country.
• Analyzing the effect of a catastrophic event, like an earthquake or terror attack, on a region's growth.

In the California tobacco case, researchers constructed a synthetic California from a weighted combination of other U.S. states. The synthetic control closely matched California's cigarette consumption trend before 1989. After the proposition passed, a persistent gap opened between real California (with steeper decline) and its synthetic counterpart, providing strong evidence for the policy's effectiveness.

Common Pitfalls

1. Poor Donor Pool Selection: Including units that are fundamentally different or that experienced related shocks will corrupt your synthetic control. The donor pool must be comprised of plausible alternative histories. Correction: Justify your donor pool selection with substantive knowledge of the context and test robustness by varying the pool.

1. Overfitting the Pre-Treatment Period: A perfect pre-treatment fit can sometimes be achieved by overfitting to noise rather than the signal. This produces a synthetic control that looks great historically but fails to provide a valid counterfactual. Correction: Use a long pre-treatment period relative to the number of predictors. Check fit on multiple pre-treatment periods and consider using a more parsimonious set of predictors.

1. Misinterpreting Placebo Tests: A significant placebo test does not prove causality; it only suggests the observed effect is unlikely under random assignment of an effect of that magnitude. Confounding shocks that uniquely affect the treated unit can still bias results. Correction: Use placebo tests as a necessary but not sufficient check. Combine them with narrative evidence and robustness checks to build a compelling case.

1. Ignoring Interpolation Bias: Because the method uses a convex combination, the synthetic control is an interpolation of donor units. If the treated unit is an extreme outlier, the method may be forced to choose a poor fit, leading to bias. Correction: Visually and statistically inspect the position of the treated unit within the donor pool's predictor space. If it lies far outside the convex hull, interpret results with extreme caution.

Summary

• The Synthetic Control Method constructs a data-driven counterfactual for a single treated unit by creating a weighted average of untreated donor units.
• Its validity depends on a strong pre-treatment fit, achieved through an optimization process that selects non-negative weights summing to one for the donor pool.
• Statistical inference is performed via permutation or placebo tests, where the method is applied to control units to create a distribution of effects against which the real effect is judged.
• It is the premier tool for policy evaluation in case study settings, such as assessing state-level laws or country-specific shocks, where randomized controlled trials are impossible.
• Critical steps involve careful donor pool selection, validation of the pre-treatment fit, and rigorous interpretation of placebo tests to avoid overconfidence in the results.