Bell Statistics

What is a synthetic control?

A synthetic control is a counterfactual built by weighting together untreated units so that their combined history closely matches the treated unit's own history before the intervention. The gap that opens up between the two after the intervention is the estimated effect.

Also called
synthetic control method, SCM, weighted counterfactual
Allon Korem

Written by Allon Korem

Chief Executive Officer

Last updated

In plain English

Difference-in-differences needs a comparison group whose trend already matches the treated group's. Often none exists — you rolled out in your three biggest cities, or in the one region where the media buy was available, and no untreated region looks anything like them. Synthetic control solves this by constructing the comparison instead of finding it: take all the untreated units, and find the weighted combination of them whose pre-intervention path most closely tracks the treated unit's.

The output is a single fictional unit — 0.4 of one city, 0.3 of another, 0.2 of a third — that behaved like the treated one for as long as you have data before the intervention. After the intervention, it continues to reflect whatever was happening to untreated markets, so the gap between the real unit and its synthetic twin is the estimated effect. The great practical virtue is that the pre-period fit is visible: you can plot the two lines, and everyone in the room can see whether the counterfactual is credible before any effect is quoted.

That transparency is also the method's own honesty check, and it enforces a discipline the alternatives lack. The weights are chosen only on pre-intervention data, so they cannot be tuned to produce a flattering answer. They are usually constrained to be non-negative and sum to one, which forbids extrapolation — the synthetic unit is a genuine blend of real units rather than an arithmetic fantasy — and typically produces a sparse solution using a handful of donors, so you can name which markets are standing in for yours.

It fits situations difference-in-differences handles badly: one or a few treated units, non-random selection into treatment, and a long pre-period. That is exactly the shape of most regional marketing questions, which is why it has become a standard tool for geo experiment analysis and for evaluating campaigns that ran somewhere specific for reasons unrelated to chance. It needs a decent donor pool of comparable untreated units and enough pre-period history — a couple of years of weekly data is comfortable, a few months usually is not.

Inference works differently and is worth understanding before presenting results. With one treated unit there is no conventional sampling distribution, so significance comes from placebo tests: apply the method to each untreated unit in turn, pretending it was treated, and see how unusual the real gap is against that distribution of fake gaps. If a third of the placebos show a gap as large as yours, you have found nothing, however clean the chart looks. Always check the pre-period fit too, since a synthetic control that tracked poorly before the intervention has no claim to track well after it.

The formula

A constrained optimisation over the pre-period, then a subtraction. The constraints are what keep the counterfactual honest.

The synthetic unit
Ŷ₀ₜ = Σⱼ wⱼ · Yⱼₜ, wⱼ ≥ 0, Σⱼ wⱼ = 1

Non-negative weights summing to one forbid extrapolation, so the counterfactual is a real blend of real markets rather than an arithmetic construction.

Choosing the weights
w* = argmin Σ_{t < T₀} ( Y₁ₜ − Σⱼ wⱼ·Yⱼₜ )²

Fitted on the pre-intervention window only. Nothing after T₀ touches the weights, which is why the method cannot be tuned towards a result.

The estimated effect
τ̂ₜ = Y₁ₜ − Ŷ₀ₜ for t ≥ T₀

A gap per period rather than a single number, so you can see whether the effect builds, holds or decays.

Placebo inference
p = rank of |τ̂| among all placebo effects / (J + 1)

With one treated unit there is no sampling distribution, so significance comes from how unusual the real gap is against fake ones. A donor pool of 20 gives a minimum p of about 0.05.

Worked example

A brand launches a sponsorship in one metro area — chosen because the team's flagship store is there, not at random. Weekly revenue per 100,000 population is available for 104 weeks before and 12 weeks after. Thirty-one untreated metros are available as donors.

Pre-period length
104 weeks
Donor pool
31 untreated metros
Selected weights
0.41 Metro B, 0.28 Metro K, 0.19 Metro F, 0.12 Metro R
Pre-period RMSE
£62 per 100k / week (1.4% of the mean)
Mean post-period gap
+£318 per 100k / week (+7.1%)
Placebo rank
1st of 32

A 7.1% lift, sustained across twelve weeks, with a placebo p-value of 1/32 = 0.031.

Note what could not have been done here with difference-in-differences: one treated metro, chosen for a reason directly related to its own performance, and no single untreated metro that resembles it. The synthetic blend of four donors tracked it to within 1.4% over two years, which is the evidence that the counterfactual is credible — and that number should always be quoted alongside the effect, because a poor pre-period fit invalidates everything after it. The placebo test is doing the inferential work: applying the same method to all 31 untreated metros produced no gap as large as the real one, which is what a p of 0.031 means here. With a donor pool of 31 that is the smallest p-value available, so a larger pool is the only route to stronger evidence.

Common misconceptions

A good pre-period fit means the estimate is valid.
It is necessary and not sufficient. A synthetic control can track beautifully and then diverge because something unrelated to the treatment hit the treated market — a store closure, a local shock, a competitor entering. The placebo distribution is what distinguishes a real effect from an ordinary amount of post-period drift.
We can add more donor units to improve the fit.
Fit improves and credibility often falls. Donors should be plausible comparators subject to the same underlying forces; including unrelated markets lets the optimiser assemble a curve-fitted blend that matches the past for no structural reason. A sparse set of sensible donors beats a dense set of convenient ones.
Synthetic control gives a confidence interval like a regression.
With a single treated unit there is no conventional sampling distribution to build one from. Inference comes from placebo tests, and the smallest achievable p-value is bounded by the size of the donor pool — twenty donors cannot produce anything below about 0.05 however large the effect.

Frequently asked questions

How much data does a synthetic control need?
A long pre-period and a reasonable donor pool. Two years of weekly observations before the intervention is comfortable; a few months rarely is, because the weights would be fitted on too little information to be stable. For donors, twenty or more untreated units is a sensible floor, both to give the optimiser room and because the placebo test's smallest possible p-value is set by the pool size.
When is synthetic control better than a randomised geo test?
It is not — it is what you use when a randomised test was not run. A geo experiment that randomises regions in advance buys its causal claim with design and needs no assumption about which markets are comparable. Synthetic control is the right tool for evaluating something that already happened in a specific place, which is a common situation and a weaker starting point.
Can I use synthetic control with several treated units?
Yes. The usual approach is to construct a separate synthetic control for each treated unit and then average the estimated gaps, which also lets you see whether the effect was consistent or driven by one market. Extensions such as the generalised synthetic control handle multiple treated units and staggered timing within a single model, and are worth using when the rollout was phased.

Related terms

  • Causal inference

    Estimating what an action caused by reconstructing what would have happened without it.

  • Difference-in-differences

    Subtract the untreated group's change from the treated group's — and everything rests on parallel trends.

  • Geo experiment

    Randomise regions instead of users — the way to test marketing that cannot be hidden from a person.

  • Incrementality

    The conversions that would not have happened anyway — and the gap between that and what platforms report.

  • Marketing mix modelling

    One regression across every channel, built on aggregate data — no tracking, and strong assumptions.

Calculate it

  • Correlation test

    Pearson r or Spearman rho, with the Fisher-z interval that says how little a small sample knows.

  • Two-sample t-test

    Compare the average of two independent groups — plan the sample size, then test the result.

  • Paired t-test

    Before-and-after or matched pairs — size the study on the difference SD, then test it.

Knowing the term is the easy part

Applying it to a live measurement problem is the part that goes wrong. If you are designing an experiment, reading a result you do not trust, or trying to work out what your marketing actually caused, that is the work we do.