A counterfactual is what would have happened to a unit under the treatment it did not receive. It can never be observed for any individual, which is why causal inference is fundamentally a problem of estimating something that is missing by construction.
A user sees the new checkout and buys. Would they have bought under the old one? That question has an answer, and you cannot observe it — the user experienced one version, and the other outcome does not exist anywhere to be measured. That unobserved outcome is the counterfactual, and the gap it leaves is what makes causal inference a distinct discipline rather than an application of descriptive statistics.
In the potential outcomes framework each unit is thought of as carrying two values: Y(1), the outcome if treated, and Y(0), the outcome if not. The causal effect for that unit is the difference between them. You always observe exactly one and never both, which Holland named the fundamental problem of causal inference. No amount of data collection resolves it, because the missing value is missing by the structure of reality rather than by an oversight in measurement.
What rescues the situation is that averages are recoverable even when individual effects are not. If treatment is assigned at random, the treated and untreated groups are alike in every respect but the treatment, so the untreated group's average outcome is an unbiased estimate of what the treated group would have done untreated. That substitution is the entire logic of a control group, and it is why randomisation is the gold standard: it manufactures a credible counterfactual rather than assuming one.
When randomisation is impossible, every observational method is an argument about how to construct the counterfactual from data. Difference-in-differences builds it from a comparison group's trend. Synthetic control builds it from a weighted blend of untreated units. Propensity score matching builds it from individuals with similar observed characteristics. Each is a different claim about what the treated units would have done, and the assumption behind that claim is always the load-bearing part rather than the estimator.
The habit worth forming is to ask of any causal claim what the counterfactual is and why it is credible. "Revenue rose 12% after the launch" implies a counterfactual — that revenue would have been flat — which is usually false because of seasonality and everything else happening. Naming the counterfactual out loud makes the strength of a causal claim immediately assessable in a way that inspecting the statistics does not.
The notation is worth knowing because it makes the missing-data structure of the problem explicit.
Yᵢ(1) and Yᵢ(0)What unit i would do treated and untreated. Both exist conceptually; exactly one is observed.
τᵢ = Yᵢ(1) − Yᵢ(0)Never computable, because one term is always missing. The fundamental problem of causal inference.
Yᵢ = Dᵢ · Yᵢ(1) + (1 − Dᵢ) · Yᵢ(0)D is the treatment indicator. The observation rule that discards one potential outcome per unit.
E[ Y(0) | treated ] = E[ Y(0) | control ]The control group's average is a valid stand-in for the treated group's missing outcome — see the two-sample t-test calculator.
A retailer launches a loyalty programme to all customers at once and reports that enrolled customers spend 34% more than non-enrolled ones. A second analysis uses a randomised holdout to construct a credible counterfactual instead.
The randomised comparison finds a 7.3% effect. The naive comparison reports 34.2%, nearly five times larger.
The fourth row is where the naive analysis goes wrong, and stating it explicitly is what exposes the error. Assuming enrolled customers would have spent £307 without the programme means assuming they were just like the people who never enrolled — when in fact customers who join a loyalty scheme are the ones who already shop most. Most of the 34% gap is who they were, not what the programme did. The randomised version constructs the counterfactual properly: the control arm is comparable by design, so £371 is a credible estimate of what the treated group would have spent. Note that the treated arm's £398 is lower than the observational £412, because the randomised treated group includes light shoppers who would never have self-selected in. Both numbers moved, and the effect that survives is a real 7.3% — worth having, and not the transformative result the first analysis implied.
When your advertising efforts are up and sales are increasing, that’s great news, but it doesn't necessarily mean the ads are the reason. Here are three popular tools to measure the lift of a campaign.
Our “Geo Testing: Unlocking True Incrementality” webinar explored how teams can measure real-world impact when A/B testing isn’t possible - using geographic experiments and synthetic controls to reveal true incremental lift in marketing and product initiatives.
Applying it to a live measurement problem is the part that goes wrong. If you are designing an experiment, reading a result you do not trust, or trying to work out what your marketing actually caused, that is the work we do.