Causal inference is the set of methods for estimating what an action actually caused, rather than what merely moved alongside it. It works by reconstructing the counterfactual — what would have happened without the action — from data, under assumptions that must be stated.
Every causal question is a question about something that did not happen. Did the campaign drive those sales — that is, would the sales have occurred anyway? Did the feature retain those users, or were they the kind who stay? The quantity you want is the difference between what happened and what would have happened otherwise, and the second half is never observed. Causal inference is the collection of strategies for constructing a credible stand-in for it.
The gold standard constructs it by force. In an A/B test, randomization creates a group that is identical to the treated group in expectation, so the control's outcome is a direct estimate of the counterfactual — and crucially it balances the variables nobody measured as well as the ones they did. When you can randomise, do; the assumptions are minimal and the argument is short.
When you cannot, the counterfactual has to be modelled, and each method is a different bet about what makes a valid substitute. Difference-in-differences uses an untreated group's change over the same period, betting that both would have moved in parallel. Synthetic control builds a weighted blend of untreated units that tracked the treated one before the intervention. Propensity score matching pairs treated units with observationally similar untreated ones, betting that similarity on measured variables implies similarity on everything relevant. Instrumental variables exploit something that shifts treatment without touching the outcome directly.
What they share is that the bet is an assumption rather than a fact, and it cannot be verified from the data that rely on it. Parallel trends before the intervention is evidence for parallel trends after it, not proof. Balance on observed covariates says nothing about unobserved ones. This is the honest difference between an experiment and an observational estimate: one buys its causal claim with design, the other with an argument. A good causal analysis therefore states its assumption prominently and tests how much the conclusion would change if it were wrong.
In marketing this is not an academic distinction, because most of the important questions cannot be randomised at the user level. Brand advertising reaches households, not cookies. Pricing changes are visible. A TV flight cannot be hidden from half the country. The practical toolkit is therefore geo experiments, which recover randomisation at the regional level, and marketing mix modelling, which models the counterfactual across all channels at once. Which to reach for is the subject of popular tools for accurate campaign impact analysis.
The potential outcomes notation makes the problem exact: two outcomes are defined for every unit and only one is ever observed.
Yᵢ(1), Yᵢ(0); observed Yᵢ = Dᵢ·Yᵢ(1) + (1 − Dᵢ)·Yᵢ(0)The fundamental problem of causal inference: for each unit exactly one of the two is observable, so the individual effect is never measurable.
ATE = E[ Y(1) − Y(0) ]Estimable even though no individual effect is, because averages need only one observation per unit and a valid comparison group.
( Y(1), Y(0) ) ⫫ D ⟹ ATE = E[Y | D=1] − E[Y | D=0]Independence between assignment and potential outcomes is what turns a difference in group means into a causal effect. Observational methods assume a conditional version of this.
( Y(1), Y(0) ) ⫫ D | XThe assumption behind matching and regression adjustment: within levels of the measured X, treatment is as good as random. Unverifiable, and false whenever an unmeasured confounder exists.
A retailer runs a four-week regional radio campaign in six markets. Sales in those markets rose 8.2% year on year. The CMO wants to know how much of that the radio caused, and three analyses are available.
The naive figure overstates the effect by roughly three and a half times. The two counterfactual methods agree on something between 2% and 3%.
The 8.2% is not wrong as a description; it is wrong as an answer, because it silently assumes the counterfactual was zero growth — and the untreated markets show that assumption was off by five points. Difference-in-differences fixes that by borrowing the untreated trend, at the cost of assuming the six treated markets would have moved in parallel. Synthetic control is more defensible here because the six markets were chosen for radio availability rather than at random, so they are probably not an average sample: it builds a weighted blend of untreated markets that tracked them closely for eighteen months, and the 0.7% pre-period fit is the evidence for that. The two methods agreeing is worth more than either alone, and the honest headline is 2-3% with the assumption named.
When your advertising efforts are up and sales are increasing, that’s great news, but it doesn't necessarily mean the ads are the reason. Here are three popular tools to measure the lift of a campaign.
Measuring complex user journeys is nearly an impossible task without the proper tools. Learn how MMM & Geo Tests can help advertisers measure the impact of their cross-marketing efforts.
When randomisation is off the table, the method and its assumptions are the whole argument. Choosing them, and stress-testing them, is what we do in Causal Inference Analysis