When allocation point and exposure point differ
A common issue in A/B testing is when the allocation point does not align with the exposure point. Learn why it matters, how it impacts your a/b tests, and how to address it.
What is the local average treatment effect?
The LATE is the effect among compliers — the units whose treatment status actually responds to being assigned. It is what an experiment with imperfect uptake identifies, and it says nothing about those who would have taken the treatment regardless or refused it either way.
LATE
In plain English
When assignment does not fully determine treatment — a prompt some users ignore, a feature on a page many never visit — the population splits into four types. Always-takers use it whatever they were assigned. Never-takers refuse regardless. Compliers use it when assigned and not otherwise. Defiers do the opposite, and are usually assumed away. The LATE is the average effect among compliers alone.
That is a genuinely narrow quantity and it is the honest one, because compliers are the only group whose treatment status the experiment actually changed. For always-takers and never-takers the assignment did nothing, so the data contains no information about what treatment does to them. The LATE does not describe them because it cannot.
The standard route to it is an encouragement design: randomise a nudge to adopt rather than adoption itself, keeping uptake voluntary while assignment stays random. The estimate is then the intent-to-treat effect divided by the difference in take-up rates between arms — the Wald estimator, which is instrumental variables in its simplest form. Assignment is the instrument, and the division scales the diluted ITT back up to the group that responded.
Its main practical weakness is that the complier population is defined by the instrument rather than by anything you can identify in advance. Compliers under an email prompt are different people from compliers under an in-product banner, so the two encouragements identify effects on different groups and the estimates need not agree. There is also no way to point at the compliers in your data — the type is unobservable per unit, which makes the LATE hard to describe to anyone who wants to know who it applies to.
For product work it is most useful as a check on ambition. If a prompt lifts adoption from 11% to 34% and the LATE is large, pushing harder on adoption has clear value. If the LATE is small, the people you can persuade are not the people it helps, which is an important and unwelcome finding that the ATT among voluntary adopters would have concealed.
The formula
The estimator is a ratio, and its shape explains both what the LATE is and why the take-up gap has to be substantial.
- The estimand
LATE = E[ Y(1) − Y(0) | complier ]Only the units whose treatment responds to assignment. Always-takers and never-takers contribute nothing.
- The Wald estimator
LATE = ITT effect / ( take-up_assigned − take-up_unassigned )The diluted ITT divided by the share who actually complied — scales it back up to the responders.
- Why a weak instrument hurts
a small denominator inflates both the estimate and its standard errorA prompt lifting adoption from 11% to 14% gives a very imprecise LATE — see the sample size calculator.
- The assumption ruling out defiers
monotonicity: assignment never reduces take-up for anyoneUsually plausible for an encouragement, and untestable.
Worked example
A savings app randomises an in-product prompt encouraging users to set up an automatic transfer. The prompt is the instrument; setting up the transfer is the treatment; the outcome is money saved over six months.
The prompt raised savings by £41 across everyone assigned, which corresponds to £178 among the 23% it actually persuaded.
The three numbers describe three populations and only one of them is wrong. The £520 naive comparison is inflated by selection — people who set up automatic transfers unprompted were already savers — and is not a causal quantity at all. The ITT of £41 is correct and is the right figure for deciding whether to show the prompt, since it reflects what happens when you deploy it to everyone including the majority who ignore it. The LATE of £178 is correct and describes what the transfer feature does for someone a prompt can move, which is the number to use when deciding whether to invest in making the feature better. Note the caveat on generalising it: compliers under this in-product prompt are not the same people a marketing email would move, so a different encouragement would identify a different group and could produce a different LATE without either being wrong.
Common misconceptions
- “The LATE is the effect on the people who used the feature.”
- It is the effect on compliers — those whose usage responded to the assignment. Always-takers used it regardless and are excluded, even though they are among the users. That distinction matters because always-takers are often the most engaged users and would likely show the largest effect.
- “The LATE generalises to anyone who might adopt the feature.”
- It is specific to the compliers for the instrument you used. A different encouragement moves different people and identifies a different effect. That is not a defect of the method so much as a fact about what the experiment could learn: it can only speak about the units its assignment actually moved.
- “Dividing by the take-up rate is a correction that always works.”
- It requires the instrument to affect the outcome only through take-up, and to move take-up substantially. A prompt that also changes behaviour directly — by reminding people about saving in general — violates the exclusion restriction. And a weak instrument makes the denominator small, inflating both the estimate and its uncertainty.
Frequently asked questions
Can I identify which users are compliers?
Not individually. Compliance type is defined by how a unit would behave under both assignments, and you only observe one, so the type is unobservable per unit exactly as a counterfactual is. You can estimate the size of each group from take-up rates in both arms, and describe compliers on average using their observable characteristics, but you cannot label a specific user.
Should I report the LATE or the intent-to-treat effect?
Both, for different questions. The ITT is what happens if you deploy the encouragement — it includes everyone who ignores it, and it is the right basis for deciding whether to ship the prompt. The LATE is what the underlying treatment does for someone it reaches, which is the right basis for deciding whether to invest in the treatment itself. Reporting only the LATE overstates what deployment delivers.
What happens if the encouragement barely moves take-up?
The estimate becomes unreliable. Dividing by a small take-up difference inflates both the point estimate and its standard error, so the LATE can be enormous and meaningless. A useful rule is that the encouragement should shift take-up by at least ten percentage points; below that, invest in a stronger prompt before running the analysis rather than trying to rescue it afterwards.
Calculate it
Knowing the term is the easy part
Applying it to a live measurement problem is the part that goes wrong. If you are designing an experiment, reading a result you do not trust, or trying to work out what your marketing actually caused, that is the work we do.