What is a conversion lift study?

A conversion lift study is a genuine randomised experiment: the platform splits the eligible audience, shows ads to one group and withholds them from the other, and compares conversion rates. Because assignment is random, the difference is causal — this is a real measurement of incrementality rather than an attribution model, and it deserves more credit than it usually gets from people sceptical of platform-reported numbers.

Its advantages over a geo experiment are practical. It randomises individuals rather than markets, so the sample is enormous and the estimate is far more precise. It requires no media reorganisation. And the holdout is managed by the platform, so the mechanics are handled. For measuring one platform's contribution it is often the most precise tool available.

The limitations are all about the boundary. It measures the platform's incrementality relative to everything else you are doing, which means the result is conditional on the rest of your media mix. It cannot see cross-channel effects — the halo from a campaign there landing in search or direct, or the cannibalization of one channel by another. And it necessarily excludes anyone the platform could not have reached, so the population is the platform's addressable audience rather than your customer base.

Then there is the conflict of interest, which is real and should be handled proportionately rather than used to dismiss the method. The platform designs the study, defines eligibility, runs the randomisation and computes the result. That is not evidence of manipulation and it is a reason to check what is checkable: confirm the holdout size, ask for the confidence interval rather than accepting a point estimate, and where the numbers matter enough, validate against an independent geo test on the same channel.

The sensible position is that lift studies and geo tests are complements. Use a lift study for precise within-platform measurement and iteration; use a geo test when you need cross-channel effects, an independent read, or a channel the platform cannot hold out. Where both are run and agree, that agreement is worth more than either alone — and where they disagree, the geo test is generally measuring the wider quantity.

An advertiser runs a platform conversion lift study on a social channel and, in the same quarter, a geo test that switches the same channel off in matched markets. The two results are compared.

The last row reconciles them. When social was switched off in the geo markets, search conversions fell by 2.9% — some of the conversions the lift study counts as social's own were being handed to search in the wider picture, and switching social off removed them from both places. The lift study is precise and boundary-limited: it correctly measures social's incrementality relative to everything else running. The geo test is imprecise and complete: it measures what happens to total conversions, cross-channel effects included. For a decision about whether to keep the channel, the geo figure of 2.2 is the relevant one. For iterating on social creative and targeting week to week, the lift study's precision is far more useful. The intervals also tell a story worth respecting — the geo test's range of 2.4% to 13.2% is honest about how little a market-level design can pin down.