What is cannibalization?

Promote a product on the homepage and its sales rise. Whether that is growth depends entirely on where the sales came from. If customers who would have bought something else bought this instead, the product's metric improved and the business gained nothing — demand moved rather than increased. That is cannibalization, and it is invisible to any measurement scoped to the thing that improved.

It appears in three recognisable places. In product experiments, a promoted item or surface wins at the expense of others: a bigger recommendation module lifts recommendation clicks and takes them from search. In media, a channel takes credit for conversions another channel would have delivered — branded search is the classic case, capturing purchases from people already intent on buying, which is why its measured ROAS is spectacular and its incrementality often close to zero. In pricing and merchandising, a discounted tier pulls customers down from a higher-margin one.

What makes it dangerous is that it is not a measurement error. Every individual number is correct. The promoted product really did sell more; the channel really did drive those conversions in the sense that its ad was the last thing touched. The failure is in scope — the metric was defined narrowly enough that substitution falls outside it, so the experiment is answering a question nobody meant to ask.

The fix is structural rather than statistical: measure at the level where substitution is contained. If demand can move between products, the metric is total revenue rather than the promoted product's revenue. If it can move between channels, no channel-level measurement will do and the answer requires a geo experiment or a model that sees all channels at once. The general rule is that the metric has to be at least as wide as the substitution you are worried about, and widening it costs sensitivity — total revenue is far noisier than one product's revenue, which is exactly why teams are tempted to measure narrowly.

Its opposite is the halo effect, where a change lifts things outside its own scope rather than draining them. Both are the same measurement failure — a metric scoped more narrowly than the effect — and both are reasons that a portfolio of individually successful experiments can add up to a flat quarter. When that pattern appears, cannibalization is the first thing to look for.

A retailer tests a homepage banner promoting its own-brand range. The experiment is judged on own-brand revenue per visitor, and the team also has total revenue per visitor available as a guardrail. Both are measured across 210,000 visitors per arm.

Judged on its stated metric this is a 28% win and one of the year's best results. Judged on total revenue it is roughly nothing. The interesting wrinkle is the margin columns: shifting £0.69 of revenue from 19% margin to 42% margin adds about £0.16 of gross profit per visitor even with no revenue growth at all, so the change may well be worth shipping — on a completely different rationale from the one the experiment was set up to test. That is the useful lesson. Cannibalization does not automatically mean a change is bad; it means the metric was too narrow to evaluate it, and the real decision needs a wider frame. What would have been indefensible is reporting +28.4% as incremental growth. Note the cost of honesty here: total revenue is noisy enough that even a genuine 1.7% lift is not resolvable at 210,000 visitors per arm.