What is a conversion rate?

A conversion rate is conversions divided by a population, and the arithmetic is the least interesting part of it. What decides the number is three definitional choices: who counts in the denominator, what counts as a conversion, and over what window. Two teams at the same company routinely report different conversion rates for the same week and both are correct, because one is counting sessions and the other users, or one is attributing a purchase to the visit that started it and the other to the visit that completed it.

The denominator choice is the one that matters most for experimentation, and it interacts directly with how the test was randomised. If you randomised users, the natural denominator is users, and a per-session rate becomes a ratio metric with its own variance problem. There is also a substantive question about who should be in the denominator at all: everyone assigned, or only those who actually reached the surface being tested. Restricting to those exposed increases sensitivity considerably, and it must be done symmetrically across arms — filtering on a behaviour the treatment itself influenced is one of the cleanest ways to manufacture a false result.

Statistically it is a binary metric, which has two consequences worth knowing. Its variance is p(1 − p) and requires no separate estimate, which is why the standard readout is a two-proportion z-test. And it is far noisier than a tidy percentage suggests: at a 3% rate the coefficient of variation is about 5.7, so detecting a 5% relative improvement takes roughly 100,000 users per arm. Low base rates are expensive to test on, and no amount of careful instrumentation changes that.

The reporting distinction that causes the most confusion is relative versus absolute. Moving from 3.0% to 3.3% is a 10% relative lift and a 0.3 percentage point absolute one. Both are true; they support very different-sounding claims, and the relative version is what appears in slide decks. Always state which is which and give the absolute figure alongside, because the absolute number is the one that multiplies by traffic to produce money.

One caution about optimising it directly. Conversion rate is a share, so it rises when the numerator grows and equally when the denominator shrinks. A change that discourages casual browsers improves conversion rate while reducing total conversions, and the metric will report a clean win. That is why conversion rate belongs beside a volume metric rather than alone, and why guardrail metrics exist.

A team reports that a new landing page lifted conversion from 2.9% to 3.4%, a 17% relative improvement, on 60,000 visitors per arm. A reviewer asks two questions: what is in the denominator, and does total conversion volume agree.

The last row is where the story changes. The variant sent 5,700 fewer people to the page, most likely because a change upstream filtered out casual traffic, and a smaller denominator raises the rate mechanically. The rate improvement is real in the sense that the page converts its visitors better; it is not worth 17% to the business, because 5,700 potential customers never arrived. The honest headline is +6% conversions, and the diagnosis is that the upstream change needs its own examination — it may be beneficial, if those visitors were never going to buy, or costly, if it turned away people who would have. Two lessons generalise. Always report the volume alongside the rate, since a share can improve by shrinking. And when the denominators differ between arms, ask why before reading anything else; that asymmetry is often a [sample ratio mismatch](/glossary/sample-ratio-mismatch) rather than a finding.