In plain English
Traffic allocation is simply what proportion of units go to each arm. It looks like a free parameter and it is not: the precision of a comparison is limited by whichever arm is smaller, so an even split extracts the most information from a fixed amount of traffic. Every departure from 50/50 costs sample size, and the cost grows quickly.
The arithmetic is worth carrying around because the intuition is badly wrong. A 90/10 split does not cost 10% more traffic — it needs roughly 2.8 times the total of an even split to reach the same precision, because the standard error is dominated by the 10% arm. An 80/20 split needs about 1.6 times. Teams frequently choose uneven allocations to limit exposure to a risky change and are surprised when the test takes three times as long as planned.
There are legitimate reasons to accept that cost. A genuinely risky change might warrant limited exposure while error rates are watched. Capacity constraints can make a new backend unable to serve half the traffic. And a permanent holdout group is deliberately small because it is held indefinitely, where an even split would be enormously expensive. What is not a good reason is a vague sense that exposing fewer users to an untested variant is more careful — most tested changes do nothing, and the traffic in the smaller arm is what determines whether you find out.
The rule that matters operationally is that allocation must not change while an experiment runs. Raising a variant from 10% to 50% moves the assignment boundaries, so users cross between arms carrying their prior behaviour, and the arms end up differing in join date as well as in treatment. This is a common cause of sample ratio mismatch and it normally invalidates everything collected beforehand. If a staged rollout is wanted for safety, run it as a separate phase that concludes before the experiment starts.
With more than two arms the same logic applies. Equal allocation across all arms is optimal when every comparison matters equally; when several challengers are each being compared against one control, giving the control arm a somewhat larger share is slightly better, since it participates in every comparison. The gain is modest and the complexity is real, so equal splits remain the sensible default for most A/B/n tests.
The formula
One expression for the standard error explains the whole subject: it is dominated by the smaller arm, and everything else follows.
- Standard error of the difference
SE ∝ √( 1/n₁ + 1/n₂ )Minimised at n₁ = n₂ for a fixed total. This is why an even split is optimal rather than merely conventional.
- The cost of imbalance
relative n = 1 / ( 4 · p · (1 − p) )p is the smaller share. 50/50 gives 1.0; 80/20 gives 1.56; 90/10 gives 2.78 — see the sample size calculator.
- Multi-arm optimum
n_control / n_variant ≈ √k, for k challengers against one controlA modest gain over equal splits, since control appears in every comparison. Rarely worth the complexity.
- Why it must not change mid-test
moving boundaries reassigns units already measuredArms then differ in composition and join date, which no analysis separates from the treatment effect.
Worked example
A team plans a test on a 4.5% conversion rate, wanting to detect a 6% relative improvement at 80% power. They are considering three allocations: an even split, 80/20 to limit exposure, and 90/10 because the change touches billing.
- Required per-arm at 50/50
- 58,900 each — 117,800 total
- 80/20 total needed
- 184,000 (1.56×)
- 80/20 smaller arm
- 36,800
- 90/10 total needed
- 327,500 (2.78×)
- 90/10 smaller arm
- 32,750
- Available weekly traffic
- 60,000
The even split finishes in two weeks. The 90/10 split needs five and a half weeks for exactly the same conclusion.
The three-and-a-half extra weeks buy nothing statistically — the same effect, the same power, the same conclusion — and they are the price of exposing 10% rather than 50% of users to the variant. Whether that is worth paying depends entirely on how bad a broken billing flow would be, which is a real consideration and should be argued explicitly rather than assumed. The alternative worth putting on the table is a short staged rollout at 10% purely to watch error rates, concluded within a day or two, followed by an even-split experiment that starts fresh. That gets both the safety and the two-week runtime, at the cost of a slightly longer overall timeline and the discipline of not treating the rollout phase as data. What should not happen is starting at 90/10, watching for a couple of days, and then raising the allocation — which is the intuitive move and invalidates everything collected up to that point.
Common misconceptions
- דA 90/10 split only costs about 10% more traffic.”
- It costs roughly 178% more. Precision is governed by the smaller arm, so a 10% arm limits the comparison regardless of how large the other side is. Reaching the same power needs about 2.8 times the total traffic of an even split, which is usually a much larger delay than teams expect when they choose the allocation.
- דSending less traffic to the variant is the cautious choice.”
- It is cautious about exposure and reckless about duration, since the test runs far longer and the change stays unresolved. If the concern is that the variant might be harmful, a short staged rollout with error monitoring answers that in a day, after which an even-split experiment answers whether it helps.
- דYou can ramp the allocation up once the variant looks safe.”
- Not within a running experiment. Changing the split moves the assignment boundaries, so users cross arms carrying prior behaviour, and the arms end up differing in when their members joined. It is a common cause of sample ratio mismatch and generally invalidates the data collected before the change.
