A/B test sample size calculator

Calculate how many users you need to avoid missing real effects. Already have results? Analyse them →

Experiment setup

Baseline conversion rate

Control group's conversion rate (0%–100%)

Minimum detectable effect (MDE)

Lift (0%–100% relative change)

Proportion in test group

Control group: 50%

Number of users per day

Optional

Test settings

α = 0.05
Power = 0.80
Two-sided

Total sample size

-⁠-⁠-⁠-

Per group

Control: -⁠-⁠-⁠-
Test: -⁠-⁠-⁠-

Results description

Planning a geo test? The Bell Experiments MCP server runs the Bell Statistics engine as tools your assistant can call — geo test design and measurement today, A/B testing next.

Get the MCP server

When to use it

“Statistics is the art of never having to say you are wrong…” We may not be able to avoid being wrong altogether, but as A/B testing analysts we can do our best to control the probability of it happening — which is what sizing an experiment in advance actually buys you.

Use this calculator when your outcome is binary — for example, converted or not, signed up or not, or active on day 7 — and you are comparing two independent groups.

Whether you call it an A/B test sample size calculator, a split test sample size calculator, or a conversion rate sample size calculator, this is the right tool for planning your next experiment.

Common use cases include testing a new checkout flow, a pricing page, a signup form, or any change where the outcome you care about is a simple yes or no. If you already have results and need to check for statistical significance, use our A/B test analysis calculator instead.

Assumptions

The calculation assumes:

Random assignment: observations are independently assigned to the two groups.
Independent observations: each randomized unit contributes one observation, and observations from different units are assumed to be independent.
A sufficiently large sample: the sample-size calculation uses a normal approximation to the binomial. For very small expected counts, use an exact method such as Fisher's test for analysis.

How the calculation works

This A/B test sample size calculator estimates the sample size needed to detect a specified difference between two proportions (conversion rates) at a chosen significance level, power, and allocation ratio. You enter the minimum detectable effect as a lift — the relative change over the baseline — as a percentage between 0% and 100%:

Relative effect (what you enter): relative effect = (p2 − p1)p1
The difference relative to the baseline proportion. This is the minimum detectable effect (MDE) field — enter 20 for a 20% increase over the baseline.
Absolute effect (what the lift implies): absolute effect = p2 − p1
The difference between the two proportions, in percentage points — not an input here, but what the lift you entered implies.
Sample size calculation: Two-sided: n1 = [ z1−α/2·√[p̄(1−p̄)(1 + 1/r)] + z1−β·√[p1(1−p1) + p2(1−p2)/r] ]²(p2 − p1)²
One-sided: n1 = [ z1−α·√[p̄(1−p̄)(1 + 1/r)] + z1−β·√[p1(1−p1) + p2(1−p2)/r] ]²(p2 − p1)²
n2 = r·n1
p̄ = p1 + r·p21 + r
The general two-proportion formula, for any allocation ratio r = n₂/n₁ and either a one- or two-sided test. The calculator asks for the proportion of users in the test group instead of r directly, and converts it: r = s/(1−s), where s = n₂/(n₁+n₂) is the test group's share — written s so it is not confused with the conversion rates p₁ and p₂. s = 50% gives an even split (r = 1), where p̄ reduces to the simple average of p₁ and p₂ — away from it, p̄ is the weighted average shown above, not the simple one. The critical value in the first term is z₁₋α for a one-sided test and z₁₋α/2 for a two-sided one; both use the pooled proportion under the null in that first term and each group's own variance under the alternative in the second.

Interpreting the output

This A/B test sample size calculator shows the number of visitors or users you need per variation and in total to reach your chosen statistical power for the selected minimum detectable effect (MDE).

If you run your test with this sample size and the true effect is at least as large as the MDE, your test has a probability of at least the selected power (for example, 80%) to detect a statistically significant result. Smaller effects can still reach statistical significance, but the chance of detecting them is lower than the specified power.

Tip: Stopping your A/B test before it reaches the required sample size lowers its power and makes it more likely to miss a real effect. If you also check results repeatedly and stop as soon as they look significant, you increase the risk of false positives. For reliable results, decide on the sample size before launching and run the test until you reach it.

Worked example

Suppose your baseline conversion rate is 4.0% and you want to detect an increase to 4.5%.

Baseline conversion rate: 4.0%
Minimum detectable effect (MDE): 12.5% (+0.5 percentage points)
Significance level (α): 0.05
Power (1 − β): 0.80
Proportion in test group: 50% (equal allocation)
Number of users per day: 6,000

The calculator requires approximately 25,551 users per group, or 51,102 in total. At 6,000 eligible users per day, this is approximately 9 days of traffic.

This sample size gives you an 80% probability of detecting a 0.5-percentage-point effect if that effect truly exists. Smaller effects may still be detected, but with lower probability.

When to use it

Use this calculator when your outcome is continuous — for example, revenue per user, session duration, or time on page — and you are comparing two independent groups.

Whether you call it a two-sample t-test sample size calculator, an A/B test sample size calculator for continuous metrics, or a mean comparison sample size calculator, this is the right tool for planning your next experiment.

Assumptions

The calculation assumes:

Random assignment: observations are independently assigned to the two groups.
Independent observations: each randomized unit contributes one observation, and observations from different units are assumed to be independent.
Approximately normal distribution: the calculation relies on the sampling distribution of the mean being approximately normal. This holds well for reasonably large samples even if the underlying metric is skewed (e.g., revenue), due to the Central Limit Theorem.
Known or estimable standard deviation: the calculation requires a reasonable estimate of the metric's variability in the control group, typically from historical data.
One standard deviation for both groups: the form takes a single standard deviation, so the design assumes the test group's spread matches the control's. If the change is likely to alter the spread — a pricing test often moves both the mean and the variance of revenue — use the larger of the two values you expect.
Welch's degrees of freedom by default: the design plans for a Welch's t-test, the analysis that does not assume equal variances. With one standard deviation this changes only the degrees of freedom, and on an even split it changes nothing; on an uneven split it makes the design slightly more conservative. Tick the equal-variances box to plan for the pooled t-test instead.

How the calculation works

The calculator estimates the sample size needed to detect a specified difference between two group means at a chosen significance level, power, and allocation ratio, using the t-distribution to account for sample-based variance estimation. You enter the minimum detectable effect as a lift — the relative change over the baseline — as a percentage between 0% and 100%:

Relative effect (what you enter): relative effect = (μ2 − μ1)μ1
The difference relative to the baseline mean. This is the minimum detectable effect (MDE) field — enter 10 for a 10% increase over the baseline.
Absolute effect (what the lift implies): absolute effect = μ2 − μ1
The difference between the two group means, in the same units as the metric — not an input here, but what the lift you entered implies.
The standardised effect: d = (μ2 − μ1)σ
Everything the design depends on is in this one number. A difference of 5 on a standard deviation of 120 and a difference of 2 minutes on one of 48 minutes are the same experiment and need the same sample. If you know the effect only in your own units, this is where the standard deviation enters.
Sample size calculation: λ = d√[n1n2(n1 + n2)]
Two-sided power = 1 − Ft(t1−α/2, df; df, λ)
One-sided power = 1 − Ft(t1−α, df; df, λ)
The calculator uses the noncentral t-distribution to compute the sample size. It finds the smallest sample size for which the power reaches the target value. The noncentrality parameter (λ) is how far the expected effect sits from the null, given the effect size and the two sample sizes — a larger λ means the effect is easier to detect, so power increases. Two-sided power splits the significance level between both tails and counts a significant result in the direction of the effect; the chance of one in the opposite direction is negligible at any useful power and is not counted — the same convention as R's power.t.test and as the binary and ratio tabs. One-sided power places the whole significance level in one tail, which is why a one-sided test reaches the same power with a smaller sample.
Fₜ(·; df, λ) is the cumulative distribution function of the noncentral t-distribution.
t_{p, df} is the p-th quantile of the central t-distribution.
σ is the standard deviation.
n₁ and n₂ are the sample sizes of the control and test groups.
Degrees of freedom — Welch by default: dfWelch = (1/n1 + 1/n2)²[ (1/n1)²/(n1−1) + (1/n2)²/(n2−1) ]
dfpooled = n1 + n2 − 2
The form collects one standard deviation, so both groups are planned at the same variance either way and only the degrees of freedom change. At n₁ = n₂ the Welch expression is exactly n₁ + n₂ − 2, so the two agree to the last digit on an even split; away from it Welch's df is smaller and the design is a little more conservative. Ticking the box uses the pooled figure.

Interpreting the output

This A/B test sample size calculator shows the number of visitors or users you need per variation and in total to reach your chosen statistical power for the selected minimum detectable effect (MDE).

If you run your test with this sample size and the true effect is at least as large as the MDE, your test has a probability of at least the selected power (for example, 80%) to detect a statistically significant result. Smaller effects can still reach statistical significance, but the chance of detecting them is lower than the specified power.

Tip: Stopping your A/B test before it reaches the required sample size lowers its power and makes it more likely to miss a real effect. If you also check results repeatedly and stop as soon as they look significant, you increase the risk of false positives. For reliable results, decide on the sample size before launching and run the test until you reach it.

Worked example

Suppose your baseline average revenue per user is 50, with a standard deviation of 120, and you want to detect an increase to 55 — at 2,000 eligible users a day.

Baseline mean: 50
Standard deviation: 120
Minimum detectable effect (MDE): 10% (+5 in your units)
Significance level (α): 0.05
Power (1 − β): 0.80
Proportion in test group: 50% (equal allocation)
Number of users per day: 2,000

9,043 users per group, or 18,086 in total. At 2,000 eligible users per day, this is approximately 10 days of traffic. The standardised effect is d = 0.042, which is small — and that is what a standard deviation more than twice the mean does to an experiment.

This sample size gives you an 80% probability of detecting a 5-unit effect if that effect truly exists. Smaller effects may still be detected, but with lower probability.

When to use it

Use this calculator when your outcome is a ratio metric — a value calculated as one quantity divided by another, measured per randomization unit — for example, revenue per session, clicks per impression, or items purchased per session.

Assumptions

The calculation assumes:

Random assignment: observations are independently assigned to the two groups.
Independent observations: each randomization unit contributes its own numerator/denominator pair, and different units are assumed to be independent of each other.
Approximately normal distribution: the calculation relies on the delta-method approximation for the ratio's variance, which is well-behaved for reasonably large samples.

How the calculation works

The calculator estimates the sample size needed to detect a specified difference between two group ratios at a chosen significance level, power, and allocation ratio, based on the baseline ratio and its standard deviation. If you don't already know these two values, the built-in helper calculator can derive them for you — either from uploaded raw data or from summary statistics (numerator mean and SD, denominator mean and SD, and their correlation) using the delta method. You enter the minimum detectable effect as a lift — the relative change over the baseline ratio — as a percentage between 0% and 100%:

Relative effect (what you enter): relative effect = (R2 − R1)R1
The difference relative to the baseline ratio. This is the minimum detectable effect (MDE) field — enter 2 for a 2% increase over the baseline, which is how a ratio-metric target is usually stated (“a 2% lift in revenue per session”).
Absolute effect (what the lift implies): absolute effect = R2 − R1
The difference between the two group ratios, on the ratio's own scale — not an input here, but what the lift you entered implies.
The delta-method variance, per randomisation unit: V = [ σ²X − 2R·ρ·σX·σY + R²·σ²Y ]μ²Y
R = μXμY
A ratio KPI divides two quantities that both vary from user to user — for example revenue per session, where X is a user's revenue and Y is their number of sessions. R is the ratio itself, and V is its variance per randomisation unit (per user, if you randomise users). The calculator uses sd = √V as the standard deviation in the sample-size formula below. Dividing by μ²_Y converts the result into the units of the ratio — per session rather than per user, in this example.
σX², the noise from the numerator — the variance you would get if every user had the same denominator, for example the same number of sessions.
R²·σY², the extra noise from the denominator — users don't all have the same denominator, and that adds uncertainty to the ratio. Multiplying by R² puts it in the same units as the numerator.
−2R·ρ·σX·σY, a reduction when the two move together — ρ is the correlation between each user's numerator and denominator. A user with 10 sessions and €20, and a user with 2 sessions and €4, both have exactly €2 per session: a user with more sessions who also has more revenue has a ratio that moves less than either quantity does, which is why this term is subtracted. The stronger the correlation, the larger the reduction.
X is each user's numerator total (e.g., revenue) with mean μX and standard deviation σX; Y is each user's denominator total (e.g., sessions) with mean μY and standard deviation σY.
Worked example: R = €105 = 2
Bracket = 12² − 2·2·0.6·12·4 + 2²·4² = 144 − 115.2 + 64 = 92.8
V = 92.85² = 3.71, sd = √3.71 ≈ 1.93
Suppose μ_X = €10 revenue, μ_Y = 5 sessions, σ_X = €12, σ_Y = 4, and ρ = 0.6. Ignoring the correlation — setting the middle term to zero — would give a bracket of 208 instead of 92.8, more than doubling the variance and making the test look less sensitive than it really is.
Sample size per group: n1 = (z1−α/tails + z1−β)²·sd²·(1 + 1/k)Δ²
The ordinary two-sample normal formula, with the ratio's own standard deviation in place of σ. Δ is the change in the ratio itself — a 2% lift in revenue per session, not a 2% lift in revenue. k = n₂/n₁, the same allocation ratio the binary and continuous tabs use; the calculator asks for the proportion of users in the test group instead and converts it: k = s/(1−s), where s = n₂/(n₁+n₂) is the test group's share.

Interpreting the output

This A/B test sample size calculator shows the number of visitors or users you need per variation and in total to reach your chosen statistical power for the selected minimum detectable effect (MDE).

If you run your test with this sample size and the true effect is at least as large as the MDE, your test has a probability of at least the selected power (for example, 80%) to detect a statistically significant result. Smaller effects can still reach statistical significance, but the chance of detecting them is lower than the specified power.

Tip: Stopping your A/B test before it reaches the required sample size lowers its power and makes it more likely to miss a real effect. If you also check results repeatedly and stop as soon as they look significant, you increase the risk of false positives. For reliable results, decide on the sample size before launching and run the test until you reach it.

Worked example

Suppose your baseline revenue-per-session ratio is $25, with a delta-method standard deviation of 42 per user, and you want to detect an increase to $27.50 — at 1,000 eligible users a day.

Baseline ratio: $25
Standard deviation: 42
Minimum detectable effect (MDE): 10% (+$2.50)
Significance level (α): 0.05
Power (1 − β): 0.80
Proportion in test group: 50% (equal allocation)
Number of users per day: 1,000

The calculator requires approximately 4,431 users per group, or 8,862 in total. At 1,000 eligible users per day, this is approximately 9 days of traffic. A standard deviation larger than the ratio itself is typical of revenue metrics, and it is why ratio tests need thousands of users rather than dozens.

This sample size gives you an 80% probability of detecting a $2.50 effect if that effect truly exists. Smaller effects may still be detected, but with lower probability.

Frequently asked questions

Which KPI type should I choose?

Ask what one user contributes. If it is a yes or no — converted, signed up, retained — that is binary. If it is a single number, like revenue or session length, that is continuous. If your metric divides one total by another and each user contributes several of the denominator — clicks per impression, revenue per session, where you randomised by user — that is a ratio metric, and it needs the delta-method correction the third tab applies. The test for the last one is whether different users contribute different numbers of denominator units; if they all contribute exactly one, the continuous tab is the right page.

How long should I run an A/B test?

Long enough to reach the sample size you calculated, and never fewer than one full business cycle — usually seven or fourteen days. Traffic composition swings hard by day of week, and a test that starts on a Tuesday and stops on a Friday has measured Tuesday-to-Friday users, not your users. If the required sample arrives in three days, keep running to a whole number of weeks anyway. If it will take four months, the test is not worth running as designed: raise the detectable effect, pick a metric closer to the change, or reduce variance rather than quietly lowering the power target.

What minimum detectable effect should I put in?

The smallest effect that would change what you do, not the effect you are hoping for. If a quarter of a point would not survive the engineering cost of shipping the change, do not pay for the traffic to detect it. Work backwards: estimate what a one-point effect is worth annually, compare that against the cost of the build and the opportunity cost of the traffic, and let the break-even number be your MDE. Choosing it from the answer you want is how tests end up sized for effects nobody would act on.

Can I stop the test early if it is already significant?

Not with a fixed-horizon design. Checking daily and stopping at the first p below 0.05 pushes the real false-positive rate to somewhere between 20% and 40%, depending on how often you look. Early crossings are common and mostly noise: the estimate is at its most volatile when the sample is smallest, and it is precisely the runs that overshoot that cross first. If you need the option to stop early, use a sequential or group-sequential design that spends the error budget across the looks, and fix that before launch rather than after the first promising day.

Should I use a one-sided test to save traffic?

It cuts the required sample by around 20%, but only honestly if you would take exactly the same action for a large negative result as for no result at all. In product work that is rarely true — a variant that hurts conversion by two points is information you very much want. The bigger risk is procedural: switching to one-sided after a two-sided test lands at p = 0.07 is not a statistical choice, it is a way of getting the answer you wanted. Decide before launch, write it down, and default to two-sided.

Does a 90/10 split cost me anything?

Yes, and much more than the intuition suggests. A 1:1 split extracts the most information per user. A 9:1 split needs close to three times the total traffic for the same power, because the power is governed by the smaller group. Skewed splits are legitimate when you are limiting blast radius on a risky release or reusing a large existing control, but they should be a deliberate purchase, not a leftover from a cautious ramp that nobody rebalanced.

Related calculators

A/B test analysis
Read a finished experiment for a binary, continuous or ratio KPI — the lift, its confidence interval and a p-value.
SRM check
Check whether your experiment actually split traffic the way you configured it — the first thing to run, before any metric.
One-proportion z-test
Test one observed rate against a fixed target — an SLA, a benchmark, a contractual floor.
MDE calculator
Given a fixed sample size, find the smallest effect your A/B test is powered to detect — for a binary, continuous or ratio KPI.

Terms on this page

A/B testing
A randomised experiment on live traffic — and randomisation is the only part that makes it causal.
Alpha spending
Divide the 5% across your planned looks — strict early, lenient at the end, and the schedule is fixed before you start.
Alternative hypothesis
The claim you are hoping for, and the one thing a significance test can never actually establish.
Always-valid inference
Statistics that stay honest under continuous monitoring — you may stop whenever you like, and it costs about a third more traffic.
Bayesian A/B testing
Friendlier output, the same underlying evidence — and it does not fix peeking, which is why most teams adopt it.
Cluster randomization
Assign the group, not the person — the remedy for interference, paid for in statistical power.
Control group
The counterfactual, made concrete — and the reason a before-and-after comparison is not an experiment.
Expected loss
How much a wrong decision would cost, in the units of the metric — the closest any of these numbers gets to a business answer.
Experimentation platform
Assignment, exposure, analysis and the guardrails — and the last of those is what separates a platform from a flag service.
Exposure point
Assigned is not the same as exposed — and the gap between them is where most of your effect size goes.
Feature flag
The switch experiments run on — and the reason a flag that changes mid-test quietly invalidates the result.
Geo unit
How finely you cut the map: more units means more power, and more spillover between them.

Browse the full statistics glossary

Design geo tests without leaving Claude

The Bell Experiments MCP server gives your assistant the Bell Statistics engine: ask it to pick test and control markets, estimate the budget or read the lift in plain language, and get the answer with the assumptions behind it. A/B testing is next.

See the MCP server

Talk to the people who build these for a living

We size, run and read experiments where the metric, the sample size and the decision rule are agreed before anyone looks at the data. A/B Testing

References

Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical Methods for Rates and Proportions (3rd ed.). Wiley.
Deng, A., Knoblich, U., & Lu, J. (2018). Applying the Delta Method in Metric Analytics: A Practical Guide with Novel Ideas. KDD '18, 233-242.
Kohavi, R., Tang, D., & Xu, Y. (2020). Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press.

A/B test sample size calculator

When to use it

Assumptions

How the calculation works

Interpreting the output

Worked example

When to use it

Assumptions

How the calculation works

Interpreting the output

Worked example

When to use it

Assumptions

How the calculation works

Interpreting the output

Worked example

Frequently asked questions

Related calculators

A/B test analysis

SRM check

One-proportion z-test

MDE calculator

Terms on this page

Further reading

Four Ways to Improve Statistical Power in A/B Testing (without increasing test duration, duh)

What to do when you encounter Sample Ratio Mismatch in A/B Testing

Statistically speaking: How to (properly) report A/B testing results

Design geo tests without leaving Claude

Talk to the people who build these for a living

References

A/B test sample size calculator

Calculator

When to use it

Assumptions

How the calculation works

Interpreting the output

Worked example

When to use it

Assumptions

How the calculation works

Interpreting the output

Worked example

When to use it

Assumptions

How the calculation works

Interpreting the output

Worked example

Frequently asked questions

Related calculators

A/B test analysis

SRM check

One-proportion z-test

MDE calculator

Terms on this page

Further reading

Four Ways to Improve Statistical Power in A/B Testing (without increasing test duration, duh)

What to do when you encounter Sample Ratio Mismatch in A/B Testing

Statistically speaking: How to (properly) report A/B testing results

Design geo tests without leaving Claude

Talk to the people who build these for a living

References