Designing experiments

Allocation, duration and power are decisions made before a test ships. These posts cover how to make them so the experiment can answer the question asked of it.

6 posts

18 May 20269 min read
10/90 vs 10/10 in A/B Testing: How to Split Users When Unequal Allocation Is Required
When business constraints limit A/B testing treatment exposure, analysts must choose between the statistical integrity of a balanced design (e.g., 10/10) and the speed of an unbalanced design (e.g., 10/90). The unbalanced 10/90 split, which uses full traffic to dramatically accelerate the experiment, comes with risks like inflated false positives from skewed metrics, Sample Ratio Mismatch, and resource contention bias. Read on as we examine this critical trade-off to help you balance speed with statistical validity.
Allon Korem
Chief Executive Officer
19 June 202510 min read
Randomization: the ABC’s of A/B Testing
Randomization is one of the cornerstones of AB testing. But why is randomization so important, and how can we achieve it? In this post, we'll address these two key questions. Because without proper randomization, no A/B test can be considered valid.
Allon Korem
Chief Executive Officer
8 May 20258 min read
You can have it all: Parallel Testing in A/B Testing
In this blog, we’ll explain why running multiple experiments simultaneously is not only feasible but also beneficial, explore its key advantages and potential challenges, and share best practices for successful implementation.
Allon Korem
Chief Executive Officer
25 November 20247 min read
When allocation point and exposure point differ
A common issue in A/B testing is when the allocation point does not align with the exposure point. Learn why it matters, how it impacts your a/b tests, and how to address it.
Allon Korem
Chief Executive Officer
18 August 20245 min read
Size matters: How to plan test duration when using CUPED
CUPED is a powerful technique that not only enhances the sensitivity of your tests but also helps in resource optimization by reducing the number of subjects needed for experiments.
Allon Korem
Chief Executive Officer
1 July 20248 min read
Four Ways to Improve Statistical Power in A/B Testing (without increasing test duration, duh)
In A/B testing, you often have to balance statistical power and how long the test takes. Learn how Allocation, Effect Size, CUPED & Binarization can help you.
Allon Korem
Chief Executive Officer

A/B Testing at Bell Statistics

Allocation, duration and power are decisions worth getting right before a test ships. We design experiments that answer the question asked of them.

See how we work on a/b testing →

Designing experiments

10/90 vs 10/10 in A/B Testing: How to Split Users When Unequal Allocation Is Required

Randomization: the ABC’s of A/B Testing

You can have it all: Parallel Testing in A/B Testing

When allocation point and exposure point differ

Size matters: How to plan test duration when using CUPED

Four Ways to Improve Statistical Power in A/B Testing (without increasing test duration, duh)

A/B Testing at Bell Statistics

Other topics