Sample ratio mismatch, outliers and uplift that evaporates in production are all diagnosable. These posts cover how to tell which problem you actually have.
A/B testing analysts often encounter skewed distributions for different KPIs. Learn why outliers can be problematic, explore the challenges of identifying and handling them, and get practical guidelines for dealing with them effectively.
Do you check for Sample Ratio Mismatch (SRM) in every test? Learn why checking for SRM is crucial, explore common reasons it occurs, and learn how to detect, diagnose, and address it.
Understanding why the uplift seen in A/B tests often differs from real-world outcomes is essential. Explore key reasons behind this discrepancy and insights on how to manage expectations effectively.
Sample ratio mismatch, outliers and uplift that evaporates in production are diagnosable. We find out which one you have.
See how we work on a/b testing →