Free A/B Test Calculator

A 95% confidence level means there is only a 5% chance that the observed difference between your control and variant is due to random variation. It does NOT mean there's a 95% chance the variant is better — it means that if you ran this test 100 times with no real difference, you'd see a result this extreme only 5 times.

Run your test until you reach statistical significance (at least 95% confidence) AND have observed at least one full business cycle (typically 1-2 weeks). Ending a test early because it "looks like a winner" leads to false positives. Use the days-to-significance estimate in this calculator to plan ahead.

MDE is the smallest difference between control and variant that your test can reliably detect given the current sample size. A lower MDE requires more traffic. If your MDE is 2 percentage points, your test can detect a change from 5% to 7% but would likely miss a change from 5% to 5.5%.

This calculator compares two variants (A vs B). If you're testing multiple variants (A/B/C/D), you need to adjust for multiple comparisons to avoid inflating your false positive rate. A common approach is the Bonferroni correction: divide your target p-value by the number of comparisons.

The most common reason is insufficient traffic. Small differences need large sample sizes to detect reliably. If your baseline conversion rate is 3% and the true improvement is only 0.5 percentage points, you may need 15,000+ visitors per variant. Other reasons include high variance in your metric, or the change simply having no real effect.