# A/B Test Calculator

> Find out whether your test winner is real. Enter the visitors and conversions of two versions and the calculator gives each conversion rate, the uplift, a two-proportion z-test p-value, the confidence it implies and a 95% confidence interval for the difference, with a plain verdict on whether the result is statistically significant at 95%.

Free, runs in your browser, nothing is stored or sent anywhere.

## Your test

The starting values are an illustrative example, not a benchmark. Replace them with your own test’s numbers.

Visitors are the people who saw the version; conversions are the ones who did the thing you are measuring (an opt-in, an order, a click to the next step).

- **A: control**: 1000 visitors, 50 conversions (example)
- **B: variation**: 1000 visitors, 70 conversions (example)

Result: A conversion rate, B conversion rate, Difference (percentage points), Relative uplift, z-score, p-value (two-sided), Confidence, 95% confidence interval for the difference, and the verdict “Statistically significant at 95%” or “Not statistically significant at 95%”.

## What the A/B test calculator does

An A/B test shows two versions of a page to different visitors and compares how many of each group did the thing you wanted. The hard part is not running it but reading it: a version that converts 7% against 5% looks like a clear win, yet with a few hundred visitors each that gap appears by chance more often than people expect. An A/B test calculator tells you how likely that is.

This calculator runs a two-proportion z-test, the standard test for comparing two conversion rates. You enter the visitors and conversions of the control (A) and the variation (B). It returns each rate, the absolute and relative difference, the z-score, the two-sided p-value, the confidence that implies, a 95% confidence interval for the difference, and a plain verdict: statistically significant at 95%, or not.

Use it on any test where the outcome is yes or no per visitor: an opt-in, an order, a click through to the next step. It is not for averages such as order value or time on page, which need a different test.

## How the maths works

A two-proportion z-test, two-sided, with a pooled standard error for the test and an unpooled one for the interval.

1. **Conversion rates**: p₁ = conversions ÷ visitors for A, and p₂ for B. The absolute difference is p₂ − p₁ in percentage points; the relative uplift is (p₂ − p₁) ÷ p₁.
2. **Pooled rate and standard error**: Under the hypothesis that both versions convert the same, the best estimate of the shared rate is p = (conversions A + conversions B) ÷ (visitors A + visitors B). The standard error of the difference is √( p (1 − p) (1/n₁ + 1/n₂) ).
3. **z-score**: z = (p₂ − p₁) ÷ standard error. It says how many standard errors apart the two rates are. A z of about 1.96 or more in either direction is the usual threshold for 95%.
4. **p-value and confidence**: The two-sided p-value is 2 × (1 − Φ(|z|)), where Φ is the standard normal distribution, computed here with a numerical approximation of the error function accurate to seven decimal places. Confidence is 1 − p. The result is called significant at 95% when p is below 0.05.
5. **Confidence interval**: The 95% interval for the difference is (p₂ − p₁) ± 1.96 × √( p₁(1 − p₁)/n₁ + p₂(1 − p₂)/n₂ ). If it does not include zero, the result is significant at 95%; its width tells you how precisely you know the uplift.

- The test assumes each visitor saw one version and was counted once, which is how Vonza split tests work (each visitor keeps seeing the same version). It also assumes enough data for the normal approximation: as a rule of thumb, at least ten conversions and ten non-conversions in each version.
- Checking the result every day and stopping the moment it crosses 95% inflates the false-positive rate. Decide the sample size or the duration first, then read the result once.
- Significance is not size. A significant 2% relative uplift may not be worth a redesign; a non-significant 40% uplift may be worth testing longer.

## A worked example

The example the calculator opens with: 1,000 visitors on each version, 50 conversions on A and 70 on B. Illustrative figures, not a benchmark.

| Figure | Value |
| --- | --- |
| A conversion rate | 50 ÷ 1,000 = 5.00% |
| B conversion rate | 70 ÷ 1,000 = 7.00% |
| Difference | 2.00 percentage points, a 40% relative uplift |
| Pooled rate | 120 ÷ 2,000 = 6.00% |
| Standard error | √(0.06 × 0.94 × (1/1000 + 1/1000)) ≈ 0.01062 |
| z-score | 0.02 ÷ 0.01062 ≈ 1.88 |
| p-value (two-sided) | ≈ 0.060 |
| Confidence | ≈ 94% |
| 95% interval for the difference | ≈ −0.08 to +4.08 percentage points |
| Verdict | Not statistically significant at 95% |

A 40% uplift that is not significant surprises people. With 1,000 visitors a side, a 5% rate has enough noise that a 2-point gap appears by chance about 6% of the time. The interval shows the same thing: it just includes zero.

Change B to 80 conversions (8.00%) and the z-score rises to about 2.72, p to about 0.007, and the result is significant at 95% with an interval of roughly +0.9 to +5.1 points. More conversions, not a bigger gap, is what settled it.

If you need the test to finish sooner, test a bolder change (a different offer or page design, not a button colour) or send the test more traffic. Both move the numbers the test depends on.

## Split tests in Vonza Funnels

In the new Vonza Funnels, any funnel step can run a split test from the canvas: show a second version of the step’s page to a share of visitors (from 1% to 99%), each visitor keeps seeing the same version, and the step counts the metric that fits it, an opt-in, an order or a visit to the next step. Stop, resume or pick the winner with one click, and the losing page is removed.

Read the two versions’ visitors and conversions from the test and paste them into this calculator before you pick a winner. Split tests are on funnel steps; landing pages built in the website builder and standalone checkout pages do not have them.

- [Funnel conversion calculator](https://vonza.com/tools/funnel-conversion-calculator)
- [Vonza sales funnel builder](https://vonza.com/sales-funnel)
- [Best sales funnel builders](https://vonza.com/best/sales-funnel-builders)
- [Webinar funnel](https://vonza.com/webinar-funnel)

## A/B test calculator questions

### What does “statistically significant at 95%” mean?

It means the two-sided p-value is below 0.05: if the two versions really converted the same, a difference at least this large would appear by chance less than 5% of the time. It does not mean B is 95% likely to be better, and it says nothing about how big the real difference is; the confidence interval does that.

### How many visitors do I need for an A/B test?

It depends on the baseline rate and the smallest uplift you care about. Low rates and small uplifts need far more traffic: detecting a 10% relative uplift on a 2% rate takes tens of thousands of visitors a side, while a 40% uplift on a 10% rate may show in a couple of thousand. Type your expected numbers into the calculator to see what it would take, and plan the duration before you start.

### What is a p-value?

The probability of seeing a difference at least as large as the one observed if the two versions actually converted the same. A p-value of 0.03 means a 3% chance. Smaller is stronger evidence; 0.05 is the usual threshold for a 95% confidence verdict. This calculator reports the two-sided p-value, which does not assume in advance which version is better.

### Why is a 40% uplift not significant?

Because significance depends on the number of conversions, not only on the size of the gap. In the worked example, 50 against 70 conversions out of 1,000 each is a 40% relative uplift but only about 94% confidence; with 80 conversions on B it is significant. Fewer than a few dozen conversions per version is rarely enough.

### Can I use this for Vonza split tests?

Yes. A Vonza Funnels split test shows a second version of a funnel step’s page to a share of visitors and counts opt-ins, orders or visits to the next step; each visitor keeps seeing the same version. Read each version’s visitors and conversions from the test and enter them here before you pick the winner.

### What should I test first in a funnel?

The step with the biggest drop-off and the boldest change you can make there: the offer, the headline and the page design before colours and button text. Bigger changes produce bigger differences, and bigger differences reach significance with less traffic. Test one step at a time so the result is attributable.