How do you interpret results and confidence badges

CPP A/B Testing helps you compare variants using performance metrics and a confidence badge.

This article explains what you can see in results, what the confidence badges mean, and how baseline comparison works. If you want to learn where to find these views, go to How do you monitor tests in CPP A/B Testing

What you compare in results

In the CPP A/B Testing dashboard, each test shows a table of variants with metrics such as:

Impressions
Tap-Through Rate (TTR)
Conversion Rate (CR)
Cost Per Tap (CPT)
Average CPA
Confidence level badge

You can use these metrics to compare how each variant performs.

Note: If you want Apple’s official definitions for these performance metrics, see Reporting Options and Definitions.

Benchmark Summary

Each test shows a Benchmark Summary above the variant table, with the benchmark data from when the test was created (Impressions, TTR, CR, CPT, and CPA). Use it as a fixed baseline to judge whether a variant is improving on the starting point, not only how variants compare against each other.

Top Performer badge

Each performance metric (TTR, CR, Average CPT, and Average CPA) shows a Top Performer badge on the variant leading that metric so far. One variant can lead on TTR while another leads on Average CPA, so read the badges metric by metric rather than looking for a single overall winner.

Confidence level badges

Each variant shows a confidence level badge.

Green badge: the variant reached the desired confidence (statistically significant).
Gray badge: the test is still collecting data for that variant.

Baseline comparison

To help you understand performance differences, the dashboard supports baseline comparison.

Hover over metrics to see the baseline (default product page) for comparison.
Tooltips explain every metric in plain language.

Use table and chart views together

You can compare results in:

Table view for a snapshot across all variants
Chart view for trends over time

In chart view:

Each variant is represented by a line.
You can switch the metric shown using the metric dropdown (for example CR, TTR, CPT).

If there is no significant difference

A test can end without a clear winner. If no variant clearly outperforms the others:

Review metrics like impressions, CR, and TTR to choose the most promising option.
A lack of significant difference indicates the variants are likely to perform similarly over time.

The test does not guarantee one variant will outperform another. It is designed to ensure that, given your selected precision and confidence level, results are statistically sound and comparable.

Need more help?

If you have further questions on the process, contact your dedicated Customer Success Manager or contact the support team via live chat!