Keep CPP A/B tests healthy and fix issues
Some common issues can affect traffic and results. Use these insights to keep your custom product pages A/B tests stable and ensure the tests are not affected by external factors.
To better understand your test results and learn how to interpret them, go to How do you monitor tests in CPP A/B Testing.
1. Actions that can negatively affect test health
The following actions can negatively affect test health:
- Bid changes
- Budget changes
- Status changes of ad groups, campaigns, and keywords (KWs)
These changes can shift traffic and performance in ways that make variants harder to compare.
Keep the custom product page-related assets stable
Custom product page performance is directly related to screenshots. For healthier test results:
- Keep custom product page assignments stable.
- Do not change screenshots during the test.
Keep promo text stable
Promo text changes can affect ad group performance positively or negatively. For healthier test results, keep the promo text stable during the test.
2. Understand traffic changes
Some traffic loss is possible and expected during A/B testing.
Why traffic can dip during a test
During A/B testing, the system switches ad group statuses as part of running the test. Because Apple Ads needs time to reflect these status updates, you may observe a temporary dip in total traffic.
What to do
If you see a traffic dip:
- Check the test status in the dashboard (Running, Completed, or Stopped).
- Use the logs to review key events (Started, Completed, Stopped) and verify timing.
To learn where to find these views, go to Monitor tests in the dashboard.
3. Reduce test duration
Tes duration depends on traffic volume, number of variants, and your selected desired precision.
Ways to shorten a test
Increase desired precision (use a wider margin of error)
Raising the margin of error reduces the amount of data required. For example, moving from 1% desired precision to 3–5% can shorten the test duration. Keep in mind that his selection shortens the test but slightly reduces accuracy.
Note: The system caps desired precision at 10%.
Learn more about how desired precision affects sample size.
Decrease confidence level
Lowering the confidence level reduces the amount of data required to reach statistical significance. This can shorten the overall test duration, but it also slightly reduces the statistical certainty of the results.
Use higher-traffic ad groups
Ad groups with more daily taps and installs reach the required sample size faster. Selecting a higher-traffic ad group can shorten the duration.
Test fewer variants
Running fewer variants reduces the required traffic and time. For example, testing two variants instead of four significantly decreases required traffic and total test time.
Choose a longer switch interval (Switch tests)
In lower-traffic cases, daily or weekly switching can help balance exposure in a way that works with available traffic. To learn how switching periods are determined, learn how switch periods are chosen in CPP A/B Testing.
#For Apple Ads’ broader troubleshooting guidance on delivery fluctuations, you can also check Tips for Performance Issues.
Related links
- About CPP A/B Testing
- What are the requirements and limits for CPP A/B Testing
- How do you monitor and interpret tests in CPP A/B Testing
- Test duration and switching logic in CPP A/B Testing
- Frequently asked questions about CPP A/B Testing
Need more help?
If you have further questions on the process, contact your dedicated Customer Success Manager or contact the support team via live chat!