Digital marketing term
A/B Test
An A/B Test is an experiment method that shows two (or more) variants at the same time and compares which performs better using statistical evidence.
Detailed explanation
In an A/B test, the control (A) and variation (B) receive randomized traffic; you look for a meaningful difference on a primary metric (e.g., conversion rate, CTR). Successful tests need a clear hypothesis, sufficient sample size, correct measurement, and few variables changed at once.
Common mistakes include deciding too early, testing too many variants simultaneously, ignoring seasonality, and treating secondary metrics as primary. Bayesian or frequentist approaches may be preferred by different teams—what matters is applying the method consistently.
For searches like “what is A/B testing” and “how to run an A/B test,” this entry provides the framework. It relates to CRO for landing improvements, CTR/PPC for ad creative, and ROAS for revenue impact.
Frequently asked questions
- How long should an A/B test run?
- Duration depends on traffic and expected effect size. Ending early without statistical power can lead you to pick a false winner.
- What is the difference between A/B and multivariate testing?
- A/B usually compares a single change. Multivariate tests combinations of multiple elements and requires more traffic.
Related terms
Internal links for the topic cluster — read these concepts together.
- Conversion RateConversion Rate is the percentage of visitors who complete a predefined action relative to all visitors.
- Landing PageLanding Page: The destination page a visitor lands on after clicking an ad. This digigund glossary entry explains how the term is used in digital marketing.
- Dynamic Creative Optimization (DCO)Dynamic Creative Optimization is an automation technology that automatically combines and tests different image, headline, and copy components to show each user the best-performing ad combination.
- Goal CompletionGoal Completion is an analytics term for when a user completes a predefined action on a website, such as a purchase, form fill, or signup; it's a core measure of campaign success.