A/B testing

A/B testing (also called split testing) is a controlled experiment that compares two versions of a single element to measure which performs better against a defined metric. One version is the control; the other is the variant. By showing version A to one group and version B to another, you isolate the impact of the change and make decisions based on data rather than intuition.

A/B tests work because they control for variables. When you change only one element—a call-to-action button colour, email subject line, landing page headline, or checkout flow—you can confidently attribute performance differences to that specific change. This removes noise from the equation. A 15% lift in conversion rate on a redesigned form is meaningful. A 15% lift from changing two things simultaneously is meaningless; you don't know which one caused it.

The practice has become standard in B2B marketing because it compounds. A 5% improvement to email open rates, 3% to click-through rates, and 7% to landing page conversions stacks across your entire pipeline. Over a year, these incremental gains become significant revenue gains. But they only work if you run proper tests, reach statistical significance, and move deliberately rather than changing everything at once.

Prevents expensive guesses

Without testing, marketing decisions rest on opinion, trends, or what competitors do. A redesigned homepage might look beautiful but underperform. An email with personalisation might get lower engagement than expected. Testing removes the guesswork and validates assumptions before rolling out changes across your entire audience.

Builds evidence for larger decisions

A single A/B test might improve conversion by 2%. But when you run dozens of tests across your funnel, each small improvement multiplies. The tests also generate internal credibility—stakeholders see the data and buy in to further optimisation work. This accelerates decision-making across product, design, and marketing.

Uncovers unexpected insights

A/B tests often reveal counter-intuitive results. The longer form might outperform the short one. The urgent copy might underperform the educational copy. Testing exposes what your actual audience responds to, not what you assumed they would.

Define your hypothesis and metric

Start by identifying one element to test and the outcome you're measuring. Don't test 'everything looks better'—test 'changing the button from blue to green will increase form completions by 5%'. The metric must be trackable: conversion rate, click-through rate, email open rate, or time on page.

Split your audience randomly

Divide your traffic or user base equally between control and variant. Randomisation prevents selection bias. If your high-intent users all see version B, you can't claim B is better—it's just attracting higher-intent visitors.

Run the test long enough

Reach statistical significance before concluding. A 10% lift from 50 clicks is noise. A 10% lift from 5000 clicks is signal. Most platforms require at least 100-200 conversions per variant before results are reliable.

Document and iterate

Record every test, winner, and insight. This creates institutional memory and prevents repeating failed experiments. Winning tests often become the baseline for the next test—continuous improvement compounds.

A B2B SaaS firm testing email subject lines to their database of 50,000 prospects found that 'Your team is losing £5,000 per week on X' significantly outperformed 'Learn how to improve X efficiency' (32% open rate vs 18%). The test was run to 5,000 people, well above the sample size needed for significance. The winning line was then used in all future campaigns to that segment.

A management consulting firm tested two headlines: 'Strategy consulting for growth-stage SaaS' vs 'Close your critical strategy gaps in 90 days'. The second variant converted at 8.2% compared to 6.1%. The specificity of the outcome ('close gaps') and the time constraint ('90 days') resonated more than the generic category positioning.

A fintech company tested CTA button colours on their checkout page. The contrasting orange button outperformed the muted grey button by 4.3%, a seemingly small lift. Applied across 2 million annual transactions, this translated to an additional £180,000 in revenue annually with zero product changes.