Control group

Maintain an unchanged version in experiments to isolate the impact of your changes and prove causation rather than correlation with external factors.

Control group

Control group

definition

Introduction

A control group is a cohort of users or customers who do not receive the intervention being tested in an experiment. By comparing the control group to the test group, you can isolate the true impact of the change you're testing.

Types of control groups

  • No intervention: the control group continues with the existing experience
  • Placebo: receives a similar experience that should have no effect
  • Alternative intervention: receives a different change

Why it matters

B2B growth teams constantly test changes: new email subject lines, website copy variations, pricing structures. Without control groups, you can't tell which changes actually improved your metrics and which were coincidental.

Control groups are especially important for longer-duration tests and tests with high implementation costs.

How to apply it

Ensure your control group is truly comparable to your test group. If groups are mismatched, differences in outcomes might reflect group differences rather than the effect of your test.

Run your test for long enough to account for natural variation. Aim for at least 2-4 weeks for most B2B tests.

Email subject line testing

A B2B software company tested personalised subject lines with half their list. After four weeks, the personalised group had 18% open rates compared to 15% for the control.

Sales process change validation

A sales organisation assigned 20 reps to a new discovery framework and 20 to the existing process. After three months, the new process group had 68-day average deal length versus 71 days for control.

Landing page design testing

A consulting firm split traffic 50/50 between a new page design and the existing one. The new design converted 8% of visitors compared to 6% for the control group.

Keep learning

Growth leadership

How do you make all four engines work together instead of in isolation?

Explore playbooks

Data & dashboards

Data & dashboards

Build the dashboards and data pipelines that show your growth engines in one view so you can spot bottlenecks and make decisions in minutes, not meetings.

Growth team tools

Growth team tools

The wrong tools create friction. The right ones multiply your output without adding complexity. These are the tools I recommend for growth teams that move fast.

Review and plan next cycle

Review and plan next cycle

Analyse last cycle's results across all twelve metrics, identify the highest-leverage improvements, and set priorities that compound into the next period.

Revisit quarterly

Revisit quarterly

Pressure-test your strategy against market shifts, performance data, and team capacity so your direction stays relevant and ambitious.

Related books

No items found.

Related chapters

2

Creating strong hypotheses

Most experiments fail before they start because the hypothesis is vague or untestable. Learn how to write hypotheses that are specific enough to prove or disprove and tied to metrics that matter.

4

Analysing and acting on results

Statistical significance is just the beginning. Learn how to interpret results correctly, avoid false positives, and turn winning experiments into permanent improvements across your growth engines.

Wiki

Objectives and Key Results (OKRs)

Set ambitious goals and measurable outcomes that cascade through your organisation, creating alignment and accountability for strategic priorities.

Positioning statement

Define how you're different from alternatives in a way that matters to customers to guide all messaging and ensure consistent market perception.

Cookie

Store information in browsers to track user behaviour across visits and enable personalised experiences without requiring login for every interaction.

Attribution model

Assign credit to marketing touchpoints that influence conversions to understand which channels work together and deserve budget in multi-touch journeys.

Deep Work

Block extended time for cognitively demanding tasks requiring sustained focus, maximising valuable output whilst minimising shallow distractions.

OMTM (One Metric That Matters)

Focus your entire organisation on the single metric that best predicts success at your current growth stage, avoiding distraction and misalignment.

Growth plateau

Diagnose and break through stagnation by identifying which business mechanisms have reached capacity and require new approaches.

Product-led growth

Drive acquisition and expansion through product experience where users discover value before sales conversations and upgrade based on usage.

Total Addressable Market (TAM)

Estimate the maximum revenue opportunity if you captured 100% market share to size your opportunity and prioritise which markets to enter first.

Lead velocity rate

Measure the month-over-month growth in qualified leads to predict future revenue and catch pipeline problems before they impact revenue three months later.

Buyer persona

Document your ideal customer's role, goals, and challenges to tailor messaging and prioritise features that solve real problems they actually pay for.

Customer Acquisition Cost (CAC)

Calculate the total cost of winning a new customer to evaluate marketing efficiency and ensure sustainable unit economics across all channels.

Integration

Connect tools so data flows automatically between systems to eliminate manual entry, keep records current, and enable sophisticated workflows across platforms.

Deal stage

Define pipeline progression steps to standardise how reps advance opportunities and give managers visibility into where deals stall or convert unexpectedly.

Compound growth rate

Calculate your true growth trajectory by measuring the rate at which your business grows when gains build on previous gains over multiple periods.

Referral marketing

Turn satisfied customers into active promoters who systematically bring qualified prospects into your pipeline at near-zero acquisition cost.

Pareto Principle

Focus effort on the 20% of activities that drive 80% of results, systematically eliminating low-yield work to maximise output per hour invested.

Prioritisation

Systematically rank projects and opportunities using objective frameworks, ensuring scarce resources flow to highest-impact work.

Event tracking

Capture specific user actions in your product or website to understand behaviour patterns and measure whether changes improve outcomes or create friction.

Annual Recurring Revenue (ARR)

Track predictable yearly revenue from subscriptions to measure business scale and growth trajectory in B2B SaaS and recurring revenue models.