Control group

Maintain an unchanged version in experiments to isolate the impact of your changes and prove causation rather than correlation with external factors.

Control group

Control group

definition

Introduction

A control group is a cohort of users or customers who do not receive the intervention being tested in an experiment. By comparing the control group to the test group, you can isolate the true impact of the change you're testing.

Types of control groups

  • No intervention: the control group continues with the existing experience
  • Placebo: receives a similar experience that should have no effect
  • Alternative intervention: receives a different change

Why it matters

B2B growth teams constantly test changes: new email subject lines, website copy variations, pricing structures. Without control groups, you can't tell which changes actually improved your metrics and which were coincidental.

Control groups are especially important for longer-duration tests and tests with high implementation costs.

How to apply it

Ensure your control group is truly comparable to your test group. If groups are mismatched, differences in outcomes might reflect group differences rather than the effect of your test.

Run your test for long enough to account for natural variation. Aim for at least 2-4 weeks for most B2B tests.

Email subject line testing

A B2B software company tested personalised subject lines with half their list. After four weeks, the personalised group had 18% open rates compared to 15% for the control.

Sales process change validation

A sales organisation assigned 20 reps to a new discovery framework and 20 to the existing process. After three months, the new process group had 68-day average deal length versus 71 days for control.

Landing page design testing

A consulting firm split traffic 50/50 between a new page design and the existing one. The new design converted 8% of visitors compared to 6% for the control group.

Keep learning

Growth leadership

How do you make all four engines work together instead of in isolation?

Explore playbooks

Data & dashboards

Data & dashboards

Build the dashboards and data pipelines that show your growth engines in one view so you can spot bottlenecks and make decisions in minutes, not meetings.

Growth team tools

Growth team tools

The wrong tools create friction. The right ones multiply your output without adding complexity. These are the tools I recommend for growth teams that move fast.

Review and plan next cycle

Review and plan next cycle

Analyse last cycle's results across all twelve metrics, identify the highest-leverage improvements, and set priorities that compound into the next period.

Revisit quarterly

Revisit quarterly

Pressure-test your strategy against market shifts, performance data, and team capacity so your direction stays relevant and ambitious.

Related books

No items found.

Related chapters

2

Creating strong hypotheses

Most experiments fail before they start because the hypothesis is vague or untestable. Learn how to write hypotheses that are specific enough to prove or disprove and tied to metrics that matter.

4

Analysing and acting on results

Statistical significance is just the beginning. Learn how to interpret results correctly, avoid false positives, and turn winning experiments into permanent improvements across your growth engines.

Wiki

Inbound Marketing

Attract prospects through valuable content that solves real problems, building trust and generating qualified leads who approach you.

Multi-touch attribution

Distribute conversion credit across multiple touchpoints to recognise that customer journeys involve many interactions and channels working together.

Deep Work

Block extended time for cognitively demanding tasks requiring sustained focus, maximising valuable output whilst minimising shallow distractions.

Eisenhower Matrix

Prioritise tasks systematically by sorting them into urgent-important quadrants, focusing effort on high-impact activities.

Growth engine

Build self-reinforcing systems across demand generation, funnel conversion, sales pipeline, and customer value that create continuous momentum.

First-touch attribution

Credit the channel that introduced prospects to your brand to measure awareness efforts and understand which top-of-funnel activities start customer journeys.

Partner-led growth

Scale through partner relationships where other companies distribute your product to their customers in exchange for commissions or reciprocal value.

UTMs

Track campaign performance precisely by appending parameters to URLs that identify traffic sources, mediums, and campaigns in your analytics.

Founder-led growth

Build distribution through your personal brand and network where your expertise and story attract customers who trust you before your company.

Pipeline coverage

Calculate how much pipeline you need relative to quota to ensure you generate enough opportunities to hit revenue targets despite normal conversion rates.

Braindump

Clear mental clutter by transferring all thoughts, tasks, and ideas onto paper or screen, creating space for focused work.

Annual Recurring Revenue (ARR)

Track predictable yearly revenue from subscriptions to measure business scale and growth trajectory in B2B SaaS and recurring revenue models.

Stakeholder Management

Navigate competing priorities and secure buy-in by systematically understanding, influencing, and aligning internal decision-makers toward shared goals.

OMTM (One Metric That Matters)

Focus your entire organisation on the single metric that best predicts success at your current growth stage, avoiding distraction and misalignment.

Sales-led growth

Win customers through direct sales conversations where reps guide prospects from discovery to close with personalised solutions and relationship building.

Net Revenue Retention (NRR)

Track revenue growth from existing customers through expansion and contraction to prove your product delivers increasing value over time.

Data warehouse

Store raw data from all business systems in one place to run analyses and build reports that combine information across marketing, sales, and product.

Trigger

Define events that start automation workflows so the right message reaches people at the right moment based on their actual behaviour not arbitrary timing.

Marketing stack

Organise the tools that capture leads, nurture prospects, and measure performance to automate repetitive work and connect customer data across systems.

Churn rate

Measure the percentage of customers who stop paying to identify retention problems and calculate the true cost of growth in subscription businesses.