Control group

Maintain an unchanged version in experiments to isolate the impact of your changes and prove causation rather than correlation with external factors.

Control group

Control group

definition

Introduction

A control group is a cohort of users or customers who do not receive the intervention being tested in an experiment. By comparing the control group to the test group, you can isolate the true impact of the change you're testing.

Types of control groups

  • No intervention: the control group continues with the existing experience
  • Placebo: receives a similar experience that should have no effect
  • Alternative intervention: receives a different change

Why it matters

B2B growth teams constantly test changes: new email subject lines, website copy variations, pricing structures. Without control groups, you can't tell which changes actually improved your metrics and which were coincidental.

Control groups are especially important for longer-duration tests and tests with high implementation costs.

How to apply it

Ensure your control group is truly comparable to your test group. If groups are mismatched, differences in outcomes might reflect group differences rather than the effect of your test.

Run your test for long enough to account for natural variation. Aim for at least 2-4 weeks for most B2B tests.

Email subject line testing

A B2B software company tested personalised subject lines with half their list. After four weeks, the personalised group had 18% open rates compared to 15% for the control.

Sales process change validation

A sales organisation assigned 20 reps to a new discovery framework and 20 to the existing process. After three months, the new process group had 68-day average deal length versus 71 days for control.

Landing page design testing

A consulting firm split traffic 50/50 between a new page design and the existing one. The new design converted 8% of visitors compared to 6% for the control group.

Keep learning

Growth leadership

How do you make all four engines work together instead of in isolation?

Explore playbooks

Data & dashboards

Data & dashboards

Build the dashboards and data pipelines that show your growth engines in one view so you can spot bottlenecks and make decisions in minutes, not meetings.

Growth team tools

Growth team tools

The wrong tools create friction. The right ones multiply your output without adding complexity. These are the tools I recommend for growth teams that move fast.

Review and plan next cycle

Review and plan next cycle

Analyse last cycle's results across all twelve metrics, identify the highest-leverage improvements, and set priorities that compound into the next period.

Revisit quarterly

Revisit quarterly

Pressure-test your strategy against market shifts, performance data, and team capacity so your direction stays relevant and ambitious.

Related books

No items found.

Related chapters

2

Creating strong hypotheses

Most experiments fail before they start because the hypothesis is vague or untestable. Learn how to write hypotheses that are specific enough to prove or disprove and tied to metrics that matter.

4

Analysing and acting on results

Statistical significance is just the beginning. Learn how to interpret results correctly, avoid false positives, and turn winning experiments into permanent improvements across your growth engines.

Wiki

Pirate metrics

Track your user journey through Acquisition, Activation, Retention, Referral, and Revenue to identify which stage constrains growth most.

Annual Recurring Revenue (ARR)

Track predictable yearly revenue from subscriptions to measure business scale and growth trajectory in B2B SaaS and recurring revenue models.

Event tracking

Capture specific user actions in your product or website to understand behaviour patterns and measure whether changes improve outcomes or create friction.

Growth hacking

Deploy fast, low-cost experiments to discover scalable acquisition and retention tactics, learning through iteration rather than big bets.

Sales tech stack

Assemble tools that manage pipeline, automate outreach, and track performance to help reps sell more efficiently and managers forecast accurately.

Growth engine

Build self-reinforcing systems across demand generation, funnel conversion, sales pipeline, and customer value that create continuous momentum.

Compound growth rate

Calculate your true growth trajectory by measuring the rate at which your business grows when gains build on previous gains over multiple periods.

Marketing stack

Organise the tools that capture leads, nurture prospects, and measure performance to automate repetitive work and connect customer data across systems.

Product-market fit

Achieve the state where your product solves a genuine, urgent problem for a defined market that's willing to pay and actively pulling your solution in.

Drip campaign

Send a series of scheduled emails that educate prospects over time to stay top-of-mind without overwhelming them with aggressive sales pitches.

Cohort analysis

Group customers by acquisition period to compare behaviour patterns and identify which acquisition channels and time periods produce the best long-term value.

Sample size

Calculate how many users you need in experiments to detect meaningful differences and avoid declaring winners prematurely based on insufficient data.

Contact management

Organise customer and prospect information to track relationships, communication history, and next steps without losing context or duplicating effort.

Braindump

Clear mental clutter by transferring all thoughts, tasks, and ideas onto paper or screen, creating space for focused work.

Total Addressable Market (TAM)

Estimate the maximum revenue opportunity if you captured 100% market share to size your opportunity and prioritise which markets to enter first.

Last-touch attribution

Assign full conversion credit to the final touchpoint before purchase to identify which channels close deals but miss earlier influences that started journeys.

Go-to-market strategy

Plan how you'll reach customers and generate revenue by choosing channels, pricing, and sales models that match your product and market reality.

Referral marketing

Turn satisfied customers into active promoters who systematically bring qualified prospects into your pipeline at near-zero acquisition cost.

Founder-led growth

Build distribution through your personal brand and network where your expertise and story attract customers who trust you before your company.

Competitive advantage

Identify what you do better or differently that competitors can't easily copy to defend margins and win customers consistently over time.