Control group

Maintain an unchanged version in experiments to isolate the impact of your changes and prove causation rather than correlation with external factors.

Control group

Control group

definition

Introduction

A control group is a cohort of users or customers who do not receive the intervention being tested in an experiment. By comparing the control group to the test group, you can isolate the true impact of the change you're testing.

Types of control groups

  • No intervention: the control group continues with the existing experience
  • Placebo: receives a similar experience that should have no effect
  • Alternative intervention: receives a different change

Why it matters

B2B growth teams constantly test changes: new email subject lines, website copy variations, pricing structures. Without control groups, you can't tell which changes actually improved your metrics and which were coincidental.

Control groups are especially important for longer-duration tests and tests with high implementation costs.

How to apply it

Ensure your control group is truly comparable to your test group. If groups are mismatched, differences in outcomes might reflect group differences rather than the effect of your test.

Run your test for long enough to account for natural variation. Aim for at least 2-4 weeks for most B2B tests.

Email subject line testing

A B2B software company tested personalised subject lines with half their list. After four weeks, the personalised group had 18% open rates compared to 15% for the control.

Sales process change validation

A sales organisation assigned 20 reps to a new discovery framework and 20 to the existing process. After three months, the new process group had 68-day average deal length versus 71 days for control.

Landing page design testing

A consulting firm split traffic 50/50 between a new page design and the existing one. The new design converted 8% of visitors compared to 6% for the control group.

Keep learning

Growth leadership

How do you make all four engines work together instead of in isolation?

Explore playbooks

Data & dashboards

Data & dashboards

Build the dashboards and data pipelines that show your growth engines in one view so you can spot bottlenecks and make decisions in minutes, not meetings.

Growth team tools

Growth team tools

The wrong tools create friction. The right ones multiply your output without adding complexity. These are the tools I recommend for growth teams that move fast.

Review and plan next cycle

Review and plan next cycle

Analyse last cycle's results across all twelve metrics, identify the highest-leverage improvements, and set priorities that compound into the next period.

Revisit quarterly

Revisit quarterly

Pressure-test your strategy against market shifts, performance data, and team capacity so your direction stays relevant and ambitious.

Related books

No items found.

Related chapters

2

Creating strong hypotheses

Most experiments fail before they start because the hypothesis is vague or untestable. Learn how to write hypotheses that are specific enough to prove or disprove and tied to metrics that matter.

4

Analysing and acting on results

Statistical significance is just the beginning. Learn how to interpret results correctly, avoid false positives, and turn winning experiments into permanent improvements across your growth engines.

Wiki

Event tracking

Capture specific user actions in your product or website to understand behaviour patterns and measure whether changes improve outcomes or create friction.

Growth drivers

Identify the fundamental factors that directly cause business expansion, concentrating resources on activities that generate measurable results.

Attribution model

Assign credit to marketing touchpoints that influence conversions to understand which channels work together and deserve budget in multi-touch journeys.

Annual Recurring Revenue (ARR)

Track predictable yearly revenue from subscriptions to measure business scale and growth trajectory in B2B SaaS and recurring revenue models.

Drip campaign

Send a series of scheduled emails that educate prospects over time to stay top-of-mind without overwhelming them with aggressive sales pitches.

Contact management

Organise customer and prospect information to track relationships, communication history, and next steps without losing context or duplicating effort.

Stakeholder Management

Navigate competing priorities and secure buy-in by systematically understanding, influencing, and aligning internal decision-makers toward shared goals.

Pareto Principle

Focus effort on the 20% of activities that drive 80% of results, systematically eliminating low-yield work to maximise output per hour invested.

Last-touch attribution

Assign full conversion credit to the final touchpoint before purchase to identify which channels close deals but miss earlier influences that started journeys.

Eisenhower Matrix

Prioritise tasks systematically by sorting them into urgent-important quadrants, focusing effort on high-impact activities.

Inbound Marketing

Attract prospects through valuable content that solves real problems, building trust and generating qualified leads who approach you.

Cookie

Store information in browsers to track user behaviour across visits and enable personalised experiences without requiring login for every interaction.

Positioning statement

Define how you're different from alternatives in a way that matters to customers to guide all messaging and ensure consistent market perception.

Total Addressable Market (TAM)

Estimate the maximum revenue opportunity if you captured 100% market share to size your opportunity and prioritise which markets to enter first.

Braindump

Clear mental clutter by transferring all thoughts, tasks, and ideas onto paper or screen, creating space for focused work.

Pipeline coverage

Calculate how much pipeline you need relative to quota to ensure you generate enough opportunities to hit revenue targets despite normal conversion rates.

Activity tracking

Log emails, calls, and meetings automatically to understand what drives deals forward and coach reps based on actual behaviour rather than guesswork.

Sample size

Calculate how many users you need in experiments to detect meaningful differences and avoid declaring winners prematurely based on insufficient data.

Hypothesis testing

Structure experiments around clear predictions to focus efforts on learning rather than random changes and make results easier to interpret afterward.

Sales tech stack

Assemble tools that manage pipeline, automate outreach, and track performance to help reps sell more efficiently and managers forecast accurately.