Global Control Group: Theory and Practice

What if marketing doesn’t actually work — and we just don’t notice it? To understand whether a company’s marketing strategy is truly effective or if the budget is being wasted, global control groups are used. This is a helpful tool that allows you to distinguish real impact from mere coincidence.

In this article, Denis Barov, lead systems analyst at Altcraft, explains how to measure the effectiveness and results of a marketing strategy, how global control groups work, why they are needed, how to use them, and what insights they can provide.

What is a Control Group

A control group is a part of the customer base that is deliberately excluded from marketing communications for a certain period of time. This is done for the sake of a clean experiment — to objectively evaluate whether the marketing strategy as a whole is working and if it really influences customer behavior.

The target audience is divided into two parts:

• Main (experimental) group — receives all planned marketing activities: emails, push notifications, promotions, personalized offers.

• Control group — does not receive any messages under the studied strategy. This is the so-called “clean” group: it’s not affected by direct marketing, though it may encounter the campaign via other channels — such as a website banner, in-store ad, or flyer on the street.

After some time, the marketer analyzes key metrics — conversion rate, revenue, number of orders — and compares the behavior of the two groups. If the indicators are nearly identical, the current strategy is ineffective and provides no significant uplift.

Why does business need this? Traditional metrics — conversions, clicks, sales — don’t always show the full picture. For example, a marketer might mistakenly attribute sales growth to the latest ad channel (last-click attribution model), even though the customer was already going to buy without additional prompts. Control groups help avoid such false conclusions.

Difference Between Global and Local Control Groups

Depending on the objective, there are two types of control groups: global and local. Both approaches help analyze results, but in different contexts and with different exposure durations. Here’s the difference.

A global control group operates over a long period — weeks, months, or even a full year — and is excluded from all marketing communications. It’s used to measure the overall impact of marketing. The group is regularly updated (e.g., monthly) and may be rotational — the participants can change.

A local control group is created for a specific campaign or A/B test. For instance, when launching a new promotion, a portion of the audience is excluded to measure the specific effect of that campaign.

Key Steps for Working with GCG

1. Define the experiment goal

Before starting with a global control group, clearly define the experiment's goal: what changes or hypotheses are being tested (e.g., the impact of a new ad campaign on purchase volume). Predefine the key metrics to measure success: conversion rate, average order value, frequency or number of purchases, revenue per customer, etc.

2. Group formation

A portion of clients from the entire customer base or a specific segment is randomly selected to form the control group. Typically, this is 5–10% of the audience, sometimes more, but usually no more than 20–25%. For example, if the database contains 100,000 users, a 5% control group would include 5,000 clients. In some cases, the group size may vary depending on the experiment’s goals.

3. Exclude from communications

Clients in the control group are fully excluded from all marketing activities across all channels — emails, SMS, push, ads, personalized banners, promotions, etc. Only transactional or mandatory communications are allowed (e.g., payment confirmations, tickets, security alerts).

4. Isolation period

The control group remains isolated from marketing for a certain period. This duration is set by the marketer — for example, a month, a quarter, or the full length of a specific campaign. In practice, 1–3 months is optimal for most purposes — long enough to obtain reliable results, but not so long that customers forget about the brand.

5. Conduct marketing activities

While the control group is on pause, standard marketing activities are carried out with the rest of the audience (the main group). These may include email newsletters, SMS campaigns, push notifications, personalized banner displays, loyalty program launches, and so on.

6. Collect and analyze results

After the test period ends, the key moment arrives — comparing the results. You measure the same metrics in both the main and control groups. In most cases, the following indicators are examined:

• Conversion (CR%) — the percentage of customers who completed a target action (purchase, service order, registration, etc.).

• Average Order Value (AOV) — the average purchase amount.

• Revenue per customer — the income generated from each customer. In MoEngage studies, for example, the control group brought in $15 per customer, while the active group brought in $24 — clearly indicating a positive impact from the campaigns.

• Number of orders/sales — the total number of transactions in each group.

• Engagement — page views, clicks, time spent in the app, email opens, subscriptions.

• Retention — repeat purchases or visits.

The control group’s metrics show how customers behave without any marketing influence. By comparing these indicators with the main group, you can understand how much your marketing impacted the result.

For example, if the conversion rate in the control group (without marketing) is 5%, and in the main group (with marketing) it’s 12%, this means marketing activities increased conversion by 7 percentage points. That’s a good result. But if the difference is small (e.g., 15% in the control group vs. 16% in the main group), then marketing had almost no impact on customers, and the strategy needs improvement.

7. Decision-Making

When the experiment is complete and the data is collected, marketers assess whether the campaign was successful. For example, if revenue in the main group is 10% higher than in the control group and the difference is statistically significant, it means marketing generated a +10% increase in sales, and you can estimate the approximate ROI. Based on the analysis, further actions are taken:

• Scale successful activities to the entire audience. The control group may also receive the missed offers later, if relevant.

• Adjust or cancel ineffective campaigns. For example, you can shut down a channel that didn’t perform to avoid wasting budget.

• Draw conclusions by segments. Perhaps the campaign produced growth in one audience segment but not in another. In that case, future marketing budgets can be redirected to where there was an effect.

How to Select a Control Group

Representativeness. The control group should closely resemble the main group in key characteristics such as demographics, behavior, purchase history, and other important factors. This ensures that changes in behavior are due to the experiment itself, not external influences.

Randomization. Participants are assigned to groups randomly — either manually or using an algorithm. This helps avoid bias in the results and eliminates human influence.

Stratification. Sometimes the audience is pre-divided into homogeneous subgroups (strata) based on important criteria — for example, purchase frequency, region, or device type. Then, within each stratum, a control subset is randomly selected. This ensures the groups are as comparable as possible.

Group size. The control group should be smaller than the main group — typically 10–25% of the total audience. This maintains broad marketing coverage while still collecting enough data for analysis. The key requirement is statistical significance: the sample must be large enough to detect real differences in metrics rather than random fluctuations. Typically, at least 1,000 people are recommended, depending on the audience size and experiment goals.

To check whether your group is large enough, you can use online statistical significance calculators. These help you calculate the required sample size based on the expected difference in metrics and the acceptable margin of error.

Conditions and Limitations of GCG

Key Metrics and Conversion Calculation

Analysis using control groups helps accurately measure the impact of marketing on key business indicators. The main metric in such experiments is the conversion rate (CR%), which shows the percentage of customers who completed a target action (e.g., purchase, registration, subscription).

In addition to CR%, derivative metrics are often measured as well:

• Incremental CR — the increase in conversion rate resulting from the campaign.

• Uplift% — relative growth, calculated as (CRmain - CRcontrol) / CR_control × 100%.

• Segmented conversion — you can calculate CR for different customer categories (new vs. returning, male vs. female, etc.) within each group.

How to calculate conversion? What should the rate be? Read more in this article.

Control groups allow you to accurately measure at which stage of the funnel a campaign increases conversion. In analytics, CR is often tracked step by step: email open → click-through → cart → purchase. Comparing this with the control group shows where the campaign influences behavior and where it doesn’t. This makes it possible to precisely improve the stages that are limiting growth. Even a small increase in conversion — for example, from 2% to 2.2% — can lead to a significant revenue boost across a large audience.

How to Tell if the Campaign Worked: Calculating the Result

Let’s say you sent a mailing with a personal 10% discount to a base of 20,000 clients. Another 2,000 people were placed in a global control group — they did not receive the message and were unaware of the discount.

Results:

• In the main group: 2,400 orders → CR = 12%
• In the control group: 160 orders → CR = 8%

Now let’s calculate effectiveness:

1. Absolute increase (Incremental CR):

12% – 8% = +4 percentage points — this is how much the conversion rate increased due to the campaign.

2. Relative increase (Uplift):

(12% – 8%) / 8% × 100% = +50% — this is how much marketing improved the result in percentage terms.

So, out of the 2,400 orders in the main group, about 1,600 would have happened without the mailing, and the remaining 800 orders were driven by marketing. This is the incremental effect, proven through the GCG experiment.

Why Statistical Significance Matters

The difference between groups may be random, especially when the sample size is small. That’s why it’s important to check how statistically reliable the difference is.
In our example, the groups are large, and the difference of 100 buyers is most likely significant. But if the groups were smaller, the difference might have occurred by chance. That’s why experts often use statistical tests (e.g., chi-square test, t-test) or ready-made A/B test calculators to verify that the p-value is below the acceptable threshold (usually 0.05).

Other Metrics When Analyzing via GCG

Why a Control Group Is Needed

Global control groups solve several key tasks in marketing analytics:

Accurate measurement of marketing effectiveness. The main goal of a GCG is to objectively assess whether marketing has an actual impact. Without a control group, we compare “before and after” results or look at trends over time, but there’s always a question: what if sales increased on their own? GCG provides a parallel universe — as if marketing never happened.

Marketing budget optimization. When it’s clear which campaigns truly deliver results, the budget can be redistributed more wisely. For example, if a regular SMS campaign shows no sales increase compared to the control group, it might be worth reducing its frequency or switching to more effective emails or messengers. The budget spent on those ineffective SMS messages can be redirected to more impactful tools — like retargeting or website improvements.

Precise segmentation and targeting. Using GCG is closely tied to working with audience segments. First, when forming a control group, you usually ensure it is representative — meaning it mirrors the rest of the base in key traits (e.g., mix of new and returning customers, geography, activity level, etc.). This encourages thoughtful segmentation from the start. Second, experiment results may differ by segment: perhaps in one city the mailing increased sales by 15%, and in another by only 5%; or returning customers responded, but new ones didn’t.

These insights help refine segmentation — highlighting the groups most responsive to marketing and focusing efforts on them. Over time, this leads to personalization: once you know who truly benefits from marketing, you can stop bothering those it doesn’t affect or design a different approach for them. For example, if the control group shows that customers aged 50+ barely react to push notifications (with purchases similar to the control), it makes sense to exclude them from push campaigns and try a different channel (e.g., phone calls, email).

Avoiding false positives. In marketing, there’s a risk of mistaking random success for a pattern — the so-called false positives. For instance, you launch a new creative ad, and sales jump by 10% that month. It’s tempting to credit the ad. But what if the whole market grew by 10% at the same time or a competitor temporarily left the market? Without a control group, you'd assume the success was yours and continue investing in that ad. The control group protects against such errors. It acts as a “negative control” — showing what happens if nothing is changed.

Examples of Using GCG Across Different Industries

The global control group method is virtually universal. Let’s look at how it is applied in practice across various sectors.

Retail

Chain stores and retailers actively use control groups to assess the effectiveness of marketing campaigns. For example, a supermarket launches a promotion: it sends customers coupons for 10% off beverages. Meanwhile, 5% of customers are randomly selected into a control group — they don’t receive the coupons.

After a month, the results are compared: did the average check amount change? Did beverage sales increase? If sales in the main group (who received coupons) rose by 15%, the promotion clearly encouraged purchases. If there’s no difference, the discount may have been too small or customers would have bought the products anyway.

E-commerce

Online stores use GCGs for continuous A/B testing of hypotheses to improve conversion rates. For instance, a large e-commerce retailer launches a new product recommendation strategy on its website and in email campaigns. It needs to verify how the new strategy affects sales.

To do this, 10% of users remain on the old version of the site and don’t receive the new emails — that’s the control group. The remaining 90% — the main group — receive updated recommendations.

After a few weeks, results are compared: did users add more items to their carts, make more purchases, return to the site more often? In e-commerce, conversion rate (CR) is the key metric of effectiveness. Control groups help detect even small changes. For example, if CR rises from 2.5% to 3%, that’s only a 0.5 percentage point difference — but in reality, it’s a 20% increase, which means a significant sales boost.

Travel

In the travel industry, demand is heavily influenced by external factors like seasons, holidays, and travel restrictions. That’s why airlines, hotels, and tour operators increasingly use control groups to better assess the impact of marketing on sales.

Various indicators can be analyzed: number of bookings, average check, share of repeat trips. The control group helps determine which changes are driven by marketing campaigns and which are caused by external conditions.

For example, a hotel booking service wants to evaluate the effectiveness of an email campaign with personalized vacation offers. It excludes 5% of users from the mailing — that’s the control group. After the season, it compares the booking rates. Suppose 8% of those who didn’t receive emails booked a hotel, while 11% of those who did receive emails booked. The difference is 3 percentage points, or a 37.5% relative increase. That’s a strong argument in favor of email marketing.

Control groups are especially useful during sudden market changes. Suppose popular destinations are closed and overall travel demand drops. Booking numbers will fall both among those who received emails and those who didn’t. But if the drop is less severe among recipients, the campaign helped retain some customers and soften the impact of external factors.

Global Control Groups in Altcraft Platform

The Altcraft platform provides built-in tools for working with global control groups. With these tools, marketers can accurately measure the impact of email campaigns, push notifications, SMS, and messenger messages.

Setup takes just a few steps. You select who to include in the control group — the entire database or a specific segment. You specify the group size (from 5% to 25%, but no less than 1,000 profiles), the frequency of recalculation, and the selection method — random or from a pre-prepared list. To avoid skewing the experiment results, you can prevent the same client from being included in the GCG two times in a row.

Setting up a global control group in the Altcraft marketing automation platform

Important: participants of the global control group do not receive marketing messages through any connected channels — whether email, push, SMS, or messengers.

At the same time, clients must continue to receive critically important notifications — such as receipts, order confirmations, delivery updates, or password recovery messages. In Altcraft, this is managed through an exception system: you predefine which campaigns, scenarios, or resources should bypass the GCG rule. This ensures essential communications are preserved even when a client is temporarily excluded from marketing.

To evaluate the results, you can go to the analytics section and compare the behavior of the control and main audiences. The platform will display differences in key metrics and help you draw conclusions based on the data.

Conclusion

Global Control Groups (GCGs) are the "gold standard" for evaluating marketing effectiveness. They reveal the true impact of advertising by filtering out the influence of other factors. If a company wants to get the most out of its marketing investments, GCGs are essential.

Modern platforms like Altcraft simplify the implementation of this tool. All you need to do is formulate a hypothesis, launch a control group, and collect accurate data. Then use that data to refine your marketing.

This is what a data-driven approach is all about: making decisions based not on intuition, but on experiments and facts. Smart use of GCGs turns marketers into researchers, and campaigns into a series of experiments where each launch brings new insights.

Everyone wins: the business gains growth and efficiency, and customers receive more relevant offers. That’s why GCGs are the foundation of effective marketing. Without them, it’s impossible to build a truly evidence-based audience strategy.