Global Control Group: Theory and Practice
What if marketing doesn’t actually work — and we just don’t notice it? To understand whether a company’s marketing strategy is truly effective or if the budget is being wasted, global control groups are used. This is a helpful tool that allows you to distinguish real impact from mere coincidence.
In this article, Denis Barov, lead systems analyst at Altcraft, explains how to measure the effectiveness and results of a marketing strategy, how global control groups work, why they are needed, how to use them, and what insights they can provide.
What is a Control Group
The target audience is divided into two parts:
- Main (experimental) group — receives all planned marketing activities: emails, push notifications, promotions, personalized offers.
- Control group — does not receive any messages under the studied strategy. This is the so-called “clean” group: it’s not affected by direct marketing, though it may encounter the campaign via other channels — such as a website banner, in-store ad, or flyer on the street.
After some time, the marketer analyzes key metrics — conversion rate, revenue, number of orders — and compares the behavior of the two groups. If the indicators are nearly identical, the current strategy is ineffective and provides no significant uplift.
Difference Between Global and Local Control Groups
Depending on the objective, there are two types of control groups: global and local. Both approaches help analyze results, but in different contexts and with different exposure durations. Here’s the difference.
Key Steps for Working with GCG
Before starting with a global control group, clearly define the experiment's goal: what changes or hypotheses are being tested (e.g., the impact of a new ad campaign on purchase volume). Predefine the key metrics to measure success: conversion rate, average order value, frequency or number of purchases, revenue per customer, etc.
A portion of clients from the entire customer base or a specific segment is randomly selected to form the control group. Typically, this is 5–10% of the audience, sometimes more, but usually no more than 20–25%. For example, if the database contains 100,000 users, a 5% control group would include 5,000 clients. In some cases, the group size may vary depending on the experiment’s goals.
Clients in the control group are fully excluded from all marketing activities across all channels — emails, SMS, push, ads, personalized banners, promotions, etc. Only transactional or mandatory communications are allowed (e.g., payment confirmations, tickets, security alerts).
The control group remains isolated from marketing for a certain period. This duration is set by the marketer — for example, a month, a quarter, or the full length of a specific campaign. In practice, 1–3 months is optimal for most purposes — long enough to obtain reliable results, but not so long that customers forget about the brand.
While the control group is on pause, standard marketing activities are carried out with the rest of the audience (the main group). These may include email newsletters, SMS campaigns, push notifications, personalized banner displays, loyalty program launches, and so on.
After the test period ends, the key moment arrives — comparing the results. You measure the same metrics in both the main and control groups. In most cases, the following indicators are examined:
- Conversion (CR%) — the percentage of customers who completed a target action (purchase, service order, registration, etc.).
- Average Order Value (AOV) — the average purchase amount.
- Revenue per customer — the income generated from each customer. In MoEngage studies, for example, the control group brought in $15 per customer, while the active group brought in $24 — clearly indicating a positive impact from the campaigns.
- Number of orders/sales — the total number of transactions in each group.
- Engagement — page views, clicks, time spent in the app, email opens, subscriptions.
- Retention — repeat purchases or visits.
The control group’s metrics show how customers behave without any marketing influence. By comparing these indicators with the main group, you can understand how much your marketing impacted the result.
For example, if the conversion rate in the control group (without marketing) is 5%, and in the main group (with marketing) it’s 12%, this means marketing activities increased conversion by 7 percentage points. That’s a good result. But if the difference is small (e.g., 15% in the control group vs. 16% in the main group), then marketing had almost no impact on customers, and the strategy needs improvement.
When the experiment is complete and the data is collected, marketers assess whether the campaign was successful. For example, if revenue in the main group is 10% higher than in the control group and the difference is statistically significant, it means marketing generated a +10% increase in sales, and you can estimate the approximate ROI. Based on the analysis, further actions are taken:
- Scale successful activities to the entire audience. The control group may also receive the missed offers later, if relevant.
- Adjust or cancel ineffective campaigns. For example, you can shut down a channel that didn’t perform to avoid wasting budget.
- Draw conclusions by segments. Perhaps the campaign produced growth in one audience segment but not in another. In that case, future marketing budgets can be redirected to where there was an effect.
How to Select a Control Group
Conditions and Limitations of GCG
| Limitation | Description | What to Consider |
|---|---|---|
| Small database | If the customer base is small, the control group is too small for statistical analysis | Altcraft recommends using GCG with a base of at least 20,000 profiles. |
| Potential audience loss | GCG participants don’t receive promotions and may not purchase without incentives | To prevent churn, don’t keep clients in GCG for too long — use participant rotation. |
| Cross-channel influence | Participants may still see marketing messages through other channels, even while in the control group (e.g., targeting, offline, etc.) | It's difficult to fully eliminate external exposure — it's important to classify channels by level of influence and evaluate impact within the groups. |
Key Metrics and Conversion Calculation
Analysis using control groups helps accurately measure the impact of marketing on key business indicators. The main metric in such experiments is the conversion rate (CR%), which shows the percentage of customers who completed a target action (e.g., purchase, registration, subscription).
In addition to CR%, derivative metrics are often measured as well:
- Incremental CR — the increase in conversion rate resulting from the campaign.
- Uplift% — relative growth, calculated as (CRmain - CRcontrol) / CR_control × 100%.
- Segmented conversion — you can calculate CR for different customer categories (new vs. returning, male vs. female, etc.) within each group.
Control groups allow you to accurately measure at which stage of the funnel a campaign increases conversion. In analytics, CR is often tracked step by step: email open → click-through → cart → purchase. Comparing this with the control group shows where the campaign influences behavior and where it doesn’t. This makes it possible to precisely improve the stages that are limiting growth. Even a small increase in conversion — for example, from 2% to 2.2% — can lead to a significant revenue boost across a large audience.
How to Tell if the Campaign Worked: Calculating the Result
Let’s say you sent a mailing with a personal 10% discount to a base of 20,000 clients. Another 2,000 people were placed in a global control group — they did not receive the message and were unaware of the discount.
- In the main group: 2,400 orders → CR = 12%
- In the control group: 160 orders → CR = 8%
Now let’s calculate effectiveness:
12% – 8% = +4 percentage points — this is how much the conversion rate increased due to the campaign.
(12% – 8%) / 8% × 100% = +50% — this is how much marketing improved the result in percentage terms.
So, out of the 2,400 orders in the main group, about 1,600 would have happened without the mailing, and the remaining 800 orders were driven by marketing. This is the incremental effect, proven through the GCG experiment.
Why Statistical Significance Matters
In our example, the groups are large, and the difference of 100 buyers is most likely significant. But if the groups were smaller, the difference might have occurred by chance. That’s why experts often use statistical tests (e.g., chi-square test, t-test) or ready-made A/B test calculators to verify that the p-value is below the acceptable threshold (usually 0.05).
Other Metrics When Analyzing via GCG
| Metric | What it Measures | How GCG Helps |
|---|---|---|
| Average Order Value | The average purchase amount | Shows whether the discount reduced profit margins. Group comparison reveals if there's profit cannibalization. |
| LTV (Lifetime Value) | Long-term customer value | Helps assess whether aggressive marketing lowers loyalty and repeat purchases. |
| Retention rate | The percentage of customers who return to make another purchase | GCG shows how marketing affects retention — especially important for subscriptions and games. |
| ROMI / ROI | Marketing return on investment | Allows precise calculation of how much revenue was generated by marketing beyond the natural baseline, without guesswork. |
Why a Control Group Is Needed
Global control groups solve several key tasks in marketing analytics:
These insights help refine segmentation — highlighting the groups most responsive to marketing and focusing efforts on them. Over time, this leads to personalization: once you know who truly benefits from marketing, you can stop bothering those it doesn’t affect or design a different approach for them. For example, if the control group shows that customers aged 50+ barely react to push notifications (with purchases similar to the control), it makes sense to exclude them from push campaigns and try a different channel (e.g., phone calls, email).
Examples of Using GCG Across Different Industries
The global control group method is virtually universal. Let’s look at how it is applied in practice across various sectors.
Retail
Chain stores and retailers actively use control groups to assess the effectiveness of marketing campaigns. For example, a supermarket launches a promotion: it sends customers coupons for 10% off beverages. Meanwhile, 5% of customers are randomly selected into a control group — they don’t receive the coupons.
After a month, the results are compared: did the average check amount change? Did beverage sales increase? If sales in the main group (who received coupons) rose by 15%, the promotion clearly encouraged purchases. If there’s no difference, the discount may have been too small or customers would have bought the products anyway.
E-commerce
To do this, 10% of users remain on the old version of the site and don’t receive the new emails — that’s the control group. The remaining 90% — the main group — receive updated recommendations.
After a few weeks, results are compared: did users add more items to their carts, make more purchases, return to the site more often? In e-commerce, conversion rate (CR) is the key metric of effectiveness. Control groups help detect even small changes. For example, if CR rises from 2.5% to 3%, that’s only a 0.5 percentage point difference — but in reality, it’s a 20% increase, which means a significant sales boost.
Travel
In the travel industry, demand is heavily influenced by external factors like seasons, holidays, and travel restrictions. That’s why airlines, hotels, and tour operators increasingly use control groups to better assess the impact of marketing on sales.
Various indicators can be analyzed: number of bookings, average check, share of repeat trips. The control group helps determine which changes are driven by marketing campaigns and which are caused by external conditions.
For example, a hotel booking service wants to evaluate the effectiveness of an email campaign with personalized vacation offers. It excludes 5% of users from the mailing — that’s the control group. After the season, it compares the booking rates. Suppose 8% of those who didn’t receive emails booked a hotel, while 11% of those who did receive emails booked. The difference is 3 percentage points, or a 37.5% relative increase. That’s a strong argument in favor of email marketing.
Control groups are especially useful during sudden market changes. Suppose popular destinations are closed and overall travel demand drops. Booking numbers will fall both among those who received emails and those who didn’t. But if the drop is less severe among recipients, the campaign helped retain some customers and soften the impact of external factors.
Global Control Groups in Altcraft Platform
The Altcraft platform provides built-in tools for working with global control groups. With these tools, marketers can accurately measure the impact of email campaigns, push notifications, SMS, and messenger messages.
Setup takes just a few steps. You select who to include in the control group — the entire database or a specific segment. You specify the group size (from 5% to 25%, but no less than 1,000 profiles), the frequency of recalculation, and the selection method — random or from a pre-prepared list. To avoid skewing the experiment results, you can prevent the same client from being included in the GCG two times in a row.
At the same time, clients must continue to receive critically important notifications — such as receipts, order confirmations, delivery updates, or password recovery messages. In Altcraft, this is managed through an exception system: you predefine which campaigns, scenarios, or resources should bypass the GCG rule. This ensures essential communications are preserved even when a client is temporarily excluded from marketing.
To evaluate the results, you can go to the analytics section and compare the behavior of the control and main audiences. The platform will display differences in key metrics and help you draw conclusions based on the data.
Conclusion
Global Control Groups (GCGs) are the "gold standard" for evaluating marketing effectiveness. They reveal the true impact of advertising by filtering out the influence of other factors. If a company wants to get the most out of its marketing investments, GCGs are essential.
Modern platforms like Altcraft simplify the implementation of this tool. All you need to do is formulate a hypothesis, launch a control group, and collect accurate data. Then use that data to refine your marketing.
This is what a data-driven approach is all about: making decisions based not on intuition, but on experiments and facts. Smart use of GCGs turns marketers into researchers, and campaigns into a series of experiments where each launch brings new insights.
Everyone wins: the business gains growth and efficiency, and customers receive more relevant offers. That’s why GCGs are the foundation of effective marketing. Without them, it’s impossible to build a truly evidence-based audience strategy.
We’ll show you the platform and find a solution tailored to your business goals
Book a demoYou might be interested in:
Enter mobile push notifications – a new-age marketing tool with the ability to revolutionize the way you engage with and retain customers, whether potential, existing, or lapsed.
Read moreUncover email marketing secrets, avoid common mistakes, and optimize your strategy for maximum impact and conversions.
Read moreDiscovering ways to better know your customers for efficient work with them?
Read more