Control Group Marketing: Measure Your Loyalty Program ROI
Every week without a control group, you might be giving rewards to customers who would have come anyway—without realizing it.
A control group in marketing is the only honest way to find out whether your loyalty program really drives sales. Without one, you don't know if you're winning repeat customers or just handing out free coffee to people who would have come anyway. The principle comes from medicine, but it works without a statistics degree. This article explains what a control group is, why the usual before-and-after comparison misleads you, and how stampa uses it to measure your card's ROI.
The short answer
A control group is a randomly selected small group of your customers. They deliberately don't get the stamp card or your promotions. You then compare how often this group comes back with the large group that has the card. The difference is the real effect of your program. Both groups experience everything else together: the same weather, the same season, the same construction outside your door.
Why before-and-after misleads you
The obvious approach is comparison: revenue in the month before the card versus the month after. The problem: lots of other things changed between those months.
Imagine you launch the card in March. April revenue is 12 percent higher. Was it the card? Or the first warm weekend? The construction that closed the street in March? The new office building that opened in April? You can't know.
The other way around: you launch the card in October. November revenue is 5 percent lower. Did the card hurt? Probably not. November is just slower in many businesses. Maybe it would have been 10 percent slower without the card. You can't know that either.
The before-and-after comparison mixes your program's effect with everything else that happened. It's not a measurement—it's a guess.
How the control group solves this
The idea is simple: instead of comparing two time periods, you compare two groups in the same time period.
- Group A (the majority): gets the stamp card, push notifications, promotions.
- Group B (the control group): gets none of that. They shop normally, but you count them too.
Both groups experience the same April with the same weather and the same construction. If Group A comes back more often in April than Group B, it's not because of the weather. Group B had that too. It's because of the one difference between the groups: the card.
The key word is random. The control group can't be customers who happened not to want a card. They might just be less interested in general. It has to be randomly selected so both groups are equal on average: same mix of regulars and occasional customers, same age distribution, same everything except the card.
Test yourself: Are you measuring honestly?
- Do you know exactly how many extra visits your card brings?
- Can you rule out that the sales increase was due to weather?
- Do you have a group of customers who don't get a card?
- Is this group randomly selected, not chosen by interest?
- Do you wait for the ROI result until you have enough data?
If you answered no to more than two questions, you're probably not measuring what your card really does.
How stampa measures ROI with a control group
This exact principle is built into stampa's ROI measurement. Here's how it works:
- Random selection. About ten percent of customers who scan your QR code are randomly assigned to the control group. They don't get a card or promotions. The customer doesn't notice.
- Both groups are counted. Return visits are tracked for everyone, whether they have a card or not.
- Compare return visits. After a few weeks, you compare: how often does the card group come back versus the control group? The difference is the extra visits the card brought.
- Convert to euros. Extra visits times your average transaction value equals extra revenue. The dashboard shows it as X extra visits, roughly Y euros.
- Only when it's reliable. The result only appears when the difference is statistically reliable. Until enough customers and weeks have accumulated, stampa tells you honestly: not enough data for a confident result yet.
That last point matters. Many tools show some ROI number from day one. With twenty customers, that's pure chance. Honest measurement waits until it can be sure.
What statistically reliable means, without jargon
Imagine you flip a coin ten times and get heads seven times. Is the coin rigged? Probably not. Seven out of ten happens with a fair coin all the time. Flip it a thousand times and get heads seven hundred times—the coin is almost certainly rigged.
It's the same with the control group. If 20 customers with a card come back an average of 2.1 times and 3 customers without come back 1.8 times, that tells you nothing. With such small numbers, one customer who happened to be on vacation can flip the result. If 500 customers with a card come back 2.1 times and 55 without come back 1.8 times, that's a real result.
Statistically reliable or significant means: the difference is so big and the groups are so large that chance is a very unlikely explanation. How long it takes depends on your customer volume. A café signing up a hundred new cards a week gets there in a few weeks. A nail salon with ten new customers a week needs months.
A worked example
A café with 600 active customers. 540 have the card, 60 are in the control group.
| Card group | Control group | |
|---|---|---|
| Customers | 540 | 60 |
| Visits per customer in 8 weeks | 5.4 | 4.5 |
| Difference per customer | 0.9 visits | |
| Total extra visits | 540 × 0.9 = 486 | |
| At €6 average transaction | around €2,900 |
The card brought about 486 extra visits in eight weeks, roughly €2,900 in revenue. Subtract the redeemed rewards—say 100 free coffees at €0.80 cost, so €80. Net: around €2,820 in revenue that wouldn't have happened without the card.
Without a control group, you wouldn't have this number. You'd see revenue went up, but not whether it was the card or spring.
The cost of measurement
The control group costs something: ten percent of your customers don't get the card. If the card works, they come back a bit less often. In the example above, that's 60 customers times 0.9 visits times €6, roughly €320 in eight weeks.
In return, you get certainty. You know whether the reward pays off. Whether push promotions do anything. Whether changing the stamp count worked. Without this certainty, you spend years giving out rewards without knowing if they work. That almost always costs more than the control group.
What happens if you don't change anything
Without a control group, you're flying blind. You see revenue fluctuate, but not why. You change the reward from 10 to 8 stamps, and revenue goes up. Was it the change? Or summer season? You don't know. So you change it back. Maybe revenue falls. Was it the change back? Or autumn? You don't know again.
The result: you tweak things without knowing what they do. You give out rewards without knowing if they're worth it. And every week you lose repeat customers because you don't know which promotion would have brought them back.
What else you can use this principle for
The control group is the foundation for any honest measurement. The same principle underlies A/B testing (two versions of a promotion run against each other). How to use it to compare push messages or rewards is explained in A/B Testing in Your Small Shop. And which metrics to watch alongside ROI is covered in Measuring Your Loyalty Program: 7 Key Metrics.
Conclusion
A control group is a randomly selected small group of customers who don't get your loyalty program. Their behavior shows what would have happened without it. The difference from the card group is the real effect. It's the only measurement that accounts for weather, season, and chance. stampa builds it in, compares return visits between both groups, and only shows the result when it's reliable. You can get started free in five minutes and have measurement running from day one—up to 100 active customers, no credit card, no risk.