A/B Testing in Your Small Shop: Which Offer Works Better?
Every week without testing, you're spending money on the wrong offer — set up in 5 minutes, free up to 100 customers.
Which of two offers brings more customers back? An A/B test answers this question without guesswork. You split your customers randomly into two groups, give each a different version, and count who came. The math fits on a napkin. Here's the step-by-step process, a real café example, and the rules for when a difference actually matters.
What an A/B test is (and why gut feeling isn't enough)
You split your customers randomly into two groups. Group A gets version A, for example "Free cappuccino after 10 stamps." Group B gets version B, for example "20 percent off your next purchase." After a set time, you count how many from each group came back.
The key is randomness. If you give the discount to regulars and the free product to new customers, you're comparing regulars to newcomers, not the offers. Randomness ensures both groups are similar on average. Only the offer explains the difference.
Without a test, gut feeling decides: "The discount went over well, it was busy on Saturday." That Saturday is always busy gets ignored. The A/B test replaces this feeling with a number.
The difference from a control group
An A/B test compares two offers against each other. A control group compares one offer against no offer at all. Both answer different questions:
- Control group: Does my offer do anything at all, or would customers have come anyway?
- A/B test: Which of two offers works better?
If you want both answers, use three groups: A, B, and a small group with no offer. How a control group works in detail and why it's so honest is explained in the article Control Groups in Marketing.
A/B test in five steps
1. Set one question. Change only one thing per test: reward, message, timing, or discount size. If you change two things at once, you won't know which one worked.
2. Split randomly. With digital stamp cards, it's simple: even card numbers in Group A, odd in Group B. Both groups should be the same size.
3. Set a time frame. Before you start, not after. For a café, two weeks is enough; for a barber, more like six weeks, since visits are further apart.
4. Define success. Usually: the share of customers who came at least once during the period. Alternatively: revenue per customer or redemptions.
5. Analyze and decide. The math comes next.
The math: café example
A café with 300 active cards tests two push messages for a slow Tuesday. 150 cards get version A: "Tuesday: Free cookie with every coffee." 150 cards get version B: "Tuesday: 15 percent off everything."
After four Tuesdays, the café counts how many customers from each group came at least once on a Tuesday:
| Group | Recipients | Came on Tuesday | Response Rate |
|---|---|---|---|
| A: Free cookie | 150 | 33 | 22.0% |
| B: 15% off | 150 | 24 | 16.0% |
The formula for response rate:
Response Rate = customers who came ÷ recipients × 100
Version A is at 22 percent, version B at 16 percent. The difference is 6 percentage points. In other words: the cookie brought in about 37 percent more customers than the discount.
Plus: the cookie costs the café about 30 cents; 15 percent off a 6-euro purchase costs 90 cents. Version A is both more effective and cheaper.
When a difference really counts
33 versus 24 customers sounds clear-cut. But with 150 people per group, chance can shift some customers around. A simple rule of thumb for small shops, no formulas needed:
- Difference under 3 percentage points: No result. Both versions are roughly equal; pick the cheaper one.
- Difference 3 to 6 percentage points: A hint, not proof. Repeat the test or extend the time frame.
- Difference over 6 percentage points with at least 100 customers per group: Reliable enough to act on.
The smaller the groups, the bigger the difference needs to be. With 40 customers per group, even 10 percentage points can be chance. So for small shops: test versions that differ clearly, not "14 versus 15 percent off." And repeat a close test once before you overhaul your program.
What you can test in a small shop
The best tests answer decisions you're making anyway:
- Reward type: Free product versus discount versus extra. Which form fits your margin is covered in the article on choosing the right reward for your loyalty program.
- Stamps needed: 8 versus 10 stamps to earn a reward. Heads up: only test this on new cards; don't change existing ones.
- Message wording: Straightforward ("Today 2 for 1") versus personal ("Anna, your Tuesday coffee is waiting").
- Timing: Message at 8 a.m. versus 11:30 a.m. For lunch customers, timing often matters more than content.
- Time limit: Open-ended offer versus "only until 2 p.m." Examples of short-term offers are in Flash Promotions.
Self-check: Is your test fair?
Answer these yes or no:
- Did I set the question (what I'm testing) before I started?
- Are both groups the same size and randomly split?
- Did I set the time frame in advance and stick to it?
- Am I changing only one thing (reward, message, timing)?
- Does my team treat both groups the same (no extra mention of one version)?
- Am I evaluating by the metric I set beforehand (not switched later)?
Every "no" ruins the result. A fair test needs discipline, not statistics.
Three mistakes that wreck any test
Changing the question afterward. If response rate shows no difference, suddenly you're comparing revenue; if that doesn't work, you look at transaction size. Set your success metric upfront and stick with it.
Stopping the test as soon as one version leads. After three days A is ahead, after ten days B is. Stick to your planned time frame.
Treating groups differently. If your team mentions the cookie offer at checkout but not the discount, you're testing your team, not the offer.
What happens if you don't change anything
Without testing, every offer runs on gut feeling. You give 15 percent off because it feels right, even though a free cookie brings more customers and costs less. Every week without a test, you waste money on the wrong offer. You don't notice because you never compare.
With an A/B test, you know after two weeks which version works. You save money, bring in more customers, and make decisions with numbers instead of feelings.
Takeaway
An A/B test in a small shop isn't science, it's discipline: one question, random groups, fixed time frame, one calculation. With digital stamp cards, you already have the customer numbers for splitting and the scans for analysis in your dashboard. Start now: With stampa, the test is set up in 5 minutes, free up to 100 active customers, no credit card, cancel monthly.