How to Measure Results by Segment, Not by Average
How to measure results by segment: split every overall rate by group, find who responds best and worst, and decide whether to find more or stop sending.
To measure results by segment, take every overall rate you report, split it by the groups you have defined, and read what each group did on its own. The overall figure averages groups that behave completely differently, and it hides the two findings that matter most.
This is the second payoff of segmenting. The segmentation you built for sending is the segmentation that makes the numbers readable. Here is how to read them, what to do with what you find, and what to check before you trust it.
How to measure results by segment: why an average misleads
An overall open, conversion or win rate is a single number that combines everyone you sent to. If your base contains engaged contacts, cooling contacts, dormant contacts and customers, the average describes none of them. It is a blend, and a blend can look acceptable while one group does very well and another does terribly.
Split it by segment and two things usually appear.
- One group that responds far better than the rest. That tells you who to find more of.
- One group that responds terribly. That is either the cheapest thing to fix or the clearest thing to stop paying for.
Neither finding is available from the average. Both are available from a table with one row per segment.
The table to build
Build the same table after every send and every campaign. Keep the rows to the segments you defined, and keep the columns to the outcomes that matter to you.
| Segment | Who received | The outcome you chose | Compared with the rest |
|---|---|---|---|
| Engaged | Actual recipients, not segment size | Your own measure of response | Better, similar or worse |
| Cooling | Actual recipients | The same measure | Better, similar or worse |
| Dormant | Actual recipients | The same measure | Better, similar or worse |
| Customers | Actual recipients | The same measure | Better, similar or worse |
The second column deserves care. The count of people in a segment and the count who actually receive the message are two different numbers. If you only look at one, you are looking at the wrong one. Use recipients as the base for any rate, and keep the matching count beside it so the gap stays visible.
Find the best performer and learn from it
The segment that responds best is a lead, not a trophy. Ask what its members have in common, and whether you can find more people like them. The answer might be a fit fact, a source or a recent behaviour. Each of those is something the base can already record.
This is where the earlier decisions pay back. If source is a field, you can ask which source sends people who engage and which sends people who vanish. If fit is recorded, you can ask which kind of contact becomes a customer most often. Those questions decide where to spend the next month, and they are unanswerable when everyone is one pile. The recording decisions are in tags vs fields in a contact database.
Find the worst performer and decide
The poorest group asks you to choose between two honest options.
| If you find | It probably means | Decide |
|---|---|---|
| A group that is large and unresponsive | Many are silent, wrong or not a fit | Fix the segment or stop sending to it |
| A group the message does not suit | The copy is general, not written for them | Rewrite for the group |
| A group full of test or internal addresses | Rates are distorted | Exclude them by rule and re-measure |
The guide frames this as fix or stop. For a dormant group the answer is usually the second: one win-back and then a rest, as in how to retire dormant contacts. For a group you care about, rewrite the message, since sending the identical newsletter to every segment is the same broadcast with extra steps.
Check the result is not an artefact
Before you act on a split, test that the groups are what you think they are. Several things can make a segment look better or worse than it really is.
- Test addresses. They open and click everything and score as your most engaged contacts. If they sit in a segment, its rate is inflated.
- Duplicates. A person counted twice distorts both the count and the rate. See how to prevent duplicate contacts.
- Groups too small to learn from. Over-segmentation produces groups too small to learn anything reliable from. If a row has very few contacts, treat its figure as a hint, not a finding.
- Stale membership. A static list used as though it were current includes people who have since bought, left or unsubscribed.
A segment of a dirty base is a precise wrong answer. Duplicated, stale or mislabelled records are hidden behind a confident looking count.
Read the sales side by segment as well
Marketing is not the only place an average hides things. Sales results blur in the same way. Handed the whole base, a salesperson calls the newest names or the ones they recognise. Handed a segment, defined by high fit, a recent deliberate action and not yet contacted, they call the people most likely to buy this week.
Measure that queue as its own group. If the share above the sales ready threshold is almost everyone, the threshold has stopped meaning anything. That is a segment size finding, and it shows only when you look at the group on its own.
Do not forget the customers
Customers are a segment with their own job, and they deserve their own row. They are the most valuable segment and the one most often sent the wrong things: acquisition offers for what they already bought, or a welcome they already received. A row for customers lets you see whether the messages meant for them are working, separately from everything aimed at prospects.
Early customers also double as references, testers and sources of introductions, a point developed in a companion playbook. That value is invisible unless they are recorded as a distinct group.
Turn the finding into one change
The analysis is worth little unless it leads to a single change you can make before the next send. Pick one, not five.
- Write the best and worst segment down with the outcome you saw.
- Choose one action: find more like the best, or fix or stop the worst.
- Make the change, and note what you expect to happen.
- Look at the same table after the next send.
This is a small loop, and it is the reason the segments exist. A segment earns its keep when it changes what someone receives, starts something automatically, or makes a number readable. Measuring by segment is the third of those.
An afterwards checklist
The guide's sending checklist ends with four items you can use as written.
- I compared the result by segment, not just as one overall number.
- I noted which segment responded best, and whether I can find more people like them.
- I noted which segment responded worst, and whether to fix it or stop sending to it.
- Customers received customer messages, not acquisition offers.
If you want to see how the sections fit together before you start, what is contact segmentation gives the overall picture.
A worked reading, built from the guide's own groups
Suppose, for illustration only, that your table shows the engaged group responding far better than the rest and the dormant group responding poorly. Nothing in that is a surprise, and that is the point: the table has turned a vague worry into two named facts. What you do next follows from the rules above.
| What the table shows | What it tells you | The move |
|---|---|---|
| Engaged responds best | This is where your strongest material lands | Send your best offer and your direct asks here, and find more people like them |
| Cooling responds weakly | The general message is not holding them | Send your single most useful piece, less often |
| Dormant responds worst | Sending more will not help | One win-back, then a rest |
| Customers respond to the wrong thing | They are receiving acquisition messages | Give them onboarding, expansion and references |
The example is built from the group descriptions, not from any real figures. Your own table will show your own pattern, and the useful habit is to read it every time rather than only when something feels wrong.
Keep the definitions steady
A comparison is only fair if the segments mean the same thing each time. If you quietly change a rule between two sends, a movement in the numbers may be the rule and not the people. Write the definition in one sentence, name the segment for the rule, and change both together when you must. A segment named for a campaign makes this harder, because the name stops describing the rule and someone reuses it assuming it means something else.
Frequently asked questions
Which rate should I split first?
Whichever one you already report as a single overall figure. Open, conversion or win rate all average groups that behave differently, so pick the one you rely on most.
What if my segments are too small?
Then you may have over-segmented. Groups too small to learn anything reliable from are a real failure. Merge them or treat their results as hints.
Do I need a tool to do this?
No. A table with one row per segment and the same columns each time is enough. What matters is that the segments are defined consistently, so the rows mean the same thing from one month to the next.
How often should I look?
After each send or campaign, and again in the monthly pass. The monthly check confirms the segments still mean what they say and the gap between size and recipients is not growing.
The full method, with the checklists, is in The Segmentation Playbook, a free 35-page playbook.