Home/Blog/Conversion Funnels/Funnel Cohort Analysis: Why Step Totals …
Conversion Funnels

Funnel Cohort Analysis: Why Step Totals Mislead You

Funnel cohort analysis means each step counts only people who completed every earlier step. Check your tooling and set the time window from behaviour.

Cover of the FIKR Space guide: Funnel Cohort Analysis: Why Step Totals Mislead You

Funnel cohort analysis means that each step counts only the people who completed every earlier step, not everyone who happened to perform that action in the period. If your report counts steps independently, the shape will still look like a funnel and every drop-off percentage will be fiction.

This is the trap that quietly invalidates a large share of funnel dashboards. It is not a subtle statistical preference. It is the difference between a diagnosis and a decoration, and it is worth a post of its own because the error cannot be seen on the chart.

Funnel cohort analysis: what it counts and what it refuses to count

Consider two ways of building the same report. In the first, the report counts how many people did each thing during the period. In the second, it follows people: a person appears at step three only if they also passed step one and step two.

MethodWhat step three containsWhat the drop-off means
Independent totalsEveryone who did the step three action in the period, whoever they areNothing reliable: the groups at each step may be different people
Cohort progressionOnly people who completed every step before itThe share of a known group who carried on

Here is how it goes wrong. Suppose a thousand people appear at step one and a hundred at step three. Under independent counting, those hundred may not be among the thousand at all. They might have arrived earlier, come in through another route, or returned from a previous period. The report still draws a funnel, and it still shows a drop from a thousand to a hundred. But nobody can say that any of those thousand people was lost.

The danger is practical. You may go on to optimise a step that is not losing anyone, because the percentage that pointed at it was produced by mixing groups. That is exactly the kind of confident wrong decision a funnel is meant to prevent.

Why the error is so hard to see

A badly counted funnel looks like a good one. The bars shrink from left to right. The percentages are plausible. Nothing flags that the groups are not the same people, because the tool did what it was asked: it counted.

That is the reason the guide names this as the single most common invalidating defect in funnel dashboards. The chart is shaped like a funnel and the percentages mean nothing, and the defect is invisible from the chart alone. A broken funnel still draws a chart. So the check has to be deliberate, done once and then remembered.

Free guide, 30 pages: The Funnel Playbook. Send me the guide.

How to verify how your tooling counts

You do not need to read documentation for hours. You need to ask a direct question and confirm the answer with your own data.

  1. Read what the report says it does. Look for wording about people who completed earlier steps, as opposed to people who performed an action.
  2. Test it with your own journey. Walk the funnel yourself, from a device that is not yours, and watch whether you appear at each step in order.
  3. Test it with a path that skips a step. If a person can reach a later step without an earlier one, see whether the report counts them at that later step. Under cohort counting they should not be there.
  4. Compare the totals with something you know. Sanity check the drop-off numbers against something you can verify independently.

Step three is the sharpest test. A cohort report should refuse to count someone who skipped ahead. If it counts them, you are looking at independent totals, whatever the label says. The guide's checklist makes this a box to tick: confirm that your tooling counts cohorts, not independent step totals.

Cohort counting depends on a clean definition

Counting a cohort through a funnel assumes the funnel is a true sequence. If someone can reach step four without step two, then step two is not part of the funnel, and every percentage calculated from it is fiction. This is why the structure comes first: see how to define funnel steps for how to write each step as a state reached in order.

It also depends on being able to recognise the same person at every step. If the anonymous visitor and the known user are treated as two people, a cohort will lose its members at the point they identify themselves, and you will see a cliff that is an instrumentation gap. The identity join and the events that feed each step are covered in how to track funnel events.

Set the time window from real behaviour

Cohort counting needs one more rule: how long a person has to complete the next step before they are counted as dropped. That is the time window. It is easy to treat it as a technical setting, and it is in fact one of the most influential choices in the whole funnel.

WindowWhat goes wrong
Too shortPeople who were simply thinking it over are recorded as lost
Too longRecent data never settles, so you cannot read a change

The guide's advice is to set it from how people actually behave. Look at how long the transition takes for the people who do complete it, and set the window comfortably beyond that. A person who needs a weekend to decide is not lost on Saturday. A window based on a guess may treat them as if they were.

Do this step by step, as each transition may take a different amount of time. The gap between arriving and giving an email is not the gap between completing setup and paying, and a single window for all of them hides that.

Say the window every time

Whenever you quote a conversion rate, say the window out loud. Two people quoting different windows will disagree forever without ever discovering why. One says the step converts well, the other says it converts badly, and both are right for the window they are using.

This is a small habit with a large effect on meetings. A rate with no window attached is only half a fact. The guide's checklist asks for two things here: set a window based on real behaviour, and quote it whenever you quote a rate.

FIKR Content, multi-channel publishing: reads, listens, opens, clicks in one dashboard. See FIKR Content.

Questions to put to any funnel report

Whether the report is a tool's built-in view or a spreadsheet you built yourself, the same short list of questions applies. Ask them before you let any percentage influence a decision.

  • Does each step contain only people who completed every previous step?
  • What is the time window, and was it set from how long the transition actually takes?
  • Can the same person be recognised at every step, before and after they identify themselves?
  • Do the drop-off numbers agree with something I know independently?
  • Would the report count someone who reached a later step without an earlier one?

If you cannot answer one of them, treat that as the next task and not as a detail to settle later. The guide puts the rule plainly: if you cannot tick a box, that is your next job.

A funnel you can follow with your finger

There is a simple mental picture that keeps cohort counting honest. Imagine tracing a single person through the steps with a finger, from the first to the last. At each step you can say yes, this same person did that. A cohort funnel is the sum of many such traces, and no trace is allowed to jump. An independent-totals report is the sum of many people who each did one thing, with no traces at all. If you cannot imagine following one person through your report, the report is not telling you about people moving through a sequence.

Free guide, 30 pages: The Funnel Playbook. Send me the guide.

What cohort counting makes possible

Once your counts are cohort-based, the rest of the funnel work becomes trustworthy. You can rank steps by absolute loss, as described in how to find where a funnel loses people, and know the numbers refer to people you can actually follow. You can compare segments without wondering whether the groups were mixed. And you can judge a change by what happened to the same kind of cohort before and after.

Without it, the other methods inherit the flaw. A beautiful ranking of independent totals is still a ranking of fiction. That is why this check belongs early, at the point where the funnel is defined, not later when something looks odd.

FIKR Content, multi-channel publishing: reads, listens, opens, clicks in one dashboard. See FIKR Content.

Frequently asked questions

How can I tell if my funnel counts cohorts?

Test it. Walk the funnel yourself, then try a path that skips a step. A cohort report should not count a person at a step unless they completed every earlier one. If it does, the steps are being counted independently.

Does a cohort funnel look different on the chart?

Not necessarily, and that is the problem. An independent-totals funnel is shaped the same way, with percentages that mean nothing, so the chart cannot tell you which you have.

How long should the time window be?

Comfortably longer than the time the transition takes for people who complete it. Look at real behaviour, not at a guess, and review it if recent data never seems to settle.

Why does the window need to be quoted?

Because different windows give different conversion rates for the same funnel. Without it, two people can disagree indefinitely without finding out why.

The full method, with the checklists, is in The Funnel Playbook, a free 30-page playbook.

LG

Luis Goncalves

// Founder & CEO at FIKR Space

Three-time founder. Built and exited Evolution4All before this. Now building FIKR Space — the operating infrastructure underneath every innovation ecosystem (startups, accelerators, governments, investors). Lisbon-based, works global.

Claim your founding member spot.

Every founding member is hand-reviewed. Tell us who you are and how you want to co-create — we get back inside 48 hours.

Role
Job Title
Country
Cookies

We use cookies.

Essential ones keep the site working. The rest only run if you say so. No dark patterns, no pre-ticked boxes.Privacy policy

Co-create with us. It's about building it together.Co-Create With Us