◀ Course contents Part 2 · Module 2-07

Driving User Insights

Turn piles of data into "aha" moments your team can act on

Every team drowns in data, dashboards, surveys, interview notes. Insight is what's rare: the understanding of why users behave the way they do, sharp enough to change what you build. This guide teaches the craft of getting from one to the other.

Ready?

1

Data vs. Insight

There is a ladder that runs from data to decision, and most teams stop one rung short and then wonder why nothing changed.

Only the third one is actionable, because only the third one tells you what is wrong with someone’s understanding rather than with a number.

The distinction matters because “insight” is used loosely enough that dashboards claim to produce them. A dashboard produces data. Turning data into insight requires a step no tool performs: forming a claim about why people behave as they do, which can be wrong and therefore has to be argued for.

The test is straightforward. If someone reads your insight and cannot disagree with it, it is data. If they read it and say “no, I think it is actually because…”, you have written something with a mechanism in it — and a mechanism is the thing a team can design against.

The ladder from data to insight Three stacked layers: data, a measured fact; a finding, a pattern in the data; and an insight, an explanation of why people behaved that way. Only the top layer can be designed against. Insight they think the payment failed, so they stop Finding most abandons follow a card validation error Data 40% abandon at the payment step what you can act on
One piece of evidence, written three ways. Only the top sentence is capable of being wrong — a reader can answer it with "no, I think they leave because the total changes at the last step", and that disagreement is something a team can go and settle. Nobody can disagree with 40%.

Everyday example, the doctor's visit

"Your temperature is 39°C" is data, a true fact, but on its own it doesn't tell you what to do. "You have a throat infection, so take this antibiotic for a week" is an insight, it explains why you have the fever and points to an action. A thermometer full of readings is useless until someone works out what's causing them. Product teams drown in the thermometer readings (dashboards) but what they actually need is the diagnosis.

An example ladder

Data: "40% of users abandon checkout at the payment step."

Finding: "Abandonment spikes right after the order total updates."

Insight: "Users abandon because the shipping fee appears only at the last step, the total jumping 20% at payment feels like a trap."

Data describes. Insights explain, and an explanation tells you what to try next.

Up the ladder from a number

Data: 40% abandon at payment. Finding: most of those abandonments follow a validation error on the card field. Insight: people do not know their card was rejected for a formatting reason, so they assume the payment failed and stop. Only the third sentence tells anyone what to build.

Quick check

Which of these is an actual insight, not just data?

2

The Two Sources of Raw Material

Insight has two raw materials, and neither one is sufficient alone.

Behavioural data — analytics, logs, session recordings — tells you what happened, at scale, without anyone’s memory in the way. It is unbiased about frequency and completely silent about motive.

Direct contact — interviews, support tickets, sales calls, watching someone work — tells you why, in the person’s own terms. It is rich about motive and unreliable about frequency, because five people are five people no matter how vivid they were.

They work as a pair, in a specific order that is worth being deliberate about. Analytics locate where: a 40% drop at the payment step, found in a day. Conversations explain why: the field silently rejected card numbers containing spaces, found in a week of talking to five people.

Reverse the order and it still works, differently — a comment in a support ticket suggests a hypothesis, and analytics tell you whether it affects eleven people or eleven thousand.

What does not work is either one alone. Quant without qual produces confident stories about causes nobody checked; qual without quant produces vivid problems that may affect almost nobody — and both feel like understanding while you are doing them.

Two sources, two jobs Two panels. Behavioural data tells you what happened and how often, at scale, but not why. Direct contact tells you why in the person's own terms, but cannot tell you how common it is. Behavioural data tells you WHAT Analytics, logs, recordings Unbiased about frequency Silent about motive Locates the problem Direct contact tells you WHY Interviews, tickets, calls Rich about motive Unreliable about frequency Explains the problem quant points the qual · qual explains the quant
Neither panel is the better one, and the line underneath is the whole figure. Read the third bullet in each: one source is silent about motive, the other unreliable about frequency — each is exactly the shape of the other’s gap, which is why a team holding only one of them still feels well informed.

Everyday example, a detective on a case

A detective uses two kinds of evidence. The security camera counts how many people came through the door and exactly when, that's the quantitative side: precise, wide-reaching, tells you what happened and how much. But the camera can't tell you why anyone did anything. For that, the detective interviews witnesses, the qualitative side: rich, explanatory, but only a few voices. Crack the case and you need both: the footage points you to the right moment; the interviews reveal the motive. Numbers find the crime scene; conversations find the culprit.

The What

Quantitative

Analytics, funnels, A/B tests, large surveys. Tells you what is happening, to how many, and how often. Great at finding where to look.

The Why

Qualitative

Interviews, session recordings, support tickets, reviews. Tells you why it's happening. Great at explaining what you found.

They work as a pair

Analytics spot the 40% checkout drop (what). Five interviews reveal the surprise shipping fee (why). Quant without qual leaves you guessing at causes; qual without quant leaves you unsure if five loud voices represent five thousand quiet ones.

Numbers find the crime scene; conversations find the culprit.

The pair, in one investigation

Analytics located a 40% checkout drop within a day. Five interviews explained it within a week: the field silently rejected spaces in card numbers. Either half alone would have left the team either knowing where without why, or why without knowing it mattered.

Quick check

Your funnel shows most sign-ups stall at the profile-photo step. You want to know why. What's the best next move?

3

Synthesis: Finding the Pattern

Synthesis is the step between having material and having something to act on. It is unglamorous, it is where the value is created, and it is the part most often replaced by summarising.

The mechanical version works well. Put every observation on its own note. Cluster by similarity before naming anything, because naming early makes you sort the rest of the notes into the names you already chose. Then label each cluster once it has formed.

Then look for the tension. Not what people said, but where what they want and what they do pull against each other. They want the tool to make the decision and they do not trust it to. They want to move fast and they cannot afford to be wrong. A cluster with no tension in it is a theme; a cluster with tension is an insight waiting to be written.

Two warnings. Do not synthesise alone — one person clustering their own notes will find their own prior beliefs. And keep a line back to the source for every claim, because the moment a finding loses its evidence it becomes an opinion with unusual confidence, and it will be quoted for years.

From raw material to insight Four steps: collect observations, group them into patterns, name the tension inside each pattern, and state the insight as something that can be acted on. 1 Observe what was said and done 2 Cluster group like with like 3 Name the tension what conflicts 4 State the insight so what, now what
Steps one and two are note-taking. The two highlighted panels are the job, and they are the two most often skipped — a cluster with nothing pulling against itself is a theme, and a theme read out at the end of a research review is a summary wearing an insight’s clothes.

Everyday example, sorting a pile of laundry

Faced with a huge pile of clean laundry, you don't start by inventing labels. You pick up items and start making piles as natural groups appear, socks here, shirts there, then a "kids' clothes" pile you didn't plan for. Affinity mapping works the same way: break your notes into individual scraps, then let the piles form from what's actually there, and only name each pile once it exists. If you'd pre-decided the bins ("pricing," "speed," "design"), you'd just cram everything into those and never notice the surprising new pile, the one that's usually the real insight.

1

Atomize

Split everything into single observations, one fact or quote per note. Ten interviews often yield 200+ notes.

2

Cluster

Group similar notes together, without predefined categories. Let the groups emerge from the data.

3

Name the themes

Once a cluster is solid, give it a name that captures the pattern ("distrust of automatic payments").

4

Write insight statements

Convert the strongest themes into: [observation], because [reason], which means [implication].

The pattern under the quotes

Twelve interviews produced ninety quotes and a summary nobody acted on. Clustered, the same material showed one tension repeated in eleven of the twelve: people wanted the tool to make a decision for them and did not trust it to. That sentence changed the roadmap; the ninety quotes had not.

Quick check

Why cluster the notes first and name the themes after, instead of starting with categories like "pricing", "UX", "performance"?

4

The Traps That Fake Insights

Some things look exactly like insight and are not. They are more dangerous than having no insight at all, because they carry the authority of research while pointing in an arbitrary direction.

The loud minority. Three furious customers produce more words than three hundred content ones. Volume of feedback measures intensity, not prevalence, and the loudest segment is rarely the largest.

The confirmed prior. You went looking for evidence about onboarding, so you found evidence about onboarding. Everything else in the transcripts was filtered out before anyone noticed it was there.

The insight with no mechanism. “Users want it to be simpler” cannot be argued with, cannot be designed against, and will be agreed to by everyone in the room.

The defences are simple and slightly tedious. Count how many people a claim came from, and write that number next to it. Keep the link back to the raw source. And ask what evidence would have disconfirmed this — if nothing could have, you did not run a study, you ran a search.

Four things that look like insight A checklist separating a real insight from four convincing imitations: the loud minority, the promoted quote, the confirmed prior, and the claim with no mechanism. Real insight Fake insight Says how many people it came from Keeps a link to the raw source Could have been disconfirmed Contains a mechanism you can design against Volume of complaint read as prevalence One vivid quote promoted to a fact Only the evidence you went looking for "Users want it simpler" — unarguable
Nothing in the right-hand column is invented — every line of it came out of real research, which is what makes it more dangerous than having no insight at all. The four ticks are procedural rather than clever: a number, a link, a disconfirming test, a mechanism. A claim that cannot supply all four belongs on the right.

Everyday example, three traps in daily life

You already know these traps from outside work. Confirmation bias: only reading news that agrees with what you already think, and feeling more sure as a result. The say/do gap: everyone who swears they'll start the diet "on Monday" , the intention is genuine, the behavior rarely follows. The loud minority: judging a restaurant by the one furious online review while a hundred happy diners say nothing. Research falls into the exact same traps, just dressed up in charts, which makes a wrong conclusion feel authoritative. Naming the trap is how you catch it.

Confirmation bias

Noticing only evidence that supports what you already believe. Antidote: ask "what would prove me wrong?" and weight surprises more than confirmations.

The say/do gap

People honestly misreport their own behavior. "I'd definitely use this" costs nothing to say. Antidote: prefer behavioral evidence, what people actually do.

The loud minority

Five passionate voices can sound like the whole market. Antidote: check every qualitative theme against quantitative scale before acting.

The insight that was a headcount

A finding read "users want better reporting" and was cited in three planning meetings. It came from four interviews, three of which were with the same customer's team, and the fourth had said the opposite. The finding was real in the notes and had been quietly promoted from a quote to a fact somewhere between the notes and the deck.

Quick check

In a survey, 80% of users say they would "definitely use" a proposed feature. What does this actually tell you?

Drill what you learned
Scenario 1 easy

Analytics show 40% of users abandon checkout at the payment step. Your CEO says: 'Obviously we need more payment methods.'

What do you do?

Scenario 2 easy

You've finished ten user interviews and have pages of messy notes. The team wants findings tomorrow.

What's the right synthesis move?

Scenario 3 easy

A stakeholder shares a survey: 80% of users say they would 'definitely use' a proposed feature. They want it greenlit this week.

What does this evidence actually support?

Scenario 4 easy

A report states: "Users who use Feature X retain 3x better than users who don't." A VP concludes: "So let's push everyone to use Feature X and retention will triple."

What's the flaw in that reasoning?

Scenario 5 easy

During synthesis, you notice one interview flatly contradicts the neat theme forming from the other nine. You're tempted to set it aside as an outlier.

What's the disciplined move?

Scenario 6 medium

Support tickets are full of complaints about a specific feature. A PM concludes "everyone hates this feature" and wants it removed.

What's the sampling trap here?

Scenario 7 medium

In an interview, a user says: "I'd pay $50 a month for this in a heartbeat." The PM writes down "users will pay $50/month" as an insight.

What's wrong with treating that as an insight?

Scenario 8 medium

You write: "Users are frustrated with onboarding." A stakeholder asks, "So what should we actually do?" and you have no complete answer.

Why does the statement fail as an insight?

Scenario 9 medium

A PM runs five user interviews, all with people recruited from the company's most-active-users list, and generalizes the findings to the entire user base.

What's the sampling problem?

Scenario 10 medium

Quant shows a sudden 15% drop in weekly usage. You have no idea why. A colleague says "let's just wait a month and see if it recovers."

What's the insight-driven response?

Scenario 11 hard

A designer asks users, "Don't you think the current navigation is confusing?" Most agree. The designer reports "users find navigation confusing" as a key insight.

What undermines this finding?

Scenario 12 hard

Your analytics clearly show WHERE users drop off (a specific screen), and interviews clearly explain WHY (a confusing label). A stakeholder says "we only trust hard numbers, ignore the interviews."

What's the counter-argument?

Scenario 13 hard

After synthesis, you have a strong qualitative theme from interviews: "users abandon because they don't trust the auto-payment." You're about to build a big fix.

What should you check before committing?

Scenario 14 hard

A team presents "insights" that all happen to justify the feature the founder wanted to build all along. The data was real, but only supporting evidence was highlighted.

What's happening, and how do you guard against it?

Scenario 15 hard

You have a genuinely strong, well-validated insight. But it lives in a 40-slide research deck that no engineer or designer ever opens.

What's the failure, and the fix?

Drilled it. Now apply it to a real situation.

Put it to work
From lesson 1

Superhuman turned a survey answer into a strategy

Superhuman

Rahul Vohra has written publicly about how Superhuman measured product-market fit using Sean Ellis's question: how would you feel if you could no longer use this product? Very disappointed, somewhat disappointed, not disappointed.

The raw number was 22% "very disappointed" — below the 40% benchmark usually cited. The data on its own said "not there yet" and nothing more.

What turned that number into an insight?

From lesson 2

Two sources, one answer

A cycling app

Route-planning usage is down 18% over two months. Two investigations run in parallel: an analyst queries the warehouse, a researcher runs seven interviews.

What each source produced
SourceFinding
Behavioural dataDrop is entirely on rides over 40km; short rides unchanged
Behavioural dataUsers who abandon do so after adding the third waypoint
Interviews"It keeps rerouting me down roads I said I didn't want"
Interviews"I gave up and used paper for the big one"

What does each source contribute, and what would either miss alone?

Select all that apply — there are 3 to find.

From lesson 3

Twenty-three notes, four themes

A freelancer invoicing tool

Twelve interviews have produced twenty-three distinct observations. The team needs to get from a wall of notes to something a roadmap can be built on.

Order the synthesis steps, first to last.

Drag the rows, or use the arrows, then check.

  1. Write each observation on its own card, in the participant's words First. Splitting to atoms before grouping is what stops you carrying an interpretation in from the interview. The participant's own phrasing preserves the detail that later turns out to matter.
  2. Cluster the cards by what they have in common, without naming the groups yet Second. Naming too early makes every subsequent card get sorted into an existing bucket rather than challenging the shape.
  3. Name each cluster with a sentence that states the tension, not a label Third. "Chasing payment" is a topic. "They will not chase an invoice because they are afraid of losing the client" is a finding — it has a because in it, and it implies work.
  4. Check each named theme back against the raw cards and against the data Last. This is where themes that felt right get discarded. A theme nobody actually said, or that the data contradicts, dies here rather than on a roadmap.
From lesson 4

Four findings, two of them fake

A meal-planning app

Four statements from the same research round, all presented with equal confidence in the readout.

Work out which ones would survive scrutiny.

  1. Step 1 of 3

    "80% of users said they would use a weekly meal-planner." What is wrong with it?

    Fake. It measures agreeableness, not demand.

  2. Step 2 of 3

    "Users who plan meals on Sunday have 2.3x the retention of those who don't." What is the risk?

    Fake as stated — real as an observation, fake as a reason to build a Sunday prompt.

  3. Step 3 of 3

    The other two: "six of eight participants abandoned at the dietary-preferences step, all citing the same confusing wording" and "four participants independently kept a paper list alongside the app". Are these sound?

    Two real findings, two fake ones, presented with identical confidence — which is the actual danger.

Notification