Customer Feedback vs Behavior Data — What People Say and What They Do

Short answer: feedback is reliable for motives, feelings, and context, and unreliable for predicting behavior or explaining small usability problems. Behavior data is reliable for what happened and how often, and silent on why. They disagree often, and the disagreement is informative. Use behavior to find the problem and feedback to understand it.

Table of Contents

1. The Gap Between Saying and Doing
2. What Each Is Good For
3. When They Disagree
4. Combining Them
5. Asking Better Questions
6. Key Takeaways

The Gap Between Saying and Doing

People are not lying when their words and actions differ. They are:

• Bad at predicting themselves. "Would you use this feature?" gets a polite yes.
• Bad at remembering detail. Nobody recalls which field confused them last Tuesday.
• Rationalising afterwards. A reason is constructed to fit what they did.
• Being kind. Especially to a founder asking in person.
• Answering a different question. "Too expensive" often means "I did not see the value".
• Unaware. They do not notice the three seconds they spent hunting for a button.

Meanwhile, behavior data records the hunt for the button and cannot say what they were thinking.

What Each Is Good For

A rule of thumb: ask people about their goals and past experiences. Watch them for anything about how they use an interface.

When They Disagree

Disagreement is not a reason to discard either. It is a prompt. "They say it is easy and take two minutes to find the export button" means the task is harder than they realise, and probably harder for people who gave up and were never surveyed (survivorship bias).

Combining Them

Start with behavior to find where.

1. A funnel shows the step that loses people.
2. Recordings show what happens there.

Then use feedback to find why.

1. Ask people who were at that step. Their behavior gives you a specific, concrete question: "I noticed people often pause at the plan selection. What were you weighing up?"

Or start with feedback and check with behavior.

1. Support tickets say "I cannot find invoices".
2. Recordings show how people look for them and where they look first.
3. The count of sessions searching settings tells you how widespread it is.

Feedback gives hypotheses. Behavior tests them, and sizes them.

Identify users in your analytics so that someone who gave feedback can be found in recordings, and so survey answers can be compared with what that person did:

deepsync("identify", user.id, { plan: "pro" });

A single open question at a key moment is cheap and valuable: "How did you hear about us?" at signup, "What nearly stopped you?" after purchase, "What is the main reason you are leaving?" at cancellation.

Asking Better Questions

• Ask about the past, not the future. "Tell me about the last time you did this" beats "Would you use…?"
• Ask for specifics. "What did you do next?"
• Avoid leading. Not "Was the checkout confusing?" but "How was checking out?"
• Ask why, more than once.
• Do not pitch. You are there to learn.
• Watch for politeness. Strong praise with no usage is a warning.
• Ask non-customers and former customers, not only fans.
• Keep surveys short. One to three questions.
• Time the question to the moment, when memory is fresh.

Treat any single piece of feedback as one data point. One customer's request is an anecdote. The same request, with behavior showing many people attempting the task, is a finding.

Key Takeaways

• Feedback explains why; behavior shows what and how many.
• Disagreement between them is a lead, not an error.
• Ask about past experience, not hypothetical futures.
• Size every piece of feedback with behavior before acting.

Conclusion

Listen to what customers say, and watch what they do. Neither alone is the truth. Together they are close to it.

Start free with DeepSync.