Thursday, September 22, 2011

Fallacious Product Management

One of the things I love about being a product manager is that it is much like being a scientist. Customer Development and Lean Startups are all about iterative experimentation, using data to understand your customers and to shape your product. While customer development and lean methodology are discussed a lot in the startup scene, the basic tenets involving hypotheses and experimenting apply just as well to product management at a company of any size.

If you think back to your elementary (primary) school science class, you'll remember that science is about finding truth, not about proving oneself correct. While easy to say, in practice, it can be very difficult and many scientists have been embarrassed as a result of not adhering to it.

Scientists and philosophers have spent countless years understanding how to make rational decisions. Knowing some of the common traps that lead us to make irrational conclusions is the first step.

Confirmation Bias

We sure do like to be right (in my house, it is called "being a Jensen" and my wife accuses me of it regularly!). We like being right so much, we unconsciously structure our lines of reasoning around the assumption we are right. In other words, we don't ask questions to prove ourselves wrong; instead, we ask questions to prove ourselves right.

In your product, this shows up as features nobody actually cares about that have data apparently showing how valuable they are. Unfortunately, the data didn't come from asking the right questions.

Are you asking questions to find the truth or to prove yourself right?

Appeals to Authority

Paul Graham said it was so. Or maybe it was Steve Jobs. No, it was Bill Gates. Wait, it was...well, it doesn't matter. When people are experts, it is worth listening to what they have to say about their field of expertise. The danger comes in listening to what they have to say about things they are not experts on and using that as a basis of your decision making (Yes, it's a fine line!).

In your product, this shows up as features added because an analyst or an investor or a VP thinks are cool.

Are you listing to experts instead of to your customers?

Misaligned Motivations

Whenever financial numbers are released in the news, it is fascinating to watch both political parties take the same information and claim how it proves their viewpoint is correct. Of course, reality is far more nuanced than that, but it illustrates an important point: we see what we are emotionally invested in seeing. The more emotionally tied we are to an idea, the more "evidence" we manage to dig up to support it.

In your product, this shows up as features that are added to support a set of customers that do not provide value to your product (the customers you should fire!).

Do you have an emotional connection that is clouding your vision?


If I asked who who was more likely to know whether a brain injury was caused by trauma, a neurosurgeon or her administrative assistant, you would probably say the surgeon. Certainly, the surgeons would. Turns out the administrative assistants were just as good as the surgeons at determining the cause of brain damage. Who do you think would be more likely to do research to find out the real cause?

In your product, this shows up as features being added based on your "deep" knowledge of the market with no quantitative or qualitative data to back them up.

Are you the neurosurgeon, completely sure you know the answers?


Which are you more likely to die from: an auto accident or a plane crash? We all know the answer is the four-wheeled death-mobiles called cars (using similarly fallacious logic, I tie risk to the number of wheels, so I ride a motorcycle: less wheels, less risk!). Yet how many people still make travel decisions based on the last catastrophe they heard about on the news? Plane crashes are rare, so they get news coverage; auto accidents are common, so they are ignored. People make decisions based on how familiar the data is, not on how valid the data is.

This is a particularly insidious trap, because you have to distinguish between things you are hearing a lot because they are important and things you are hearing a lot because people are talking about them!

In your product, this shows up as features being prioritized because "everybody talked about it at the last trade show" without any further market validation.

Are you digging to find the real needs that you wouldn't hear about otherwise?

Summing Up

As you can see, our brains are wired well for thinking poorly. It takes effort to overcome those ingrained thought processes, but the benefits of the effort are huge.

If you really want to dig in beyond these five and work on becoming a rationalist, you can dive into How the Mind Works or spend some time studying this fairly extensive list of fallacies.

What are fallacies you've found that negatively impact your product, your company, or yourself? What tools do you use to keep from making irrational decisions?

No comments: