The Tale of Three Researchers
As former director of consumer research at Procter & Gamble, and a 20-year veteran of the company, I was often asked what makes the difference between a good consumer researcher, and a great one. I used to answer with a list of qualities of a great researcher: strong leader, great communicator, comfortable with analytics, good people skills, curious, etc. But since we all have a naturally high opinion of ourselves, everyone I shared that with assumed they possessed all of them, therefore they must be a great researcher, and don’t need to improve on anything.
Giving them an example of the work of a particularly skilled researcher didn’t work either, and for the same reason. They would conclude that if they had worked on the same project, they would have gotten the same amazing results. Again they conclude, “I’m a great researcher!”
What they need to really see the difference is an example of truly identical projects and how they were handled by a good researcher and by a great one. But that rarely happens in the real world. Not being able to find a true story of such an occurrence, I did the next best thing: I made one up.
That’s right. I just made one up. It’s ok to make up a story under one condition. That is that your audience knows you made it up. If you make up a story and your audience thinks its true, you risk losing credibility when they find out it wasn’t. But as long as you tell them up front, all is well.
Below is the story I tell them today. And, yes, I tell them I made it up.
Once there was a market research manager who had three bright young researchers working for her, but was only able to promote one of them. To determine which it would be, she gave each of them this challenge. “The one who best helps the next brand manager who comes through our door will win the promotion.”
Soon an eager brand manager came to the research department with this request, “I have several new ideas for my brand, and need you to place a concept test to pick the best one.” The research manager explained the competition, and asked that the brand manager meet with each of her researchers separately. At the end of the week, the research manager called all three researchers in to present their recommended plan. The first researcher had designed the perfect concept test. Her survey design included a separate test leg for each of the brand manager’s new ideas, plus one for the current concept to compare against. The test called for just the right number of respondents in each leg to be statistically reliable. The age, education, income, and ethnicity of the respondents were perfectly representative of the country’s population. And he had included every conceivable question in the survey to help pick the winning concept.
“Well done,” said the research manger.
When the second researcher stepped up to make her presentation, she proposed an entirely different test. “But the brand manager asked for a concept test. Why have you designed something different?” asked the boss.
She responded, “Well, when I looked at the new concepts, I realized they were all very similar. In fact, I don’t believe them to be testably different. They all describe the same product benefit as the brand currently offers, with only minor differences in the words used to explain how it works. I know he asked for a concept test, but I didn’t think that would answer the question he came in with, “Which one is the best concept?” Asking different people to assess each concept will result in scores that are all about the same for each of his new concepts. My test will show all the concepts to the same group of people and ask them pick the one with the most convincing wording. It should be easier to find a winner, and much cheaper to execute.
“Excellent,” said the research manager. Then she asked the third researcher to present his research plan.
“Actually, I don’t have one,” he said sheepishly.
“What? You’ve had and entire week. I thought you wanted this promotion as much as the others.”
“Oh I do,” replied the third researcher. “I just don’t believe this brand needs any more research.”
“I’m listening,” responded the very curious boss.
“Well, I understand the brand manager’s question was which of his new concepts was the best. But after talking to him about his business, and looking at the research we already have on that brand, I realized that wasn’t the right question to ask in the first place. The current concept scored very high when it was first tested last year. And the television commercials developed to advertise that concept are doing very well in market. Awareness is at an all-time high, and consumers are playing back exactly the concept we hoped they would. The problem with the brand isn’t the concept. The problem is price. Our value ratings have been dropping all year because our competitors have been running more coupons and sale prices in the Sunday paper.”
“So the right question to ask would have been ‘What price do we need to have to be competitive?’ But we already know the answer to that question, because we conducted a pricing study 18 months ago. We know exactly how much we need to reduce our price. We’ve never done it because we couldn’t do that and afford the increase to our advertising budget that was already approved. So I checked our media schedules and found out we’re now advertising well beyond the saturation point for our category. In fact, about 10 percent of our ad budget is completely wasted. If we cut that wasted ad spending, we’d have just enough to reduce our price back to competitive levels. And that’s exactly what I recommend we do.”
It should be no surprise that researcher number three got the promotion.
The lesson, of course, is to emphasize the value of asking the right question. Researchers are sometimes treated like “test waiters.” Business partners come in and attempt to order up a specific type of consumer test. “I’ll take two product tests and a brand equity study to go, please.” Even an average researcher can design, execute, and analyze the test asked for. A better researcher would first make sure it’s the right test to answer the business question asked. The best researcher, however, will make sure the right business question is being asked in the first place. Telling a young researcher to make sure the right question is being asked isn’t good enough. A story like this—real or imaginary—makes the point far better.
Next time you need to make a point, but don’t have exactly the right true story to illustrate it, consider inventing one. As long as it’s a plausible story—one that could happen even if it hasn’t happened—and as long as your audience knows you made it up, it will still be more effective than the boring memo or bullet-point-laden slides you were going to use.
