650.823.3042
Focus groups are ubiquitous. In many marketers’ minds the phrase “focus group” is synonymous with the phrase “market research”. Why? Focus groups are easy to understand, easy to set up, fairly quick to turn around and, best of all, the results are almost always, if the moderator has any skills whatsoever, sufficiently vague to support any conclusion the client wants supported. This makes focus groups particularly popular among ad agencies.
Having said that, focus groups serve a vital role in research. If the marketing problem is the battlefield, then focus groups may be the Marines, hitting the beachhead first, digging in and establishing a foothold on foreign soil. An important, often essential, task. But they’re not the entire Army, Navy and Air Force. They’re just not.
Focus groups are a powerful tool that is overused and misused. They have two primary advantages:
But these advantages create disadvantages. The primary disadvantage of qualitative research methods in general, and focus groups in particular, is that they are unreliable predictors of the population. That is, they can expand our list of possibilities, but they cannot (or should not) be used to identify the best of the possibilities.
I have often heard the following logic: “We did 10 focus groups and we had 10 participants per group. So our sample size was 100. And they all liked our new product idea! That’s gotta be significant!”
No, actually it isn’t. The concept of sample size is based on the assumption of independent responses. Flip a coin 100 times. Each flip is independent of the other flips. Record the result, heads or tails, and you have 100 data points. A sample of 100. But the responses of focus group participants are not at all independent. Interactions between participants affect opinions in the same way juror deliberations change verdicts. You might argue that 10 focus groups, with ten participants each, have a sample size of 10. The results from each group are, after all, independent, right?
Well, the results may (or may not) be independent but there’s another problem. The notion of sample size is also predicated on the assumption that the data being collected are identical. Like 100 flips of a coin. It wouldn’t make much sense, for example, to ask 50 people their favorite color and another 50 their favorite song and then claim you have a sample of 100. But that’s exactly what can happen in focus groups. The questions are not always the same, the context is not the same. Sometimes, even the moderators are not the same. The answers from one group cannot, strictly speaking, be compared to the answers from another group. The great power of focus groups to adjust on the fly and pursue a course of inquiry that was unanticipated is also their great limitation.
It’s quite simple, really. Because of the advantages and disadvantages discussed here, focus groups are generally appropriate for just two purposes:
They are not appropriate for evaluating pre-existing ideas. Let this be your mantra: Focus groups are generative, not evaluative. In other words, focus groups are for mucking around, exploring. Like a kid off in the woods, looking under rocks just to see what’s there or stealing off into Grandma’s attic and rummaging through her old boxes. You never know what you’ll find and you often find something surprising and wonderful. But what you won’t find is an inventory of items that reflects all attics everywhere.
Quantitative research is exactly the flip side of this coin and has two advantages:
The primary disadvantage of quantitative research is that issues are only measured if they are known prior to the beginning of the survey (and, therefore, have been incorporated into the questionnaire). Each respondent is asked exactly the same question, in the same way, with no outside influences.
Thus, quantitative research is appropriate when:
Generally, quantitative research is not appropriate as an initial learning phase or as a method to develop creative ideas. Another mantra: Quantitative research is evaluative, not generative. Ah, the beauty of symmetry, yes?
No, not quite. Focus groups have another trick up their sleeve. A trick that’s very hard for quant methods to match. And that trick is creative insight. By creative insight, I don’t mean advertising. I mean taking a quantum leap from where you are over a dark abyss and landing somewhere else. Somewhere that logic cannot lead you.
A few years ago, I moderated 18 focus groups in four countries. Category was ice hockey. Purpose of the groups was to explore new product ideas across the complete line of hockey equipment, head to toe. Talking to these young players for hours and hours about their gear (and then rehashing it all with the client afterwards) eventually gave me a unique insight as to how they viewed the category and various brands.
We hadn’t intended or expected to uncover a segmentation model of the hockey category. But in a Toronto hotel room in the middle of the night, I feverishly (literally) sketched out a segmentation scheme that was to revolutionize the hockey industry.
Mucking about with players, just talking about gear, what they liked and didn’t and why, and then discussing the group results long into the night each day of the trip with my client, an insight, emerged that gave the client powerful leverage in the category. Brand naming and positioning strategies were tailored to this segmentation scheme, new products were developed to target specific segments and their unique needs. The segmentation model, once imagined, and the consequent branding strategies were confirmed and refined with quantitative research. But they were born from focus groups.
Hockey marketing remains today profoundly affected by the creative insights that came accidentally from these groups.
It is difficult for any quant methods, so rigidly based on logic and rules of inquiry, to allow a creative leap to a fundamentally new place. Einstein’s genius was not that he was able to invent and solve difficult equations but that he was intellectually free enough to jump over previously unquestioned assumptions. That creative leap is difficult to manufacture. Focus groups, and other qualitative techniques, have a unique ability to help us do just that. They can take us places quantitative methods can never go.
Prior to founding MACRO, Mr. McCullough was the senior vice president for a leading international strategic consulting and design research firm. Before that, he was market research manager for a $300 million division of Levi Strauss & Company.
Mr. McCullough has spoken before the American Marketing Association, the Marketing Research Association and various universities on a wide range of marketing topics. He is a past board member of the American Marketing Association, San Francisco chapter, and has been quoted in Canadian Packaging Magazine. He has chaired and lectured at applied multivariate statistics seminars for several years and is a member of numerous professional associations.