Chris Ledden

June 23rd, 2013

My company is building out our minimum viable product (MVP) right now and I wanted to get confirmation on the things we have included, versus the things we have put into the 'next release" list.  

I was thinking about enlisting the help of a focus group, but a few people have said to stay clear of them for your first version.  You know, the Steve Jobs' quote "people don't know what they want until you show it to them".  I, on the other hand, think that if the questions are well structured, and the goals are defined, then regardless of the stage of the product, insight can be gained.



Patrick Remy Business Strategy & Execution

June 23rd, 2013

The biggest danger in focus groups (any market research, in fact, and focus groups are rarely classified as “research”) is the beginner danger. Not knowing how to listen with the mysterious “third ear” that discerns that which you are looking for lurking in the too-many-words of what people are saying. The second biggest danger (ditto all market research) is the lamp post and drunk problem, which I also characterize as the “political” problem…used more for support than illumination. The list goes on, but I minted a concept years ago I call “value-added market research,” which requires going through the following three gating questions up front: 1. What are the questions you’re trying to answer? (clear; specific) 2. Can research answer those questions? (often, no it can’t) 3. What changes will you make as a result of the answers to those questions? (often, none) The first key market research value-add is to prevent most of it from even happening; killing the 80% that shouldn’t happen immediately quintuples the value of what remains. This kill switch comes from a realization that they don’t really know the question(s), research (of any kind) can’t answer the question(s), and/or they already know what they’re willing to know, and actions won’t change based on the research. The research is just a stall that provides more time for reflection, which may be useful, from which a direction emerges. So I just counsel then to do that well and skip the cost (and more importantly, delay) of the supposed research. There may be other ways to gain advance feedback on a product concept or a prototype product, but the typical approach to focus groups usually goes about these in an entirely wrong way. I am confident in this generality; I am also confident there are exceptions, but they don’t happen accidentally. I would estimate between 80% and 90% of market research is worthless simply because of this lack of up-front discipline(s), Maybe it’s just too hard, and responding to an edict to “do research” by going through the motions is the only way to minimize pain and fill in the mandatory check box (which is to say, lip service). Often it’s intended to make one feel better or safer; I would argue those motivations are illusions, even dangerous illusions (the lamp post and drunk problem). I speak as one who has done hundreds of focus groups and almost as many quantitative research projects, and in many/most cases, gotten great utility from them. But not when undertaken as a neophyte or following the lead of a research vendor (even at the high end…I became Burke’s (“since 1931”) biggest nightmare on a Microsoft quantitative strategy research survey on Office some years back, and wound up having to walk them through their job in way too many places). There are too many ways to fail and too few to succeed to rely on luck. So I am both a huge advocate and huge skeptic of market research. The difference is doing it right (enough) and it’s mostly about what happens in advance (like so many other things in life). …I appear to have stumbled over one of my soapboxes…sorry… Pat

Brent Goldstein

June 23rd, 2013

I think there's a big difference between a group of people that would just provide opinions, and a prospective set of customers.

People always tend to think about what would be 'nice or useful', but this is largely a mis-direction if not in the context of what a customer would actually use and most importantly, be willing to pay for.

If you can get a small set of candidate customers, then the feedback will be more focused on the minimum value-add.

And getting past "what you want" into the 'what is your real problem" is a critical step. This is the value of working directly with customers. It's true that they might not now what they need, but they should know their problems and pain points so that you can invent the best solution.


Bill Kelley

June 23rd, 2013

I may have to pull rank here a bit, having worked with Steve Jobs. 

What Steve was referring to was form factor and feature set. People do know what they want in an emotional/goal sense. In terms of major functionality, people do know what they want. In other words, people knew they wanted their lives to have a soundtrack. They just didn't know that would be solved with a memory device in their pocket.

If you have the ability to make an empathic leap over a lot of experimentation and discovery and evolving taste and visualize the ideal way your product solves the basic need, then you don't need a focus group for the MVP.

(this is a whole other discussion, but focus groups are difficult to derive actionable information from. Use a moderator who has done extensive work in prototyping to weed out comments of people trying to wear the 'product manager' hat and find the pure expression of personal interaction with the product.)

Any client exposure of product features needs to focus on the end goal, not be sidetracked by the means of interaction to use the product. 

Steve Jobs was not an inventor and he was not a designer. He was the world's best curator. 

Anyone creating and evolving a product should certainly get client input, but then be willing to accept or reject that input based on deep empathic understanding of the product's ultimate goal, which is almost always to effect a change on a more conceptual and even emotional level.  

Ron Shigeta Co-Founder at Biomatica

June 23rd, 2013

I think that most products need to be seen by potential users and other people who do development.  You needn't feel compelled to take their advice.   If you don't feel sure its worth what you want it to be, you'll often be surprised what people say or try to do when you show them your product and let them interact with it.   You might want to just show it to a few people at first rather than making a big deal of finding a demographically matched focus group.  Its amazing what you can overlook in development.  

Just a note, but I think the Apple 'they don't know what they want' thing is for people who have a lot of experience in turning out successful products, but they certainly talk to each other.  Jobs famously had a network of world-class friends and contractors who he'd show things to or give chunks of the product design to.  By no means did they make their products in a vacuum. 

I have one data analysis project we're in a similar space I would be up for trading previews... lmk

Brahm Singh Eng Head - Deep Search at Quixey

June 23rd, 2013

Adding to what others have already said on the thread,

"Never design in vacuum"

I was in Google and took a product-design class from the design-head of Google Glasses. The main point that he stressed on was that you should prototype fast and prototype often. 

When the product is in the initial stages, you need feedback. Lots of it. At that time, you haven't spent too much effort on it. It is best time to iterate with different possible prototypes - you will need to throw away the effort you spent on prototypes that did not work.

The beauty of this whole thing is - you don't always have to get feedback from your potential buyers. Sometimes - yes. Always - no. User interaction is a funny thing - you can test different user interaction prototypes (will people see this button? will they be able to hold this cup?) on any person and you will get similar results. A busy cafe is the best place.

So get out there and start testing :)

Dave Angelow Board Member at HAND Austin

June 23rd, 2013


Great comments from others and would suggest that you look at Rob Adam's and the concept/book - If you Build it Will they Come? (

A few ideas to offer, feedback is valuable but not all equal.  Everyone will have an opinion yet some more valuable than others.  Assuming you've defined your target market/buyer persona, the input from those in this category help you understand if you're solving their problem or some form of pivot is needed.  

Mimicking the sales process helps test the sales messaging as well as validate you're solving the problem.  Focus groups could be a means to get insight, yet so can applying the tools you'd use to actually sell your offering once launched.

Hope this helps

Alexander Sanchez Product Management at SmartThings

June 23rd, 2013

It seems as though enough people have weighed in, therefore I will keep this brief.

Focus groups are one way of obtaining feedback and input.  But, they're costly and subject to "flocking."  Additionally, the marginal return for every additional group interview decreases as you approach 10 groups vs. 10 participants (see imagewkBve1X9Pw -

If I were a startup and working on an MVP, I would solely rely on contextual interviews with customers and users to obtain the feedback I need to iterate.

Thomas Knoll I get to help people build their ideas and their teams

June 23rd, 2013

Why make it a group? Why not follow a more "conventional" customer discovery/problem validation process? I have learned much more from 1-on-1 conversations than I ever have from a group.

Candice Hughes, PhD, MBA

June 24th, 2013

Why not just allow customers to interact with the product/prototype the way they normally would in using it? Observe them and have them answer some pre-developed questions about the interaction. This will show you where flaws are in the design. Is it difficult to open packaging? Did the product do what it was supposed to? Did users get frustrated by any portion of the experience? Did they get confused? And so on.

A focus group may be too formal for this early stage of product evaluation and as people mentioned, responses may be altered by being in a group.

Ron Shigeta Co-Founder at Biomatica

June 23rd, 2013

I agree - I think whether you want to do a real focus group at an early stage might validate your business plan if that's the main issue you're facing.  

Like all strategic decisions, it must vary a lot from case to case.  In the end you always have to evaluate the FG feedback which will be all over the board and often ask you to go  in a new direction that might not be best.  I guess maybe the answer kind of depends on how sure you are of where you are going.  

In the end, its not clear a positive focus group means that people would buy the product.  If you need the validation for your strategy or to make sure that the actual users can use it the way you think it will or you have specific decisions to make about the product, a focus group , after several rounds of more casual demo/feedback sessions seems useful. 

I enjoyed reading the business case in "Power of Habit" where they took early versions of Febreeze out and found all these customers who absolutely had to have the product, but then nobody bought it because although the chemicals removed all odors, people didn't want to buy it because it didn't convey a feeling that the customer had done anything.  It took them a few rounds to realize that they also had to add a scent to the product.