Business · Data

How Much Data Is Enough To Trust A Survey Result?

Abhinandan Chada Android Developer at Code Brew Labs

January 31st, 2017

I'm working on a survey to validate my business idea. Assuming I have a good sample, how many responses is enough.

Say there was just one question: "Would you use a service like this?". How many yes's should I want to have enough confidence to move forward?

Tatyana Deryugina Assistant Professor at University of Illinois at Urbana-Champaign

February 2nd, 2017

I think that depends on the product, your business model, and your target market. If you want the general population to use your product and you're planning to make money through advertising, it better be a high fraction of your respondents. But if you're planning on charging people to use the product and it's more of a niche thing, you may be ok with a small share of people who are willing to pay a lot (but then you should be sure they're willing to pay a lot). Also, I recommend making sure that people understand the product from the description by running the description past a few people and asking them what product they think is being described.

Natasha Homer-Earley Founder

January 31st, 2017

I think you need more than one question. For instance if your service is a paid service ie charges for something , then you need to test which price brackets people in your survey would pick. Ask then would they be willing to pay at all ? Typically investors like survey's of 500+ ( 1000 is ideal) and in terms of people willing to use your service, pay for your service, I would be aiming for 40-60% positive response rate. it depends on how big your potential user base is...if you get a 20% response rate but that market offers you a billion people, then that's relative, from a investors point of view. Hope that helps.

Rupert Meghnot MBA CXO, Burnout Game Ventures, LLC

February 7th, 2017

If you're just practicing Lean Startup Methodology, then I counsel my clients to ask (via personal, face-to-face interviews) at least 100 people. Pick the TOP people in your target market (if you're B2B, ID your top potential customers; if B2C, break down your target demographics & then interview people in that target market). Otherwise, you want to get at least 1000 responses (1200 preferable), for a sufficient level of confidence regarding the data received. Since the typical response rate is 2-3%, that means you need a target market email list of at least 50,000 (this is where a service comes in). The questions are fairly straightforward (after you explain the potention - non-patentable - product/service, all multiple choice): Where would you like to purchase this? How much would you pay for it? What color/configuration? etc. Good luck!

James Abel Software/Hardware Engineer

February 1st, 2017

Take a basic class in statistics, or work with someone that has. You shouldn't be conducting surveys yourself if you don't know how to answer this question. Good survey companies do know how to answer this. If you have the money, then pay them. to do it. Good ones also have the skills to ask better questions. It'll probably cost on the order of a thousand dollars or so to get a survey company to conduct the survey for you.

That being said, if you just want a rough order of magnitude number but don't care about understanding the math behind it: for any answer get a few to several hundred responses for each selection.