QA testing · Bugs

Have app companies tested their app internally versus using third parties?

Brandon Bright Co-Founder at Hango

June 26th, 2015

I have tried multiple testing platforms and I am losing hope. I am starting to think that it is quicker and more efficient for me to test the app myself amongst our founding team and a small group of friends. We know the app and find way more valuable results then the third party testing services. 

Do you agree?
Which phones and OSs are you testing on and where do you get the hardware/software?

Mark Dostie CTO of Artificial Intelligence Dev

June 26th, 2015

The key to testing is to be well prepared and know specifically what you wanted tested. Do you have features that need in depth testing or you just want someone to hammer on it? Which locations do you care about? Do you need to test specific carriers? Phones? OS? Try and come up with a mixture of specific tests you want performed and "free time" testing where the testers explore.

Over the last several years I have been successful using Crowd sourced testing environments where you select your audience make-up and then they get a group of 100-1000 testers together and they run through your scripted and unscripted tests and report back. In many cases they provided not only screenshots but videos of issues we had not seen. The other nice thing is that they provided suggestions. In our case when the product was young we also got them to sign NDAs to reduce/prevent disclosure or negative feedback. In the end we ended up with some long-term customers who tested it and loved it enough to stick with it.

One last note, if you do go with external testing - be responsive to the testers. Fix what they suggest (if you agree) and get it back to them ASAP. You gain a lot of good will when you communicate and are responsive to testers.

Alex Kim Tech @ Chatgrid

June 26th, 2015

Are you early development phase? Than self-test might works. For OS, especially Android, you have to choose what model do you want to support (and device models (if your app depends on any hardware and  if want to provide service with quality. e.g. I use camera, I thought all has back and front but some only has back camera. same for btle))

After I release to beta customers, I found UX is not fully tuned and has to adjust last few weeks. I would check every input/button and interaction designed reasonable. 

At minimum I will have different screen size devices. if budget allow, popular versions.

And for the 3rd party testing service, I used services and also help other to run the service. I found if you well prepared, you will get better result. e.g. test binaries for testers, test cases, response team for any question/issues.  but most of company, they pass apps and expect bug coming in, that does not work well for functional phase testing. 


Scott Elrod mHealth technologist☁ex-COO/CIO@Cloud 9-tech for behavioral health■ex-CIO@AmeriDoc(now Teladoc) healing 1.5M patients

June 29th, 2015

Live hardware testing is rarely practical for a startup. Try https://testcloud.xamarin.com/devices
Or if you are searching for bugs/human feedback, something like: http://www.usertesting.com/plans