Contractors · Hiring engineers

What questions would you ask to vet a contract development company?

Rachel Ratliff Voki Mobile

October 7th, 2014

I'm a non-technical solo founder, and I hired a US development company last year to work on my app. I just had to fire them for being unable to get the job done, and hired a new company in the Ukraine on the recommendation of a developer friend. They seem good so far. But I and other non-technical founders are handicapped by having no idea what to look for in a developer, so I thought it would be useful to have a list of questions that could provide at least an initial vetting of companies. Of course, ideally we would also have a technical advisor interview for technical skills, but this would provide a good starting point. Here are some examples of learnings I acquired from my recent experience, would love to hear other thoughts:

How many developers do you have available in each specialty [iOS, android, back-end, whatever your needs are]? If your contractor doesn't have depth of expertise,  your project will get put on hold for vacations, family emergencies, etc.

Will I have direct access to communicate with the developers when I need to? If you have to communicate through the project manager without direct access to developers, communication will take longer (read, delays), and you may not get the full story on what issues are arising. 

How often do you meet with your clients during a project? The answer should be weekly or bi-weekly at the very least, plus as needed.

I'd like to see a sample project plan. if it's not very detailed and organized, with individual features/tasks broken out by amount of time, then it may indicate they have no idea how to plan for a project, which will lead to delays.

Do all of the employees work full time at your company? Some smaller shops may use contractors themselves, or be a lose amalgamation of coders, which leads to you having even less control over the project, and most likely delays. 

is there a dedicated project manager? Not necessarily dedicated to your project, but dedicated to the task of project managing. If it's the designers or coders themselves having to manage the project it won't be done properly, which will, you guessed it, lead to delays.  



 

Seth Caldwell Creator SnapChallenge

October 7th, 2014

It would be wise to hire someone technical that has participated in multiple successful project deliveries that can look at the work the team is producing, and have a weekly status call. You would only pay for 2-3 hours of their time a week (likely around $200-400), but it will both motivate the team to deliver well (because team will be made knowledgeable about this person's past work and feel like they can't 'sweep under the rug' anything that normally they would when reporting to a non technical person). They may also be motivated to impress and feel like they are a part of something more legitimate with someone like this on board. Honestly the project plan broken out with individual features is something non technical people love to see, but in my opinion is like the chocolate swirls on the plate that are mostly for decoration. The meat of the work is yet to be discovered in most cases, and while plans are nice, technical people often schedule an entire day for a task that is done in 30minutes, just to be safe and give stretch time for testing, integration, and rewriting. You won't realize this being non-technical, but oddly, even someone that can complete the task in 30mins may agree that a day is an ok estimate. The problem is when the contractor is contracting for multiple people, completing these 30 min tasks (hastily with bad unreviewed, untested code), then billing for an entire day. This is why you need someone to oversee their work. If you don't, they _will_ deliver bad quality and bill for high quality (most of the time, especially when overseas).

Corey Blaser

October 7th, 2014

References, you need references. Ask for real companies that you can call and make sure you get the person/s they actually worked with. 

You will want to ask about the quality of the work, how are their communications, would they go with them again (and if they have done multiple projects with them), and whether they kept to their timelines and budgets. And if they pass that, make sure that you can get along with them as individuals. Every company has a personality, some you will mesh with and others you will not. It doesn't mean they are a good or bad company, it means they may or may not be right for you.

Kevin Goldstein IT

October 8th, 2014

I think Josh's approach was good, but it exemplifies some of the difficulties in dealing with outside consultants (time estimates, delivery dates, cost, implementation methodologies, etc). It is extremely important that you have an understanding of what you want your application to do, how it will look, where it will run, etc. The more granular the approach, the better, with clearly defined priorities set by you, not allowed to grow organically (such as a good look and feel is more important than speed, low cost is more important than maintainability, etc). SLA's and expected behavior of the final product should be considered before hand, and built in to the original agreement. Additional things to be considered: Who is going to maintain the product? who is going to do upgrades on the product when new features are required? How are they delivering it to you? What penalties will be incurred if they are late on delivering? (this last one has more to do with your expected go live date and cost of the original product). What software stack is being used and are there licensing issues? 
It has been my experience, that almost by definition a contracting agency will miss their original time projection. Knowing this in advance will allow you to plan better from the business side, and you can estimate how much leeway is available to you. A good project plan with clearly identifiable and verifiable milestones will help you with estimating the project's status, and it's the responsibility of the agency to provide one with you.
Larger concerns about architecture and overall software design approach can also be pretty well estimated from the project plan, as well as pre-implementation documentation that they should provide.
Good luck!

Josh Moser Founder at VinoServant, Inc.

October 7th, 2014

Rachel - I am not a technical person either. I hired a firm in India to build a site and application for my project. I put together a word document (13 pages) that discussed the following: technology, how to bid, milestones and then all the features and functionality. I then followed this up with a 20 sheet excel document that broke out all the screens. Using these two documents I then interviewed 40 firms in the US, India and China. When I decided on the firm I wanted to use, I then told them to take my documents, and put together a spec document (software requirements specifications). I paid them $500 for this document. My thinking was that if they couldn't create a document that laid everything out, then they wouldn't be able to build my site and application. Initially, they thought the document would take 2 weeks, but there were a number of revisions (7 to be exact), so it took 5 weeks. No big deal at the end of the day. During the project when a question about functionality or features came up, we just went back to the document. It saved a ton of time, and everyone was on the same page. I think I only had 2 or 3 conversations with the development firm via Skype. We used Basecamp for all communication regarding the project. In the past, I have forwarded these docs to other FDers. Let me know if you want to see the docs - 

Sridhar Rajagopal

October 7th, 2014

A book that I read a while ago comes to mind - "App Empire: Make Money, Have a Life, and Let Technology Work for You" by Chad Mureta.

An interesting read, it goes into the kinds of considerations you would have with a development team, especially as a non-technical person (Chad Mureta was non-tech) and what technologies, processes and procedures should you be aware of or learn about. You don't have to have or want an App Empire, but the lessons would still come handy for you.

Arif Minhas Senior Director - Head of IT Ops, Apps Support, Infra, NW, Data Center & Service Desk at CIB Bank

October 9th, 2014

Hi Rachel; A couple of key items that are essential for both parties: 1. Are they asking you for a BRD (Business Requirements Doc)and a FSD (Functional Specs Doc … which the Dev Co will have to enhance) 2. So you need to have these prepared upfront - else they may be good dev & PM but will build something quite different to what u need 3. Assuming you have done your home work and already prepared a BRD and an FSD of sorts. 4. Does the dev co use any standard dev platform/tools or do they code in generic languages --- there are many pros/cons 5. Will they do a mockup first (say using a tool called Balsamic Mockup) - you can use this yourself and give them a very precise idea of what you want. I found this valuable myself and it speeded up the dev. 6. Next is horizontal and vertical scalability of the app infra - this must be properly designed 7. The data structure is another key element to properly understand and design for. Regards; Ari Arif Minhas CReative Minds 5467 Bimini Court Mississauga, ON L5M 6T9 Cell: 647-221-1641 Home-Off: 905-813-0065 CONFIDENTIALITY NOTICE: Proprietary/Confidential Information belonging to CReative Minds and its affiliates are contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and you are asked to notify the sender by reply email.

Aleksandra Czajka

October 9th, 2014

Not sure it was said before and i don't want to throw out a one liner here, but, don't hire a company if it's a startup project. partner with someone that has her benefits and desires sown into the success of the company. there's more reasons why but i won't waste my time explaining if you're not interested in this direction

Tony Daou Entrepreneur, Purchasing Professional, Engineer and Leader with Supply Chain, Manufacturing and Process Eng. experience

October 8th, 2014

Andrew Hopper Technology & Design Strategist

October 8th, 2014

I echo what other's have said about hiring a technical advisor or adding one to your board who can supervise the development work.

That said, these tips should help in the meantime.  

1.) Look for evidence that the development shop is using industry standard best practices as this is an indicator of higher quality.  Some keywords you can search for include: TDD, test driven development, and continuous integration.

2.) Make sure the development shop uses a framework like Ruby on Rails, Django or Grails.  These frameworks have predefined ways to accomplish  tasks that reduce the likelihood that you'll receive poor quality work.

3.) Make sure they use GitHub or an equivalent and turn on email notifications of updates.  GitHub is a tool software developers use to collaborate and store versions of code they're working on.  By monitoring GitHub you can track productivity and you'll also begin to understand how things work like when you ask for a copy change.

4.) Make sure the development shop is using continuous integration and ideally continuous deployment.  Continuous integration is a process where all code is tested each time updates are made to make sure each module functions correctly with the other modules.  Continuous deployment will make these changes available on a server for your review immediately.  

If you're looking for a dev shop in the US, these are fantastic:
http://www.cantina.co
http://www.viget.com
http://www.terriblelabs.com
http://thoughtbot.com

Steve Owens

October 7th, 2014

There is a good list here: http://www.finishlinepds.com/resources_faqs.html Also, some good white papers on the subject here: http://www.finishlinepds.com/documents.php Get a good technical person on your board of advisors and give him a little skin in the game. Regards, Steve Owens - Finish Line PDS A Better Way for Small Companies to Develop Products e | Steve.Owens@FinishLinePDS.com p | 603 880 8484 w | www.FinishLinePDS.com 94 River Rd | Hudson, NH | 03051 Click for Product Development White Papers