Hiring · Code tests

How to test a programmers coding ability?

Mohsin Khan

August 5th, 2014

I am interviewing a lead programmer and I have already done the 'cultural fit' test and found they were a good match. Now I am trying to see what their coding ability is like, since they will be coding our whole site. 

This person in particular is older and has been a part of a couple start ups. They do not have any public projects that we can view, but their track record indicates they know how to code. 

Our company is a marketplace and I was thinking of having him create a craigslist crawler that could autopost items in craigslist for us. Does that seem like a fair test for their coding abilities?

Shobhit Verma

August 5th, 2014

Most of my engineer friends are tired of unpaid challenges. Though I strongly believe a project is the best test, please do pay them for their time. 


August 5th, 2014

You get what you pay for.

Roshan Diwakar CTO and Principal Consultant at Xtreme Automation Corp

August 5th, 2014

A two-week independent project is always a better fit.

A techie doing a code and design review after that project is done and delivered would be a perfect follow-up. They can talk about design decision, coding practices, algorithms etc.

A techie doing a straight interview is a no-no. In this era of multiple platforms, languages, libraries, frameworks:

{set of techie 'interviewer' questions} and {set of potential founder knowledge required for success} is almost an empty set.

Sure you can ask basic fundamental CS questions of algorithms and data structures or threads or map-reduce or nosql or javascript or rails and test their knowledge. But, none of those are predictors of success. 

For a startup, the best predictor is execution. Why not do a real world test of execution?

So, you are on the right path. Just follow it up with a design/code review with a techie.

Jessica Alter Entrepreneur & Advisor

August 5th, 2014

Based on your discussion posts, I really think you should have a technical advisor at a minimum. There are tons of great people in the FD:Advisors network. But you can't do this on your own.

Rob G

August 5th, 2014

...that would be an interesting test of his/her personal/professional ethics since what you are asking them to do is, i believe, in violation of CL's terms of use.  Of course they might also question your ethics for asking.   As an FYI, there are likely CL crawler projects on GitHub and i know there are some Ruby gems that do what you are suggesting. 

Michel Floyd Founder, cloak.ly

August 8th, 2014

One of the best techniques I've seen is a "pair programming" session with the candidate's future manager or someone from that team. The interviewer sets up a problem, typically a real one they are familiar with, and then they work with the candidate to solve it over the course of a couple hours. The real value here is that the interviewer can directly follow the candidate's thinking processes and assess their familiarity with the tools they're going to be using. This actually works remarkably well remotely, with a screen share session.

This is not a hypothetical answer, I've hired fantastic developers this way and my failure rate (bad hires) is essentially zero.


August 5th, 2014

Not in my opinion. That challenge seems pretty easy and doesn't test one critical thing: writing flexible code. I think it is good to test them by giving them a challenge, but telling them that part of the challenge is that you will later change or add one of the requirements. The change shouldn't be big. That way you can tell if he makes brittle code that isn't flexible to changes in the real world or not.

Quinn Goldstein Consultant at GCG

August 5th, 2014

agree get a techie to help you evaluate their tech skills. 

David Berry Lead Android Engineer at Moxie

August 7th, 2014

As somebody that's been doing contract development for most of the last thirty years, the only reasonable ways to approach this without running off a lot of talent is to either establish expertise through an interview process or a short-term contract as an evaluation period, or both.

What Stephen is saying is spot on.  Every experienced developer has been taken advantage of through the "solve our problem in the interview process so we don't have to hire you" and would wisely shy away from doing anything that looked like free work.

Owen Rubel Creator of API Chaining, IO State, API I/O Abstraction and modern API Automation

August 8th, 2014

See  this is a turnoff; creating an application, hosting it and coming in and explaining it. Ok sure... maybe if you were Google. But for everyone else... forget it. This is why I say '15 minutes'. It would be like me saying to a Marketing person, 'create me a marketing campaign, now put it up on all social media sites, now come in and explain it.' SERIOUSLY?

CRUD may be simple but it is time consuming as you are explaining. And alot of developers will look at any test that is taking them longer than a few minutes to do and go 'screw this'; I know I've done it. The only people who will do long, drawn out tests are the desperate.

You need to respect the fact that they have OTHER interviews and other things they do in their spare time. 15 minutes is what you should ask at a max and more than that is asking too much.

You are not giving them an exam and if you are, you are insulting them... and that is not the way to start a relationship.