How to test a programmers coding ability?

August 5th, 2014

I am interviewing a lead programmer and I have already done the 'cultural fit' test and found they were a good match. Now I am trying to see what their coding ability is like, since they will be coding our whole site.

This person in particular is older and has been a part of a couple start ups. They do not have any public projects that we can view, but their track record indicates they know how to code.

Our company is a marketplace and I was thinking of having him create a craigslist crawler that could autopost items in craigslist for us. Does that seem like a fair test for their coding abilities?

Shobhit Verma

August 5th, 2014

Most of my engineer friends are tired of unpaid challenges. Though I strongly believe a project is the best test, please do pay them for their time.

Regards,

Shobhit

Roshan Diwakar CTO and Principal Consultant at Xtreme Automation Corp

August 5th, 2014

A two-week independent project is always a better fit.

A techie doing a code and design review after that project is done and delivered would be a perfect follow-up. They can talk about design decision, coding practices, algorithms etc.

A techie doing a straight interview is a no-no. In this era of multiple platforms, languages, libraries, frameworks:

{set of techie 'interviewer' questions} and {set of potential founder knowledge required for success} is almost an empty set.

Sure you can ask basic fundamental CS questions of algorithms and data structures or threads or map-reduce or nosql or javascript or rails and test their knowledge. But, none of those are predictors of success.

For a startup, the best predictor is execution. Why not do a real world test of execution?

So, you are on the right path. Just follow it up with a design/code review with a techie.

Jessica Alter Entrepreneur & Advisor

August 5th, 2014

Based on your discussion posts, I really think you should have a technical advisor at a minimum. There are tons of great people in the FD:Advisors network. But you can't do this on your own.

Rob G

August 5th, 2014

...that would be an interesting test of his/her personal/professional ethics since what you are asking them to do is, i believe, in violation of CL's terms of use. Of course they might also question your ethics for asking. As an FYI, there are likely CL crawler projects on GitHub and i know there are some Ruby gems that do what you are suggesting.

Michel Floyd Founder, cloak.ly

August 8th, 2014

One of the best techniques I've seen is a "pair programming" session with the candidate's future manager or someone from that team. The interviewer sets up a problem, typically a real one they are familiar with, and then they work with the candidate to solve it over the course of a couple hours. The real value here is that the interviewer can directly follow the candidate's thinking processes and assess their familiarity with the tools they're going to be using. This actually works remarkably well remotely, with a screen share session.

This is not a hypothetical answer, I've hired fantastic developers this way and my failure rate (bad hires) is essentially zero.

Anonymous

August 5th, 2014

Not in my opinion. That challenge seems pretty easy and doesn't test one critical thing: writing flexible code. I think it is good to test them by giving them a challenge, but telling them that part of the challenge is that you will later change or add one of the requirements. The change shouldn't be big. That way you can tell if he makes brittle code that isn't flexible to changes in the real world or not.

Quinn Goldstein Consultant at GCG

August 5th, 2014

agree get a techie to help you evaluate their tech skills.

Anonymous

August 12th, 2014

I love designing garages, I had an interview where I was asked about this, I asked additional questions about Snow Load and R values for the insulation, these were all questions asked of me when I got my building permit for my garage at my cabin.

They just looked puzzled.

It was a short interview :)

No that I have decided never to work for someone again, I never have to put myself thru this punishment.

Joseph Foster Fostercode Senior Software Engineer

September 24th, 2014

I don't have candidates "test out" for their job. I utilize their resume, try to contact their previous employers and get a sense of their skill level.

I develop questions that are open-ended and designed to have them talk through a situation so I can ascertain their line of thought and thus determine their skill level.

The things I look for are not in terminology (some of the best developers I've seen call everything by the wrong name)...I look for where they would go, what resources they would use. Is this a process they are familiar with, did they bring up the road blocks they expected...

I question them, pointedly, on common issues that arise or why they chose a certain approach (whether it's the same as mine or not)...

Ultimately, you shouldn't really have to ask very many hard technical questions or "test" people at all. That will drive away some of those lead developers with prized skill sets.

I'll just add to that, I have never hired someone who wasn't at least as qualified as I needed (whether they were aware or not). I have a 100% success rate, skill wise...personality wise I tend to rely on others ;)

Roshan Diwakar CTO and Principal Consultant at Xtreme Automation Corp

August 7th, 2014

Andy,

I disagree with a 2 hour test. A developer's yield is an exponential curve (and not linear). There are plenty of developers who struggle the first couple of hours or they are taking time to analyze the problem, collect their thoughts, sleep/shower over things and they get a brilliant insight (sometimes far superior than the ones who gave a quick solution)

A two hour test will never capture those leaps in insights. You can't linearly extrapolate a developer's ability from the first two hours of data points.

Also, a two hour test will only capture the developer's L1/L2 cache. Not the long term persistent wisdom. In the 80s/90s, You can fit the entire C language/Library in your L1 cache and come up with solutions just using that.

However, today's solution require the evaluation of pros and cons of multiple databases, frameworks, libraries, languages, techniques, algorithms, data-structures. If I'm deep into functional javascript and someone suddenly throws me an ORM problem, I'll struggle the first few minutes if not the first few hours till my brain is properly context switched and accessing the deep wisdom of ORM stored in my long-term memory.