How to test a programmers coding ability?

August 5th, 2014

I am interviewing a lead programmer and I have already done the 'cultural fit' test and found they were a good match. Now I am trying to see what their coding ability is like, since they will be coding our whole site.

This person in particular is older and has been a part of a couple start ups. They do not have any public projects that we can view, but their track record indicates they know how to code.

Our company is a marketplace and I was thinking of having him create a craigslist crawler that could autopost items in craigslist for us. Does that seem like a fair test for their coding abilities?

Jessica Alter Entrepreneur & Advisor

August 5th, 2014

Based on your discussion posts, I really think you should have a technical advisor at a minimum. There are tons of great people in the FD:Advisors network. But you can't do this on your own.

Rob G

August 5th, 2014

...that would be an interesting test of his/her personal/professional ethics since what you are asking them to do is, i believe, in violation of CL's terms of use. Of course they might also question your ethics for asking. As an FYI, there are likely CL crawler projects on GitHub and i know there are some Ruby gems that do what you are suggesting.

Anonymous

August 5th, 2014

Not in my opinion. That challenge seems pretty easy and doesn't test one critical thing: writing flexible code. I think it is good to test them by giving them a challenge, but telling them that part of the challenge is that you will later change or add one of the requirements. The change shouldn't be big. That way you can tell if he makes brittle code that isn't flexible to changes in the real world or not.

Quinn Goldstein Consultant at GCG

August 5th, 2014

agree get a techie to help you evaluate their tech skills.

Roshan Diwakar CTO and Principal Consultant at Xtreme Automation Corp

August 7th, 2014

Andy,

I disagree with a 2 hour test. A developer's yield is an exponential curve (and not linear). There are plenty of developers who struggle the first couple of hours or they are taking time to analyze the problem, collect their thoughts, sleep/shower over things and they get a brilliant insight (sometimes far superior than the ones who gave a quick solution)

A two hour test will never capture those leaps in insights. You can't linearly extrapolate a developer's ability from the first two hours of data points.

Also, a two hour test will only capture the developer's L1/L2 cache. Not the long term persistent wisdom. In the 80s/90s, You can fit the entire C language/Library in your L1 cache and come up with solutions just using that.

However, today's solution require the evaluation of pros and cons of multiple databases, frameworks, libraries, languages, techniques, algorithms, data-structures. If I'm deep into functional javascript and someone suddenly throws me an ORM problem, I'll struggle the first few minutes if not the first few hours till my brain is properly context switched and accessing the deep wisdom of ORM stored in my long-term memory.

Anonymous

August 12th, 2014

I love designing garages, I had an interview where I was asked about this, I asked additional questions about Snow Load and R values for the insulation, these were all questions asked of me when I got my building permit for my garage at my cabin.

They just looked puzzled.

It was a short interview :)

No that I have decided never to work for someone again, I never have to put myself thru this punishment.

Shobhit Verma Ed Tech Test Prep

August 5th, 2014

Most of my engineer friends are tired of unpaid challenges. Though I strongly believe a project is the best test, please do pay them for their time.

Regards,

Shobhit

Roshan Diwakar CTO and Principal Consultant at Xtreme Automation Corp

August 5th, 2014

A two-week independent project is always a better fit.

A techie doing a code and design review after that project is done and delivered would be a perfect follow-up. They can talk about design decision, coding practices, algorithms etc.

A techie doing a straight interview is a no-no. In this era of multiple platforms, languages, libraries, frameworks:

{set of techie 'interviewer' questions} and {set of potential founder knowledge required for success} is almost an empty set.

Sure you can ask basic fundamental CS questions of algorithms and data structures or threads or map-reduce or nosql or javascript or rails and test their knowledge. But, none of those are predictors of success.

For a startup, the best predictor is execution. Why not do a real world test of execution?

So, you are on the right path. Just follow it up with a design/code review with a techie.

Michel Floyd Founder, cloak.ly

August 8th, 2014

One of the best techniques I've seen is a "pair programming" session with the candidate's future manager or someone from that team. The interviewer sets up a problem, typically a real one they are familiar with, and then they work with the candidate to solve it over the course of a couple hours. The real value here is that the interviewer can directly follow the candidate's thinking processes and assess their familiarity with the tools they're going to be using. This actually works remarkably well remotely, with a screen share session.

This is not a hypothetical answer, I've hired fantastic developers this way and my failure rate (bad hires) is essentially zero.

Stephen Packer Lead Developer at Lettuce Box LLC

August 5th, 2014

I agree with Shobhit. As a techie, I believe a "code test" can be one of two things:

Related to the company [PAID]. If you pull a backlog item off your list and assign it to the interviewer, you should pay the interviewer just as you would a contractor. Good coders have been though this enough times to understand and dislike being taken advantage of for free labor. This may also be considered as a contract-to-hire situation.
Unrelated to the company [UNPAID]. This should have absolutely nothing to do with your company or the features you are building or have recently built. This will help the interviewer feel like they're being evaluated rather than exploited for free labor. This should also be less than a day of work, ideally an hour or two (contrary to popular belief, some programmers do have lives).

A code test is really intended to review code style, cleanliness, and general knowledge of the language itself. Another programmer can usually look at a code sample and approximately gauge their skill level. You can only do so much testing and evaluating before biting the bullet and making the best decision on limited information, and beyond a certain amount of effort from the interviewer you're going to start chasing away the experienced, solid, and jaded programmers that would benefit you. Like Romena said, I think it really comes down to protecting yourself with a probationary period or using contract-to-hire.

Question stats