Interview Take-Home Tests: Good or Bad?

Aug 03, 2015

interview work programming

I'll state my bias up front: I do not like interview take-home projects. I do not like them, Sam I am. They are exploitative, they lack any ability to show realistic software development, and they shift the cost from the employer to the candidate.

Recently I interviewed at a friend's startup. I went through 2 phone screens and everything was going well. It was the kind of interviewing that I like, where the process is treated as a discussion. Then I was asked if I would mind doing a small project for them. Instead of giving a resounding "yes", I said "maybe, what kind of project" as a terrible way of weaseling out of me saying no ¹. It's hard being put on the spot during an interview. Honestly, I should have had the guts to say no, but we don't all act ourselves when in these situations (interviews are already an unnatural setting). I was told the project was intended to take "1 day" to implement and the assignment details were emailed to me a few hours after the phone screen.

I was confused as to how one defines "1 day" to implement a project, since that could mean 8 hours or 24 hours. When I received the project details I was even more confused because it contained vague language about the requirements and, more importantly, said next to nothing about the expectations. One of the other big issues for me was that it seemed to be solving a very difficult problem, for which I had to wonder if my free work would then go on to be used by the company to make a profit ².

I understand the argument from companies that interviewing is costly (in time and effort), but that's the point: you are hiring someone to make your company more successful. It should cost money and the company should feel that pain. Making candidates write take-home projects only serves to shift that cost burden onto the candidate. Maybe the candidate has another job, or a hectic family life and cannot scrape 10 hours of time together to write the code. Or, perhaps this is what the company intends to discover, given that asking questions about one's personal life are illegal in most places?

Another major issue is the project details didn't state what this project was intended to tell the company. In the project details, it stated that the project should include instructions on how to run it so that it can be tested. This leads me to many questions:

• Was a naïve implementation only required, or was it testing my ability to scale a high-traffic application?
• How thorough of testing should be provided? Unit, function, integration?
• Should a specific testing framework be used?
• Should the documentation include the methodology behind the solution, or will the tester be reading each line of code?
• Should the deliverable include the ability for it to be deployed in another environments?
• How easily configurable is the deliverable? Does it matter?
• Should the deliverable be run on a different platform (Windows, OSX, Linux)?
• How well does the code read to another developer?
• Can the deliverable be run on its own, or has it been provided with a bootstrapped environment?
• Does it show a solid understanding of the problem domain (e.g. HTTP and Web)?

But reading the project details, none of this mattered; only the deliverable mattered. I could always reach out and ask these questions to the employer, but now I'm questioning whether these questions would show a lack of skill or too much. And yes, it is possible to overthink the issue, but this is why assignments provide detailed requirements and constraints.

Turning again to the emphasis on the deliverable, speak to any teacher and they will tell you that providing the answer for the math question is, while correct, not the complete solution; you must show your work. Showing one's work can even be a good thing, as a wrong conclusion can still result in a passing grade. By only providing a deliverable, a company is silently implying that this is what they value most. No company truly thinks this way, so why put emphasis for this in the assignment?

Also remember that this is an assignment done at home and for free, all in the hope of joining a new company ³. A company is only going to get so much free programming from a candidate, so they can't be surprised when the deliverable isn't polished or feels rushed. Before, dear Reader, you think I am mounting my high horse, realize that this is not being done to join the likes of Google, MacDonald-Dettwiler, or Goldman Sachs. For all I know, the one I'm interviewing with could be gone in a year ⁴.

I have, on this website, links to projects on Github that can be perused to assess my programming skill. Some of these projects even have full test suites, including the use of mocks to handle external services. Implementing a homework project only tells the company that a deliverable can be made. It doesn't say how much of the code was written by me, nor does it show much of the code was pieced together from various websites (StackOverflow, Google, etc.). Here are some questions that a company could ask when they look at a programming portfolio rather than a forced take-home assignment:

• Was the Github/Bitbucket code good enough (for any definition of "good")? How? Why? Maybe they can expose more projects (privately or publicly).
• Did they code show that the candidate can break down a problem and solve it in an understandable way?
• Did the candidate's Git commit messages convey the correct meaning and state what changes were made?
• When collaborating on projects, was the candidate congenial and civil over Git and bug reports?

There are so many aspects to software development than just programming. This is why the list of software development services that my company provides to clients is so detailed. Having been in many office environments, the issue is rarely that someone cannot code, it is that their code is unreadable or brittle. A company hiring a software developer should be looking for candidates with good marks in all of these attributes. A take-home assignment only measures one: programming.

I'll finish with one final thought: I mentioned that a good interview feels similar to a discussion. This is good because it shows that both parties have established a relationship based on common interests, which is very useful when you have to collaborate on ideas as future co-workers. Yet a take-home project destroys that relationship, because all of a sudden the conversation ends and you must prove yourself, alone in your own home, without the give-and-take that was in the original discussions. The company takes on a huge risk in alienating the candidate, causing the her to lose interest and motivation in pursuing the job. This doesn't mean the candidate would have been a bad hire, simply that nobody likes being treated that way.

When I wrote the assignment, about halfway through I lost all motivation for it and started questioning the point. I was more interested in solving the root problem implied by the project, which was a difficult network and business scaling problem that included race conditions, but I quickly became felt like a mushroom when I what I wanted was to show that I had other skills. The code I wrote included options for 12factor to help during deployment and added an idea for dynamically-configuring a processing engine, but I didn't receive specific feedback on whether these were useful, only vaguely that my entire solution needs improvement.

I realize how this sounds harsh to the companies that interviewed me. It is meant to highlight that the take-home assignment is perhaps a flawed technique (though I do have some solutions). Just think of how this technique would be viewed in other specialist industries.

Footnotes