How Effective are Technical Interviews?

Pulse, the startup I work at, is looking to hire a few more developers, and so I’ve recently been thinking a lot about the hiring process and my own experiences with interviews.

The worst interview I ever had was a phone screen for an internship, during which two interviewers spewed Java trivia one-liners at me on their speakerphone. At first I felt pretty bad because I wasn’t that experienced in Java at that point, and felt that I had done pretty poorly in the interview.  I later reasoned that I wouldn’t want to intern at a company that didn’t probe more deeply into how I tackled problems anyways.  That is the extreme end of technical interviews, but I’ve always wondered how effective technical interviews are at selecting the right people for the job.  My suspicion is that people who do well in interviews are generally fine, but that technical interviews may weed out candidates that don’t interview well but would actually do a good job.

I think of technical interviews as the kind where you go into a room with an interviewer and solve problems on the whiteboard for an hour.  Rinse and repeat 4-8 times in one day, depending on the company and type of job.  For those of you who feel calm and collected and confident during such interviews, I admire you. During the first few interviews I had for internships, I remember being so nervous that I couldn’t really think straight. My mind would sort of blank, and then think “oh shit, I need to solve this problem, think think think.” Mentally telling yourself to think while some guy stares at you is pretty freaking stressful, and it’s definitely not too conducive to problem-solving.

Stress issues aside, do technical interview problems map directly to on-the-job work? Many technical interviews are very theory-heavy, muddling the divide between Computer Science and Software Engineering.  I don’t claim to be any sort of expert on interviews and hiring, but it seems to me that a typical interview also isn’t a great indication of how well candidates will work in a team, their coding style, how passionate they are about the company, and their work ethic.

A few months ago, I had some really positive experiences with hiring processes that I believe give candidates more opportunities to showcase their skills, while giving companies more useful and relevant feedback.

The first one that really stood out to me was at a local startup.  First, I met with someone from the company for coffee to learn a bit more about the company.  I was then given a coding challenge in the form of written specs for a small Android app, which I coded up in a few hours and sent back to them.  They looked it over and brought me in for interviews, during which they asked me about my general design for the Android app.  I was also asked more conventional technical questions and coded up solutions on the whiteboard, but it was well-complemented by the coding challenge.  As a candidate, I felt that this gave me a good opportunity to showcase some of the skills that would not necessarily come across in an on-site interview, such as good coding style and design.

The other really positive hiring process I went through was at Pulse, where I ended up.  I met with the engineering team (3 people) for lunch one day, and then worked onsite on some small projects for the next week.  I was able to jump right into the codebase and show the team what I could do (I was also paid for the week), and when I accepted a full-time offer, I already knew how exciting and fulfilling my new job would be.  A friend recently commented that finding the right startup is just like finding the right relationship.  A “trial period” (maybe analogous to dating?) gives both the candidate and the existing team the opportunity to test the waters and see if the relationship is a good fit.

Both these types of processes allow the candidate to write some not-too-trivial code at his or her own pace. For software engineers who are already experienced in the job-specific platform, their expertise will shine through.  For smart people who are less experienced, they have the opportunity to show that they can learn quickly and get stuff done, without having to worry about getting tripped up on syntax in an on-site interview.  With the growing number of startups and companies hiring developers in Silicon Valley, and the dearth of available candidates with the “required years of experience,” this type of hiring might be an effective way to hire “outside of the box.”

I haven’t had too much interviewing experience (on either end), so I’m not sure how uncommon these hiring strategies are.  For larger companies, they may not even be logistically possible. Has anyone else gone through a process that was different from the more conventional technical interviews? For those of you who have more experience interviewing candidates, how effective are the hour-long technical interviews?