We studied thousands of coding interviews. The result may change how you prepare for them

Note: I wrote most of the words in this post, but the legendary Dave Holtz did the heavy lifting on the data side. See more of his work on his blog.

If you’re reading this post, there’s a decent chance that you’re about to re-enter the crazy and scary world of technical interviewing.

Maybe you’re a new graduate going through the interview process for the first time or an experienced software engineer who hasn’t even thought about interviews for a few years. Either way, the first step in the interviewing process is usually to read a bunch of online interview guides and to chat with friends about their experiences.

There are a few issues with this typical approach to interview preparation:

• Most interview guides are written from the perspective of one company. While a company may value efficient code, another may place more emphasis on high-level problem-solving skills. Unless your heart is set on one company, you probably shouldn’t put too much weight on what they value.
• People lie even if they don’t mean to. In writing, companies may say they’re language-agnostic or that it’s worthwhile to explain your thought process even if the answer isn’t quite right. However, it’s not clear if this is actually how they act! We’re not saying that tech companies are nefarious liars; we’re just saying that sometimes implicit biases sneak in.
• A lot of the “folk knowledge” that you hear from friends and acquaintances may not be based on fact at all. A lot of people assume that short interviews spell doom. Similarly, everyone can recall one long interview after which they’ve thought to themselves, “I really hit it off with that interviewer. I’ll definitely get passed onto the next stage.”

At my company, Interviewing.io, we have a platform where people can practice technical interviewing anonymously. As a result, we’re able to collect interview data and analyze it to better understand technical interviews, the signal they carry, what works and what doesn’t, and which aspects of an interview might actually matter for the outcome.

We collected everything that happened during these interviews, including audio transcripts, data and metadata describing the code that the interviewee wrote and tried to run, and detailed feedback from both parties about how they think the interview went and what they thought of each other.

We also asked interviewees some questions that we didn’t share with their interviewers. One of the things we asked was whether an interviewee had previously seen the question they just worked on.

If you’re curious, here’s how the feedback forms for interviewers and interviewees looked like.

Feedback form for interviewers.

Feedback form for interviewees.

The results

It’s worth noting that the conclusions below are based on observational data, which means we can’t make strong causal claims.

Having seen the interview question before

Conclusion