Puzzle-solving interview questions: You get what you ask for!

Interviewing: The process by which an interviewer determines via a face-to-face interaction whether an interviewee possesses the chops necessary to assist the hiring company in its quest for profitability and market domination. (My definition. Feel free to quote me.)

About a decade ago, Microsoft pioneered an approach to engineering interviews based primarily on puzzle solving. The approach became so popular that companies like Google adopted it, books were written on it, and websites dedicated to it. Candidates were even encouraged to consult up to a dozen specific tomes in order to successfully crack the puzzle-solving puzzle that the interviews evolved into. (See the Wikipedia article “Microsoft interview.”)

It bugged me when I first heard about it a decade ago, and it bugs me even more now that other companies have jumped on the bandwagon. I think it’s a pathologically bad practice for reasons I will now reveal.

When you ask a senior in Computer Science to write a function for string comparison in some arbitrary language (for example), you’re testing his or her capacity to either a) regurgitate code they successfully wrote three years earlier, or b) cram puzzle-solving code snippets into their brains in order to impress an interviewer on demand. You might as well ask a candidate to solve a Rubik’s cube blindfolded in less than two minutes for all the value you derive from a question like this.

Seriously, what do you learn from this question?

  1. That the candidate is smart. Congratulations. You just verified their cumulative GPA and spot checked specific courses on their transcript.
  2. That the candidate is a puzzle solver. Congratulations. Next time just break out a Rubik’s cube and be done with it. For the bonus round, break out a blindfold.
  3. That the candidate knows how to prep for your highly arbitrary and ultimately predictable IQ test. Congratulations. You just learned that the candidate is motivated to game your goofy interview process in order to secure a position with you.

My most significant concern is this: Fill your company with a few thousand smart kids who know how to solve puzzles and you’ll populate an engineering staff who can’t think strategically or globally about where the company needs to go, or how products need to evolve. You may find a few who have broader abilities, but your odds of finding them are about the same as discovering new hires that can hit a sweet jump shot or play the cello. You get some by chance, but not because you interviewed for it.

My assertion/prophecy ten years ago was that companies who hired spiffy puzzle solvers via spiffy puzzle solving interviews would ultimately suffer from a vast and pervasive lack of vision and global thinking among its engineers.

Memo to the industry: Your company’s biggest problem isn’t the puzzle solving ability of your engineers. It’s your collective ability to figure out what the market will cough up for the products you never thought about building.

Interview by testing for small thinking? Don’t be shocked when you find yourself working for a fading company devoid of a critical mass of engineers capable of thinking big.

5 thoughts on “Puzzle-solving interview questions: You get what you ask for!

  1. I suppose it depends on how you analyze the interviewee’s response to the puzzle question. The hazardous-to-your-company’s-health method would be as you describe, to just look for the “right” answer. But perhaps some real value can be found out of the puzzle question response by looking for out-of-the-box, creative thinking (both in problem analysis and solution exploration). Even if they get the thing wrong (which is not necessarily a bad thing since most puzzle questions are designed to be devious and tricky), they may prove the ability to look at the problem from a unique perspective and think critically as they problem solve.

  2. Whenever I went to a job fair and talked to the Microsoft recruiter he would always ask what I did for fun. When I would tell him about playing sports or skiing or something non programming related he would immediately say, “well we will look over your resume and let you know.” The people who got the interviews were the ones that said programming. I figured they just wanted code monkeys.

  3. In my opinion, there’s nothing wrong with hiring engineers primarily for their puzzle-solving skills; that’s their primary function. When you expect or allow those same engineers to decide the direction of the company is when you’re headed for disaster. Large companies like Microsoft and Google can get away with this practice if they balance out their spiffy puzzle-solvers with globally-thinking big-picture people. Smaller companies can’t afford to specialize and compartmentalize their workforce that way and should follow different hiring practices.

    Also, I second Michael’s opinion that a good interviewer can learn a lot about the candidate’s thinking process with these sorts of questions.

  4. Pingback: Dr. K’s Software Ruminations » Blog Archive » Puzzle-solving interview questions: How you do it can't save you

Leave a Reply
