I have a similar technique for phone screens. Candidates are told to do the interview where they can receive email. To avoid cheating, we send them a couple of code snippets right as we call them. Then, we talk about the code.
We found that phone screeners have a bias toward those who speak well, so we often brought in terrible programmers for a half day of interviews. What a waste, especially when flying people in. The in-person failures we experienced were of the FizzBuzz variety, not the "I don't exactly under stand concurrency" sort.
So, we started asking people to read some code and comment on it. Really basic Java stuff: Will this code compile? Trace the flow of execution through this method if the BlahException was thrown. String comparisons with ==. The idea was to see if (1) the person was at all clueful, and (2) if they had actually written Java code.
As the linked blogger noted, the disparity in responses is shocking. We found that about half the people we interviewed thought that execution of a Java method ceased once a catch block had been entered. Half! How could they ever have written robust production code? These were often people with years of Java experience on their resumes. I just don't understand how someone could write a bunch of Java code and not understand the basics of how exceptions work.
Some other parts of the code review were more nuanced. Most candidates knew that one should not compare strings using double equals, but they usually couldn't explain why. Nor did they understand why == sometimes seems to work when comparing a referenced String object to a literal.
We were happy to hire smart candidates that hadn't done a lot of Java work, but we wanted to avoid people who had supposedly spent years with a language and didn't even comprehend the basics.
I think this depends on what you are looking for in a candidate. Being able to read code is good, but if you want to hire someone who is really good now and will be 10x better in 6-12 months, you don't want someone who is opinionated and arrogant about what constitutes good code. You'll want someone who recognizes something doesn't look right and then asks why it wasn't done a better way.
This seems too small and specific to matter.
Sure, it is important to manage what is going on inside a loop iterating a million times. But without a good profiling system to know where to work, this could be wasted effort. And good design of a large system will matter more than any single function or loop.
That said, you could do a high level system design and get down into a few details.
On a similar note, in my experience (and this is mainly in C++), reading someone else's code means dealing with hundreds of files and tens of thousands of lines. Getting the system in your head is important, but this is difficult to test in an interview -- because it might take days, not minutes.
My favorite interview question is asking about projects done in spare time. This yields a great deal of information about how inherently interested the person is in hacking.
National Instruments (Austin, TX) used to do that alot and it worked great. They start with some code critique and then slowly, starting with "how would you re-write this from scratch?" they get you to write some, following by popular "optimize for speed/memory" and so on.
I just recently had somewhat similar idea: to reverse tables and ask them to interview me. Not the typical "how do you like it here?" kind, but I want to ask people to ask me technical questions and see what they can come up with.
I've had a lot of interviews and never had to do a code critique. Microsoft's had me write some simple reverse strings type things but never critique code. It would think it would work great. Being able to turn some junk into something elegant and functional is a way better skill (for an employer wanting experienced programmers) than knowing the terms and names of methodologies you learn in school.
We found that phone screeners have a bias toward those who speak well, so we often brought in terrible programmers for a half day of interviews. What a waste, especially when flying people in. The in-person failures we experienced were of the FizzBuzz variety, not the "I don't exactly under stand concurrency" sort.
So, we started asking people to read some code and comment on it. Really basic Java stuff: Will this code compile? Trace the flow of execution through this method if the BlahException was thrown. String comparisons with ==. The idea was to see if (1) the person was at all clueful, and (2) if they had actually written Java code.
As the linked blogger noted, the disparity in responses is shocking. We found that about half the people we interviewed thought that execution of a Java method ceased once a catch block had been entered. Half! How could they ever have written robust production code? These were often people with years of Java experience on their resumes. I just don't understand how someone could write a bunch of Java code and not understand the basics of how exceptions work.
Some other parts of the code review were more nuanced. Most candidates knew that one should not compare strings using double equals, but they usually couldn't explain why. Nor did they understand why == sometimes seems to work when comparing a referenced String object to a literal.
We were happy to hire smart candidates that hadn't done a lot of Java work, but we wanted to avoid people who had supposedly spent years with a language and didn't even comprehend the basics.