> It's pretty hard to get someone solving actual problems in a single day. Do you have some examples of how this might be done?
Solve a small problem - of which millions can be found in issue trackers.
Here is an example I just read on Github:
Somebody reports that reading a JSON file results in a value that clearly is an integer being misidentified as "Not an integer". The source code is a parser written in C and the code is quite readable.
Give the candidate an hour to try to figure out what is happening (obviously, the language and context must be a match for the job).
Important - I think: It would be good if the problem is unsolved and the interviewer him-/herself does not know the answer. Also: Don't send them into a room alone, watch them. I know that adds stress, but learn how to lower it, make it clear you too don't know the answer. If the candidate really can't cope with someone watching, well, okay, let them try alone, I just think if the atmosphere is right this is valuable. There is no right way to approach solving such a problem, but without and not meant for judgment, I think this is just interesting to see how different people approach problems. Of course, if the interviewer has string opinions about how it should be done that is a bad method and I myself would not like to be interviewee in that case.
> How long it took somebody else in your team to solve the same problem
I don't think that is useful. Variance is too great, even for a constant single person.
I think that anything that attempts to cast this issue as some objectifiable measurement problem is on the wrong track. Measurements work - on a statistical scale. Even then its hit or miss depending on how good the chosen values to be measured are. Just like public health issues. As soon as you try those approaches that work just fine on a large scale on individual problems you are toast. The results of those individual decisions can be measured using statistical methods, but don't try them on individuals.
> How long it took somebody else in your team to solve the same problem
I don't think that is a good measure. I know many developers who want to solve everything fast because fast is productive in their eyes. The problem is they end up having a bug or three and spend some back and forth time with QA. So, now they didn't actually solve the problem fast and they wasted QAs time. I prefer to get a more holistic understanding of the code before fixing the issue and not get any bugs back from QA. In retro, when the QA tells the team how many bugs there were, I am proud when I'm not a part of that count.
I disagree. If we're considering the debug/issue fix approach (which I quite like, because it displays code literacy and clarity of explanation in a context like that one would find in a typical workday), if I use the same task/tasks for each candidate, I'm implicitly (and tremendously) biasing myself toward a universe shaped by the tasks I chose. What I'm really looking for is a more universally distinguishably trait of lucidity and perspicuity (coupled with sufficient technical skill, which, again, bias and variance from your tests' structure is always an issue in accurately scoring), in any case.
I think this would be great. It would give me, the interviewee, a chance to feel out how it is to work with the person interviewing me. Will they crap on every solution I propose? Or will we have some back and forth and find a solution that suits us both?
Also, you don't have the added stress of writing all the code from scratch, having to whiteboard recall boilerplate that a good IDE handles for you.
Solve a small problem - of which millions can be found in issue trackers.
Here is an example I just read on Github:
Somebody reports that reading a JSON file results in a value that clearly is an integer being misidentified as "Not an integer". The source code is a parser written in C and the code is quite readable.
Give the candidate an hour to try to figure out what is happening (obviously, the language and context must be a match for the job).
Important - I think: It would be good if the problem is unsolved and the interviewer him-/herself does not know the answer. Also: Don't send them into a room alone, watch them. I know that adds stress, but learn how to lower it, make it clear you too don't know the answer. If the candidate really can't cope with someone watching, well, okay, let them try alone, I just think if the atmosphere is right this is valuable. There is no right way to approach solving such a problem, but without and not meant for judgment, I think this is just interesting to see how different people approach problems. Of course, if the interviewer has string opinions about how it should be done that is a bad method and I myself would not like to be interviewee in that case.