People love to complain about whiteboarding but it's extremely effective at assessing knowledge, far more than even most exams. Exams are used in an academic context b/c they're less resource intensive, though.
"To understand if coding interviews—as administered today—can induce stress that significantly hinders performance, we conducted a randomized controlled trial with 48 Computer Science students,comparing them in private and public whiteboard settings.
We found that performance is reduced by more than half, by simply being watched by an interviewer. We also observed that stress and cognitive load were significantly higher in a traditional technical interview when compared with our private interview.
Consequently, interviewers may be filtering out qualified candidates by confounding assessment of problem-solving ability with unnecessary stress."
This has been widely discussed on this site and is a pretty contentious topic.
For my view, consider a very bright problem solver, who bursts into tears every time a person watches or criticizes their code. Do you think that's relevant to performance?
Certainly is a balance, but studies like this usually wilfully ignore all of the other aspects of software work, that aren't sitting in a chair alone writing code.
The stakes are completely different, which is the main problem.
When being interviewed, an entire career is resting on your 45 min - 3 hr performance. And at certain places, that could potentially mean the difference between $50k/year, and $250k/year.
On the other hand, you're not risking your job every time you discuss something with your co-workers.
There will always be absolute edge cases (like the person you described, which by all means sound like an extreme case) - and I think it's fine that some tests manage to exclude these, but the problem is that it's also potentially removing candidates far away from the edge.
White-boarding or "oral exams" are effective in proportion to the skill of the person giving them. Most people have little natural skill in this even (perhaps especially) if they are expert in the domain being examined.
I learned this rather well studying abroad in Ireland. I got a final grade of 92 in a class on protein modeling, which in the UK/Irish grading system is really high.
The professor asked me if I had taken the class before.
I hadn't, but the exam was all short answer and multiple choice. Which was very different from the long-form work most of his students did, but absolutely in an American student's wheelhouse.
I was okay in the class. What I was really, really good at was the test.
