Well, we don't do the puzzle questions, and I agree that those are not useful to determine the level of software engineering experience a candidate has.
However...
99 times out of 100 you could catch that engineer by talking to him about technical aspects of past projects he worked on.
That does not work for us.
We must put each and every candidate in front of a keyboard, and bang out at least a little code.
We had one candidate, recently graduated with an MS in CS. Nice personality, good talker in general. The candidate seemed to understand CS and could explain how to solve problems.
However, even with repeated prompting, I could not get this person to even type in a loop to start to solve the problem presented.
We've interviewed multiple candidates with a BS in CS or similar who also spectacularly failed at doing even basic coding tasks.
Some people are really great talkers. They really do sound like they know what they are doing. But that does not mean they can actually do anything.
As someone else said, I'm not talking about someone fresh out of school with a few months internship experience.
>That does not work for us.
How do you know, did you try it? I said 99/100, sure there are some rare people who may slip through, but there are people who slip through a coding challenge as well.
How do you know that your coding test is better at detecting secretly terrible engineers than talking through past projects (for experienced engineers)?
>Some people are really great talkers. They really do sound like they know what they are doing. But that does not mean they can actually do anything.
That's sounds like a problem with the questions you're asking. Some people can throw out buzzwords and talk at a high level, but if you really dig down into the implementation details of project you can weed those people out.
How is this relevant to the parent? You're talking about someone with a CS degree, which means they have no experience. How can you ask them about the technical aspects of what they did on a project. "What parts of the system did you write? And what problems did you hit. What solution or bug were you proud of solving?".
Nearly all the the candidates that we've interviewed have at least a little job experience, such as an internship. And even for the ones that don't, they all have worked on team-based projects as part of their coursework.
So we end up asking them all those sorts of questions.
Working on team-based projects as part of coursework does not mean they've ever written a line of code in their life.
Going through a CS degree without learning to code is easy. Depending on your CS course you may hardly have touched any code beyond extremely entry level courses where you're being hand-held the entire way, if you pick the "right" subject.
CS is not software engineering.
Similar with internships.
For both those situations, I can understand that you want to verify ability to actually write code.
You're just agreeing with ansible - no one here is arguing that a CS degree means you can code. Rather ansible is relating his experiences with people graduating from those courses and not being able to code.
What you seem to be implying, is that this is obviously true for graduates, but couldn't possibly be true for people with real experience. (But your implication has no evidence to support it)
Exactly why do you believe that it is completely plausible for a recent CS graduate to be able to explain the technical details of their final year project when in reality they contributed zero code to that project, yet it is entirely implausible for someone with 3 years experience to be able to be able to explain the technical details of their most recent development projects when in reality they contributed zero code to those projects?
> yet it is entirely implausible for someone with 3 years experience to be able to be able to explain the technical details of their most recent development projects when in reality they contributed zero code to those projects?
Here is problem. It's not entirely implausible, just very unlikely.
The type of interview I'm talking about isn't going to catch every single bad candidate just the vast majority. But whiteboard interviews don't catch every candidate either and they have an extreme amount of false negatives.
To your point about students, they have much less work experience to talk about. The point of the interview is to look at their experience and talk about it to verity they aren't lying. In most cases the student doesn't have enough experience to tell me anything.
That being said, if a student can talk me through technical implementation details of a final project, then I think that's a pretty good indication that the student can code.
I would say the false positive rate for that test would be very close to the false positive rate for a white board interview with drastically lower false negatives.
If you're Google and you have a limitless number of people who want to work for you, you can get away with taking a vastly higher false negative rate for a slightly lower false positive rate. However, if you're almost every other company out there, you might want to reconsider.
However...
99 times out of 100 you could catch that engineer by talking to him about technical aspects of past projects he worked on.
That does not work for us.
We must put each and every candidate in front of a keyboard, and bang out at least a little code.
We had one candidate, recently graduated with an MS in CS. Nice personality, good talker in general. The candidate seemed to understand CS and could explain how to solve problems.
However, even with repeated prompting, I could not get this person to even type in a loop to start to solve the problem presented.
We've interviewed multiple candidates with a BS in CS or similar who also spectacularly failed at doing even basic coding tasks.
Some people are really great talkers. They really do sound like they know what they are doing. But that does not mean they can actually do anything.