I do think this attitude genuinely limits many startups. Software engineers will always be the first to take up hobbies like fixing old motorcycles or building a house from scratch, yet they view mediocre engineers as things that can't be fixed up in the same way.
A mediocre engineer is actually a really good bargain. Obviously a great engineer who doesn't know it is ideal but I'd toss them in with the mediocre engineers for that reason.
The "can they code test" for us has always been some variation of "fizz buzz."
Anybody should be able to do that. It's not a bad test, especially because of how well documented it is.
I agree there are terrible tests (why on earth would you have someone implement a search or sort? that's like rolling your own crypto, don't do it, don't bother learning how to do it unless it's your passion), but at the very least a simple programming test can indicate that the person can sit down at a fresh debian/windows/OSX install, and knows enough about their given language to be editing, compiling, and running code. This isn't even deploying, just able to have a functional dev environment and be able to create software.
A lot of compsci kids come out of school having worked in specialize VMs or only doing matlab stuff or, well, I don't know what they're getting up to over there but they aren't quite sure what the magic invocations are to get their Java thing running (do I need eclipse? err....) or C (how do I compile...?) or even sometimes Javascript stuff (how come it doesn't work when I double-click the html file?).
A simple fizzbuzz is a great filter for this sort of thing.
While failing interviews is basically my superpower, I think my git repositories, college graders, previous and subsequent employers, coworkers and clients might be somewhat shocked to learn that all that code was written by a guy who "couldn't code" at all.
(The autograders and CI accepting all that non-code must have been exceptionally buggy.)
But still, I failed the test. I told a friend who'd had the slot a couple hours after mine; she thought I was joking. It's FizzBuzz!
I'd seen it before. I knew what it was. The guy had changed the strings and added a factor, but who cares. I could write it in my sleep.
Just not in the form of a puddle of goo, which is pretty much the form I took in this interview. My functional IQ was roughly on par with that of my socks.
I could tell you why it happened, but it doesn't really matter. Now that interviewer had a story of the time a guy with FAANG on his resume (whose name alone had gotten me the interview) was totally incompetent.
These comments are full of those stories, mostly being used to show the test is fine and everybody sucks.
Putting yourself in the shoes of the company interviewing you: If you collapsed into a puddle of goo under pressure, to the point where you were incapable of performing a task "you could do in your sleep," why are you still a good choice to hire?
The point of my story is not that I should have been hired. That isn't my call.
The claim is often made that anybody who can code should easily pass some particular test: P -> Q. I'm offering a story of P & ~Q, which means FizzBuzz isn't really doing what it's claimed to. There's a ton of subtlety I've sidestepped and a lot more I could say, but I don't really have the time.
Because you don't hire firefighters. Senior devs should be able to reason about complex problems and come up with good solutions after a few hours or even days, depending on the problem at hand.
You test their skill of solving trivial problems in 10 minutes under pressure. I can accept if people defend that as the best proxy measure they have for the amount of effort they want to invest into each candidate. But do you seriously believe that this proxy filters the good senior devs from the bad ones?
Because the day to day job is nowhere near as stressful as the artificial constraints of the interview process.
Do people typically come in to work at 11? Why do you schedule interviews at 9 AM?
Do people typically have time to think about problems on their own for a bit and start writing up a solution after they understand it? Why don't interviewees?
B: How do you know?
A: They can't pass our test.
B: How do you know it works?
A: Because if they could code, they'd pass. Sometimes they pass, and even some of those can't code.
B: So nobody can pass your test.
A: That's right.
B: And even if they do, it doesn't mean they're good.
A: Yep.
B: It sounds like you've got neither recall nor precision. Don't you think the test could be wrong?
A: Nope! Everyone just sucks.