I think you can probably predict it: I would generate a work-sample test for it. For a network programmer, I might have them implement a 3WH coded directly to pcap_write() (which requires you to populate the Ethernet frame header). Like the best work-sample challenges, doing a raw 3WH is kind of fun if you haven't done it before.
My friends and I used to spend bar nights drinking over torturous interview questions (yes, I have always been this nerdy). For instance: we had a gruesome sequence of questions on how to implement the fastest possible traceroute that you could only clear if you knew about a trick using the IP timestamp option.
Later, I got (what I thought was) smarter about interviewing, and moved to more surgical questions. I'd ask candidates to debug a C program that segfaulted in malloc, or ask them to describe the utility functions they carried with them from project to project.
After taking over recruiting for a company that really needed to hire at a specific clip in order to balance sales and delivery, I'm embarrassed that I thought I was interviewing effectively with stuff like this.
You can't learn about someone's capabilities by putting them on the spot with trivia questions.
Thanks for the detailed response. Frankly, I was awful at recruiting: grilling candidates who claimed to know C++ on the intricacies of templates or whether they knew what Koenig lookup was. Caught a few fibbers but I don't understand in retrospect what the point was.
> You can't learn about someone's capabilities by putting them on the spot with trivia questions.
But that's not the point of these questions. These questions are a 5-10min phone pre-screen before getting to the actual interviews. They test if the candidate has experience in a given field, not if they can search for information or what are the precise bounds of their capabilities.
It's trivia, but it's trivia that is a reasonably high confidence proxy for experience in e.g. network programming/design (resp. other fields). If someone claims they have networking experience and don't know SYN/SYN-ACK/ACK it's in my opinion a large red flag.
These screens pass people who can answer trivia but who can't effectively code, and they reject people who have a gift for solving engineering problems with code but who can't answer trivia questions when put on the spot.
The ostensible reason they get deployed (I say "ostensible" because we all know that in reality the on-site interview consists of the same stupid kinds of questions) is to keep the employer from wasting time conducting more sophisticated interviews for candidates who have no hope of passing. But that's dumb for at least two big reasons:
* The filters obviously reject candidates who would do well on more serious challenges --- worse, they do it insidiously, because you can't tell that they're rejecting good people, only that you're seeing fewer bozos, which makes them look like they work. In reality, a new norm has arisen where the most qualified candidates get to skip these processes entirely, because we all know they're a crap-shoot and don't want to lose good people.
* Properly administered work sample challenges actually take less employer time than these stupid trivia screens do. That means there's literally no purpose to the trivia screens whatsoever; they do nothing but harm.
In my last comment, I just wrote off the top of my head a sketch for a work sample test that addresses the same concern as the dumb TCP/IP trivia quiz from the original post. It took me I think something like 30 seconds to come up with it. Think about how you'd score that (remember: the bar here is "must be more predictive than that dumb trivia question"). I'm thinking something along the lines of "run the code and see if it opens a new TCP connection".
Assuming your team has enough sophistication to build work sample challenges like this, try to justify the trivia interviews. I think you can't.
> These screens pass people who can answer trivia but who can't effectively code, and they reject people who have a gift for solving engineering problems with code but who can't answer trivia questions when put on the spot.
SRE's are hired to fix outages and other problems asap, knowing trivia is very important then since at that point you might be losing a million dollars per second.
My friends and I used to spend bar nights drinking over torturous interview questions (yes, I have always been this nerdy). For instance: we had a gruesome sequence of questions on how to implement the fastest possible traceroute that you could only clear if you knew about a trick using the IP timestamp option.
Later, I got (what I thought was) smarter about interviewing, and moved to more surgical questions. I'd ask candidates to debug a C program that segfaulted in malloc, or ask them to describe the utility functions they carried with them from project to project.
After taking over recruiting for a company that really needed to hire at a specific clip in order to balance sales and delivery, I'm embarrassed that I thought I was interviewing effectively with stuff like this.
You can't learn about someone's capabilities by putting them on the spot with trivia questions.