I'll try to condense it as much as possible, and add the caveat that our process is imperfect:
1. Build outreach programs that are open-enrollment and broadly targeted (for instance, the crypto challenges aren't targeted at security industry events).
2. Explicitly overcommunicate the whole hiring process to candidates up front and sell the job, before attempting to screen. Actively try to correct for the adversarial nature of the process.
3. Drastically reduce the impact of phone screen interviews.
4. Administer standardized work-sample tests to all candidates regardless of previous experience; those tests have quantifiable outcomes (which we record).
5. Administer standardized on-site interviews with quantifiable outcomes (right now, they're "exercises", but their form might change over the long term).
The only point in our process where interviewers have a lot of flexibility in terms of what they do is the phone screens. We use the phone screens to generate predictions for how candidates will do on the rest of the interview, but we don't screen with them. (We would if the phone screen revealed a candidate to be a crazy person or a jerk, but I don't think that has ever happened).
So far, the things that have been most noticeably successful for us:
* Standardized, quantifiable results that allow us to generate a model of what a successful candidate actually does, rather than a gut feeling based on how well they answer questions, or how "confident" they are, or how "well spoken", or how impressive their resume.
* Outreach to the industry as a whole, not just to the small sectors of the industry we believed a-priori would generate the best candidates. Some of our best people have come essentially "out of nowhere", and recognizing that fact I'm especially interested in optimizing for that.
It is thrilling --- best word I have for it --- to find someone who's had no relevant job experience, hire them because they do unexpectedly well on a series of quantifiable metrics, and then see 6-9 months later that they are, say, showing us how to use BKZ lattice reduction and fourier transforms to exploit elliptic curve nonce biases.
This sounds like a vast improvement over most normal hiring processes. A question though - how do you avoid legal liability from (4) and (5) in the event that they cause a disparate impact (see Griggs vs Duke Power, etc)?
I don't think about it at all, and for good reason: nothing that our process does could possibly be more risky than the crapshoot process other companies use. To wit: I know for a fact that my process isn't going to be dumb enough to ask a woman whether she's going to be able to manage her child care responsibilities and still be able to answer emergency work calls.
The danger comes from the fact that courts generally give deference to ad-hoc processes, whereas systematic processes are inherently suspect. That's why interviews are fine but IQ tests are legally suspect.
As another example in this general line of thought, compare Grutter v. Bollinger and Gratz v Bollinger. Point-based racial discrimination is illegal, but "holistic" "individualized" processes where race is just a "plus factor" are legal. (Only discrimination in favor of non-Asian minorities, of course.)
I agree that this legal doctrine is insane.
Completely tangentially, since you mentioned relevant questions which are illegal to ask, I'd love to see an economic analysis of this topic. Specifically, the classical Akerlof lemon market paper assumes a complete inability to measure quality. But in the hiring market there are only a few dimensions for which one is unable to measure quality. That complicates the analysis significantly, so I'd love to gain some intuition on the topic.
The point of Griggs vs Duke Power is that a work sample test would probably need to be "reasonably related" to the job at issue, if it has a disparate impact on minorities (meaning minorities pass at a lower rate). I have no idea how one proves in court that a test is "reasonably related", because I'm not a lawyer :-)
Yeah, I'm familiar with it; my point is that the ad-hoc procedures companies use in place of work-sample testing are much more perilous. I've been a witness to more than one legal action that resulted from them.
1. Build outreach programs that are open-enrollment and broadly targeted (for instance, the crypto challenges aren't targeted at security industry events).
2. Explicitly overcommunicate the whole hiring process to candidates up front and sell the job, before attempting to screen. Actively try to correct for the adversarial nature of the process.
3. Drastically reduce the impact of phone screen interviews.
4. Administer standardized work-sample tests to all candidates regardless of previous experience; those tests have quantifiable outcomes (which we record).
5. Administer standardized on-site interviews with quantifiable outcomes (right now, they're "exercises", but their form might change over the long term).
The only point in our process where interviewers have a lot of flexibility in terms of what they do is the phone screens. We use the phone screens to generate predictions for how candidates will do on the rest of the interview, but we don't screen with them. (We would if the phone screen revealed a candidate to be a crazy person or a jerk, but I don't think that has ever happened).
So far, the things that have been most noticeably successful for us:
* Standardized, quantifiable results that allow us to generate a model of what a successful candidate actually does, rather than a gut feeling based on how well they answer questions, or how "confident" they are, or how "well spoken", or how impressive their resume.
* Outreach to the industry as a whole, not just to the small sectors of the industry we believed a-priori would generate the best candidates. Some of our best people have come essentially "out of nowhere", and recognizing that fact I'm especially interested in optimizing for that.
It is thrilling --- best word I have for it --- to find someone who's had no relevant job experience, hire them because they do unexpectedly well on a series of quantifiable metrics, and then see 6-9 months later that they are, say, showing us how to use BKZ lattice reduction and fourier transforms to exploit elliptic curve nonce biases.