> One year, I hired 50 engineers. I couldn’t have done that in a high quality way without a system that let me interview about 500 candidates
How is this even physically possible? I don't mind interviewing usually but 2 or more a day, every single day, sounds like pure hell. Not to mention that after not touching any code for a whole year I'd be questioning your ability to be doing technical hires in the first place.
I'm very skeptical also that this ultra-standardised approach is getting anything like what I would call "top 10%" programmers. Top 10% of people who happen to do well in this particular interview setup, morelike. Teams I like to work for have a healthy mix of abilities, passions and experience. This sounds like a recipe for a room full of dull clones, none of whom have any knowledge outside a strictly specified little box. What next, psychometric screening?
I do agree with one point though - 3 people in the room is probably the right number.
I did one about 10 years ago, and let me share a funny story. The head of HR was sitting next to me and right after the psychometric test she gave me a blank sheet of paper and told me to draw a human being. I promptly asked (without even considering my question, I was ready to ask the person's gender right next, looking for requirements): with clothes on or naked? She laughed so hard that I thought I had failed and laughed at it too. I got the job, though.
That would have been an absolute perfect opportunity to to draw a middle finger and walk out. That sounds like an absolutely idotic application process.
I might have even laughed at the HR person and asked the question "are you fucking serious?"
Not really, a company that's employing those tactics is probably one not worth working for. An interview is a two way street, they are interviewing me and I am interviewing them. This would be a huge red flag.
The best companies I've worked for have been the ones who put a lot of effort into making sure they understand who they're hiring. That usually means a lot of weird questions about topics that aren't related to the technical aspects of the job.
My last round of interviews I got the tech side without any issues, but I was subjected to three rounds of the soft skills side because they were concerned my quiet affect might not fit into the team. I did what I was asked and was rewarded with working on a fantastic team that someone had put significant effort into ensuring the personalities meshed well.
Burning a bridge because you thought someone cared too much about the wrong thing is a great way to never get another callback from everyone involved in the process. Recruiters and HR people move around a lot, and have long memories.
I'd rather someone cared too much, than not enough.
I'd happily burn a bridge with someone in HR that expected me to jump through hoops and draw pictures. Those people need to hear the truth, they are bad at their jobs.
He is Director Of Engineering. A large part of his job is supposed to be hiring if they are growing.
> This sounds like a recipe for a room full of dull clones
Can you explain why? I've found that a standardized interview with asking the same questions to each candidate helps me better see the differences between candidates.
Over the years I've tried on a lot of different interviewing processes, and what the article described is pretty close to what I've ended up with as producing the best results.
> I've found that a standardized interview with asking the same questions to each candidate helps me better see the differences between candidates
Oh, I start with a standard list of topics, and of course technical questions need to be consistent across interviews to be any use. But beyond that I like to let the interview flow naturally - if the candidate seems particular passionate about one thing or another, follow up on that, and see where it goes. It's a conversation, not some form to be filled out. If you're going to do that, may as well just email them the actual form.
Maybe the author isn't as strict as I understood but I really do believe that it's in that conversation that you get to know someone's personality, what they're passionate about, how they might fit culturally and how their experience and talents might complement the team. You don't get that from a survey.
I agree with you about letting the interview flow naturally. That's how the last half of my interviews go. Often times looking at and discussing some of the candidate's code, depending on what the role is.
I don't always get it perfectly right, but when I do my interviews feel good to both sides. It just feels like we spent an hour discussing a topic we both like and we both learned something new.
> Maybe the author isn't as strict as I understood
I didn't get the idea that he is when I read it. What I did get is that he wants lots of data points confirming the same thing. A list of standardized questions or topics is just one of those data points, right?
> Not to mention that after not touching any code for a whole year I'd be questioning your ability to be doing technical hires in the first place.
I you read code from 20 years ago, you will be surprised that there is good and bad code. Most of the code techniques were developed decades ago.
If you stop coding you are going to be more slow when you have to start to do it again. But you don't forget the principles.
You will find managers that user to be developers that are bad coders. That does not mean that they have forgotten how to code, in my experience they were not good to start with and that is why they changed profession.
> Not to mention that after not touching any code for a whole year I'd be questioning your ability to be doing technical hires in the first place.
Let's not get carried away. Programming isn't easy, but are you really saying that after 1 year not coding, you can't do technical work? 1 year is maternity/paternity leave in some countries, do you really think that people coming back from maternity/paternity leave can't code any more?
> The assessments are never really correct, so a vital step is reviewing the report with the candidate
…
> How the candidate reacts to the report; which things they think are true or false and why; which things they engage with and which they skip over; those are all revealing.
I cofounded a company last year, and our focus is on facilitating 360 reviews / annual reviews for companies. The format of our assessments are fairly “psychometric”, but not because we use it to profile someone, but because it helps facilitate written and verbal feedback from your peers and manager. The excerpts you highlighted match our findings, and is a big reason why we run workshops as a part of our process (so we can explain how the information is used).
This is the same way I approached technical interviews when I was doing a lot of hiring. I didn’t usually care a lot about how someone answered a question, but rather how they reasoned about that answer.
Although not in the same "domain", I have had good results using viainstitute.org's character strengths tests to better understand my professional and academic clients before creating/editing their resumes, linkedin profiles and interview coaching. Its been around awhile and focuses on what people should do more of. YMMV.
Thanks for the link. That does look like a much more thoughtful use of the technique than I had anticipated - talking about the result afterwards is a good idea. I had been thinking their more common use was indeed to blindly discard candidates based on some nonsense quiz, totally ignoring the creativity and passion the candidate could have otherwise brought to the team.
I've actually done a couple of these tests, although for the life of me I can't remember in what context, and thought them ridiculously simplistic and obviously gameable. Now I'm curious to try some of the more subtle ones the author alludes to.
I don't understand. Is the term psychometric screening here being used to refer to personality / aptitude tests? I think I'm confused because this is not how the term is used in psychology, but maybe businesses use it this way?
I think "standardising the process" doesn't mean "multiple choice test". On the contrary i had the impression the author is putting much care in getting everyone to know the human qualities of a candidate.
But you have to define process if you want to tie in your team and consider many candidates.
This article feels like it dances around the point of technical questions and the assessment that comes with it, which is what most of us worry about when interviewing.
Soft skills are often treated as if they're there or they aren't. There's not as clear a distinction between 'junior' and 'senior' soft skills. I don't think I've ever gotten a salary bump in the offer for being a cool guy to work with (according to the interviewers). Usually it means I get the job or I don't. This isn't really that interesting to me.
The group stuff is spot on though.
- One on one is often dry. You have to have an affinity right off the bat. This is the equivalent to showing up at a bar and having a fun conversation with a stranger.
- More than two or three interviewers, and it can quickly turn into a company circle-jerk (I've had 15 interviewers once from 4 separate teams in one interview -- that fucking sucked).
One on one is often dry. You have to have an affinity right off the bat.
If I follow you on soft skills, are you implying that a company that subscribes to this logic is selecting for attributes that their current employees lack, even though they're participating in interviews that will be used to judge same?
Well yeah, because the interviewer knows the other people! The candidate, however.
My point was that if they're actually hiring for soft skills, why does it not hold sway in one-on-ones? It would seem to symbolize a weak bullpen, so to speak.
1 in 10 interviews is a hire? How can his methods be described as sucsessfull.
Speaking of my own experience I'd say it takes 6 months to find out if a programmer is good or really good (bad is easy to spot) and being able to tell that in 4h is just self-deception.
I wonder how hireing managers compares to coin tosses after some 15min telephone offsite initial screening and just filtering of resumés.
1 in 10 fitting the company culture and being a good technical fit sounds pretty good to me. What number would sound good to you and why?
As someone who has done a lot of interviews, I think his methods sound great. Through trial and error, I've ended up with an approach that is pretty similar to what's described in the article, although there are some new ideas from the article I'd like to try out.
> I'd say it takes 6 months to find out if a programmer is good or really good
I'm much more concerned about cultural fit. After that I'm happy with both good and really good programmers. A good programmer can be trained up to be really good. Most companies need a mix a junior, intermediate, and senior programmers.
Regardless of rejection rate, that is just a brutal pace to keep up for engineers who are also supposed to be writing code. A lot of interrupts for prep + interview + scoring + team debrief. And for introverts like myself, an hour of talking to a stranger is itself draining and often kills my productivity for a little while afterwards (especially if it went poorly). The occasional week of 3 interviews is fine, but for that to be the baseline blows my mind. How do they get anything done?
> Every engineer must be equally skillful with code and colleagues.
I think I’d rephrase this to say “every engineer needs a baseline level of skill with both code and colleagues,” rather than “must be equally skillful.” I don’t find the sentiment controversial in the least.
I’d love to see a company actually quantify this, instead of just assuming it is large and letting the question paralyze them into being excessively conservative and picky.
I’ve been in a situation where we needed somebody right now to backfill someone who left, and the company agonized over who to hire, taking over a month and blowing through dozens of candidates, ensuring that the product schedule slipped and losing us an untold amount of revenue.
And to pile on the anecdotes, I've been in the exact opposite situation where we needed somebody right now to handle a new requirement, and we hired someone "pretty good" after already taking longer than the rest of management was happy with to hire.
They ended up being pretty weak, having trouble shipping, eating large amounts of onboarding and management attention, as well as other engineers patching up their work to make it serviceable. They were just good enough to make it seem worth the slog too, so they sucked a bunch of extra labor out of everyone else for the better part of a year before they left.
In retrospect, I made the right decision with what I knew in the moment — but with 20/20 hindsight I should have waited.
Suppose as indicated in the article that you want to do 500 interviews in a year and have about 10% success rate. That’s 40 candidates in a month, and assuming 6 person-interviews (3 interviews of 2 interviewers each) are needed per candidate, that’s 250 a month which could be split between 18 “seasoned interviewers” at about 14 interviews a month each.
The number of interviews by “seasoned interviewers” doesn’t say so much about the rejection rate as how much the company is attempting to grow and what proportion of engineers count as “seasoned interviewers.” If almost all people are seasoned then that’s a growth rate of 250%. If only 40% of engineers count as “seasoned interviewers” then that’s a growth rate of 100%.
It's written somewhere 50 engineers from 500 interviews. Plus the one sorted out from initial screening.
Being skillfull on soft skills usually is a euphemism for handeling toxic and antisocial coworkers.
I have personally never thought that the would-take-beer-with rating is even remotely as important as technical skills. And being a corny coworker or a toxic one don't really correlate.
Soft skills aren’t really about likability (though many will like you if you have them).
They are about communicating clearly and concisely, listening carefully, respecting the people you work with, tolerating differences, disagreeing constructively, handing decisions that don’t go your way gracefully, sometimes going with the tide and sometimes fighting it, and knowing when to do which.
In interviewing, like in many other subjective measurements an important trade off is precision (percentage of hires that worked out) vs recall (good hires that got rejected).
Experienced interviewers, or an innovative process can optimize both values, but I do not believe one can reach perfection. In fact, I don’t think most companies are even anywhere close to optimums in either of those values.
Yet what’s rarely considered during interviewing is which of the two values the organization should optimize for. An early stage startup, hiring a senior engineer may choose to optimize for precision. A mature organization that needs to hire quickly to start a new initiative may choose to minimize recall.
Another way to put it - hiring is risky. The amount of risk you are willing to take should depend on circumstances and your ability to absorb failures.
Having learned this myself over a few decades through both trial and error and also researching best practice, I have to say this guy gets a ton of stuff right. I also read a few new things I now want to try.
Just to reiterate one of the more important points from the article in my own words: resumes (CVs) have almost zero correlation to the quality of candidates. Pre-screening calls can really only screen out people who are completely unqualified due to not actually having the skills listed on their resume. So I will pre-screen with some very basic tech questions. Or more likely, I'll just use a recruiter to pre-screen for me.
I'm going to finish up with a copy paste from a past comment I made to a different article with some tips that have helped me over the years:
1. Unless you've been trained to interview people, you are probably really bad at it. Think about how bad someone is at coding who never did it before and has no training. Interviewing people is no different: it is a skill that takes time and effort to learn. Get trained up and practice. As much training as you have time for. Educate yourself in your free time on a regular basis with articles about best practice. Adding more words won't make this sink in more, but I will end by saying I can't stress this point enough. You are bad at interviewing. You need to fix that before anything else if you want to hire good devs.
2. How are you measuring success as an interviewer? This is a really difficult thing to measure because how can you find out that your interview process is rejecting candidates who would make great employees? I'm not going to try to answer this here, but it's an excellent question to think about. One thing that might help is the next point.
3. Whatever your first impression is of the candidate, spend a good part of the interview trying to prove yourself wrong. Candidate seems like a perfect fit for the team and a great coder? Instead of relaxing because you're sure you already found the right person, see if you can figure out why they might be a bad fit or if their coding skills aren't as good as first impressions. Same for someone who seems weak at coding. Maybe they are just nervous. Can you make them relaxed enough so their awesome skills can come through? Maybe in both cases your first impressions will be proven correct. But if not you might have just found a diamond in the rough that other companies are skipping over.
I just went through my bookmarks and quite a few of the links are dead... I've been doing this for decades so that's not a surprise. And not really a big deal. Most of the progress I've seen has been made in the last five or so years. Technical interviews might be horrible now, but they used to be way worse!
A LOT of the articles I've read over the years I have not bookmarked because my idea of a good technical interview is the opposite of what those articles recommended.
I normally read everything interview related I come across in HN or Reddit programming subreddits but I'll also do a search every six months or so and see if anything interesting comes up.
Although I have cherry-picked advice from reading many articles over the years, more importantly I've tried really hard to come up with an interview process I myself would enjoy as a candidate. That factor is very important to me because I think the interviewing process sucks at many companies. It's painful for everyone involved and ineffective. I would tolerate painful and effective. But I think it can be somewhat painless and effective.
I remember getting at least a few valuable ideas from these:
Also ask someone in HR what soft skills you need to improve related to interviewing. And then take classes based on that.
Most importantly, and as the article suggested, interview candidates with another person who has more experience interviewing. With a variety of people. You'll learn the most by doing interviews alongside people who are better at it than you. Practice really is what makes perfect. Especially if someone can coach you.
One final thought, recruiters aren't going to give away the store and teach you how to be a great interviewer, but you can pick their brain with questions. If your company will pay for a recruiter then always go back to them with doubts about your process after every interview. They want you to hire their candidates so they will help out a little with advice. I've heard some great advice from recruiters themselves.
I'd be embarrassed to admit to these metrics. This guy has wasted not only his own time by bringing 5x the candidates needed onsite, but his team's, his employers', and the candidates themselves'. A quality hiring process would know with at least a 50% confidence level that the person is worth hiring before bringing them in for the onsite. A phone call or one-hour Skype, or even 3 supplemental ones, is a lot cheaper than the onsite.
It says a lot about the industry that not only is it common practice to have a "concave" funnel where you don't winnow people out early enough and end up making your engineers spend half their workday conducting pointless interviews instead of doing actual work, but also that management writ large is too clueless to be embarrassed.
If anything your engineers did, anything, had a 20% success rate, would you consider them "only the best"? Even if their appraisal of the top 10% of candidates was accurate and reproducible (it's not even close), I'd still expect them to achieve their results with fewer than 3, and preferably fewer than 2 onsites per hire.
Sure, the guy can point to his numbers as some sort of merit badge, but if I told you I washed my hands 400 times a day, you wouldn't consider me some kind of expert on handwashing, would you? You'd rightly think I'm some kind of crazy person.
> A quality hiring process would know with at least a 50% confidence level that the person is worth hiring before bringing them in for the onsite.
After having done a decent amount of hiring interviews and managing in my life, I don't think it's possible to have 50% confidence before you meet someone. The way you gain that much confidence is by 1- talking and listening to them, and 2- comparing them to other candidates.
> I'd still expect them to achieve their results with fewer than 3, and preferably fewer than 2 onsites per hire.
This seems hyper-focused on the minor efficiencies of interviewing, at the expense of employment efficiency; it's feels like an engineering approach that might not have all the requirements at hand.
The employee is (probably) going to be paid a six figure salary, and will have a large impact on the team. Great candidates can improve the efficiency of the people around them for years and years. Bad candidates can slow entire teams down. I've watched some excellent and prolific and opinionated engineers do major productivity damage -- millions of dollars worth, and I've watched some young CS graduates outlearn and outperform some veterans. Resumes & github accounts simply don't have enough information to know whether a candidate can help your company.
In my experience, it has been worth the time to let the hiring process go slow enough to be able to compare people's personalities, and not just their on-paper engineering credentials.
I think far too many engineers here on HN and out in the workforce dramatically underestimate how important attitude and people skills are in the hiring process, and dramatically over-estimate how important specific math & programming skills are.
As another person with some experience with hiring, I agree with you fully.
To put one of your points in my own words, your engineers are going to be spending the majority of their waking hours with this new person five days a week. Take the time to make sure it's a good fit.
As yet another person with some experience with hiring, let's not get theory and practice confused.
Taking the time to make sure it's a good fit is a worthy goal, but the industry is operating under some kind of delusion that there's a Ballmer Peak (http://xkcd.com/323/) to interviewing people somewhere between the 4th and 5th hour of the onsite interview.
Returns start diminishing very quickly after the first five minutes. This is why multiple people earlier in the process are necessary.
> Returns start diminishing very quickly after the first five minutes.
That's the opposite of a scientific approach. One of the best things I ever learned from an experienced interviewer is the spend the rest of the interview trying to disprove your first impression. Anything else is an exercise in confirmation bias. I'm not saying that I've never ended an interview early when it's obvious there is a bad fit. My time is important and so is other people's time.
A scientific approach is to pose a theory (this candidate would make a good hire) and perform a measurement (whoa, this guy's good) and then perhaps perform the measurement again, without bias, to see if the measurement is consistent and reproducible.
What you're suggesting is to spend the next several hours performing additional measurements in the hopes of finding one contrary to your first measurement. That is precisely the opposite of science. It's also how casinos stay in business.
And that's exactly what I said. Don't trust your initial impression. Measure again. Get as many data points as you can.
> What you're suggesting is to spend the next several hours
I didn't suggest that and I'm not sure where you got that idea from. My interviews are usually less than an hour and then I hand the candidate over to the next interviewer. If they are a strong no from me, I'll even talk to the next interviewer to explain why and see if they even want to continue with the candidate. They usually don't want to continue with a "strong no" candidate, but sometimes the next interviewer might not share my concerns and they do want to continue with the candidate. Could be the person is a good fit for the other interviewer's team.
On the last point we agree, but you're missing the main point here.
Videoconferencing and phone calls can take care of not all but a large measure of the personality issues that can crop up. Many a brogrammer ends up wasting an onsite by skipping this step when the candidate could have saved them both the trouble, had they just taken the 5 minutes to have a Hangout.
The efficiencies of interviewing are anything but minor, and I find it odd how the wrong six-figure hire is bandied about but the existing 2-six-figure manager spending half his workweek interviewing nonhires instead of working is overlooked.
> Videoconferencing and phone calls can take care of not all but a large measure of the personality issues that can crop up.
You seem to be making large assumptions and arguing straw man issues. What makes you think that both the author and I don’t use videoconferencing or phone calls? I didn’t say anything about it, and the article mentioned screening calls right at the top; literally the 4th sentence.
> I find it odd how the wrong six-figure hire is bandied about but the existing 2-six-figure manager
Please explain. What’s “wrong” about my suggestion that an engineer will “probably” make something near six figures? Average salary for all engineering in the US is north of $85k. The average engineering manager salary is around ~$130k.
> spending half his workweek interviewing nonhires instead of working
You seem to be assuming that the manager is always interviewing, as opposed to, say, filling a few positions, working without hiring for a few months, filling a few more positions.
The guy in the article is a recruiter, so recruiting is his full time job. If a normal engineering manager has other work, then recruiting isn’t their full time job, and they might not be hiring at all times. I haven’t ever worked in a department that was constantly hiring, neither in startups nor in huge global companies. Have you?
Suggesting that the cost of interviewing a dozen people is larger than the cost of employing one not-so-great engineer seems, from my perspective, to be off the mark.
> What makes you think that both the author and I don’t use videoconferencing or phone calls?
Maybe it would help if you understood it as "use [enough] videoconferencing or phone calls".
Answering that question:
For the author, the aforementioned 5X ratio. For you, your apparent insistence that a) you have to talk and listen to them, but b) this somehow has to take place in person to get to 50% confidence. Maybe that's true for you, but I doubt it's universally so.
> What’s “wrong” about my suggestion
I think you parsed my sentence incorrectly. "wrong" modified "six-figure hire", as in, the wrong guy who costs six-figures.
> I haven’t ever worked in a department that was constantly hiring, neither in startups nor in huge global companies. Have you?
Oh yeah. It's more like everyone had two full-time jobs, and one of them was 30 hours a week of interviewing.
> Suggesting that the cost of interviewing a dozen people is larger than the cost of employing one not-so-great engineer seems, from my perspective, to be off the mark.
Let's try from my perspective. Say you hire a not-so-great engineer, who's maybe worth $100k, for $90k. A great engineer would be worth $1M, for $90k. Both are wins.
O that it were only a dozen people. Say it was 5 hours a week of wasted interviews, for 50 weeks, and the interviewer earns $100k. This puts a loss of >$10k onto your bottom line, neglecting the "real work" the interviewer could be doing, and the other employees also doing the same thing.
> Maybe it would help if you understood it as “use [enough] videoconferencing or phone calls.”
Again, what makes you think the author isn’t using enough? Why are you so certain you have an important point here that anyone else who’s hiring doesn’t already know? I think you’re making an incorrect assumption that the author is being unreasonable.
> the aforementioned 5x ratio
That was the ratio of candidates to jobs. It doesn’t say anything about how much screening was involved. Interviewing 5 candidates for a job seems completely reasonable to me, and not out of the norm of hiring for any company I’ve ever worked or hired for. I might screen 10 or 15 before interviewing 5 on-site.
> I think you parse my sense incorrectly.
I did, thanks for the fix.
> Say you hire a no-so-great engineer, who’s maybe worth $100k....
I already mentioned I’ve seen people lose companies millions. Engineers who are great on paper, and great in interviews. The bottom end here is not a small but positive win, it’s an enormous loss.
> A great engineer would be worth $1M
Where does this number come from?
> Say it was 5 hours a week of wasted interviews, for 50 weeks, and the interviewer earns $100k. This puts a loss of >$10k onto your bottom line.
This thinking still seems like penny-wise and pound-foolish to me. I don’t consider any interviews wasted. I’m learning in all of them, and I’m comparing the candidates. All samples are valuable. Also, you’re talking about the cost of 125-250 interviews over the course of a year, if I assume it takes 1-2 hours to interview and give feedback. If we use the author’s 5x, that means you hired 25-50 people. You’re comparing a measly $10k to the cost and value of more than 25 people. Spending $10k to hire 25 people is nothing. Companies pay more than $10k for a single hire, all the time.
> O that it were only a dozen people.
A dozen people per job is what I was talking about, just to be extremely conservative... more than the 5x the author mentioned. And with a dozen people per job, I’m saying the cost of interviewing is far less than the cost (or value) of employing them, so it doesn’t hurt to interview that many.
I would think of hires as compound interest, where interviews are linear cost. Comparing them already isn’t apples to apples.
In 2017, Apple's revenues were $229B. Apple had 123,000 employees. This means the average employee at Apple pulled in $1.86M, not in salary, but in value to the company.
Check out Paul Graham's wit and wisdom with regard to only seeing the positives of interviewing en masse.
> The bottom end here is not a small but positive win, it’s an enormous loss.
You used the term "not-so-great", not "bad". You're not going to spot all the bad ones. And you can fire bad hires, limiting your downside.
Consider two possibilities:
•Hiring managers genuinely do select the top talent, which means that now the only people they have left to interview aren't qualified to hold the jobs, and thus they're wasting their time.
•Hiring managers don't know everything, try to make the best decisions, but often hire poor performers, and this means that their role in hiring is not as some kind of highly skilled "nerd whisperer" who can divine good candidates from bad candidates, but merely a utilitarian cog in the corporate machine, that tends to overestimate both its skill and its importance. If this were medicine, it would be institutional quackery.
I don't know he did mention there are phone-screens ahead of time.
Also mentioned that the reason they are bringing in more is to figure out who they are before simply disqualifying the candidate because they don't meet the specs. If you simply don't agree with that premise, suggest that, rather than you would be embarrassed to admit those metrics. That's more of an insult than describing how the method he describes specifically and the reasons behind them are wrong.
Also those 400 interviews happened in 7 years, which is one per week. I am currently in a hiring phase and we do 1 or 2 per week.
The process is usually one person doing one 20min behavioral phone screen, and maybe one technical person doing one 60min technical phone screen, give or take. Maybe a homework. Then maybe 3-6 one-on-one interviews for a 5 hour onsite. Fiddle with the numbers if you need, but this covers 90% of companies interviewing today. That's way too few phone screens and way too many onsites if you don't get rid of enough candidates at the phone screen stage.
Simply disqualifying the candidate because they don't meet the specs happens anyway- the question is, are you a highly skilled hiring manager by coming to this realization quickly, or only after you have the candidates come in and you waste an afternoon introducing them to everybody?
I personally feel this industry could use more insults going the other way. Until hiring managers realize they have been trying to source talent with their middle finger out the whole time, they're not going to realize they need to change.
Not in my experience. I used to try to screen resumes and to pre-screen on calls. And it mostly doesn't work. Some people ace the pre-screen and are assholes in person. Some people totally fail the pre-screen because it's the first ever call they had in weeks and they get super nervous.
I will use a 15m pre-screen to verify some basic tech skills but I also then offer candidates to follow up in an email if they think they could have done better.
> If anything your engineers did, anything, had a 20% success rate,
They don't have a 20% success rate. The success rate is 100% if they hire the number of developers they need and to the standard they want.
Saying it's a 20% success rate is like saying interviewing 100 people to determine the best among them is a 1% success rate. But it's a 100% success rate if you did actually figure out who the best programmer is among the 100.
So you pre-screen on calls, but do you get any of your co-workers to do it? There's no hard and fast rule that you have to spend 30 minutes to an hour per whack, either. You get most of your signal in the first 5 minutes (first impressions and all that).
Offering a "grievance process" where nervous candidates can plead their case for a mulligan telegraphs a (probably deserved) lack of confidence in your own company's hiring practices. The only thing worse than a first date that went bad, where they want to submit to you in writing how they could do better if given a second chance, is where they want you to submit that to them in writing.
Your last paragraph totally makes my point. The purpose of an interviewing process for any serious business is not to find the best candidate, but to fill the position with a qualified hire who works well with others.
The 98th percentile score should not be a failing score. You're not picking Miss America, and most of the other 49 candidates are pretty enough, especially when pretty is so subjective.
> You get most of your signal in the first 5 minutes (first impressions and all that).
One of the best things I ever learned from an experienced interviewer is to spend the rest of the interview trying to disprove your first impression. Otherwise, you're just making yourself a victim of confirmation bias.
> Offering a "grievance process" where nervous candidates can plead their case for a mulligan telegraphs a (probably deserved) lack of confidence in your own company's hiring practices.
I have a great deal of confidence in my hiring practices. I've rarely had a candidate reject an offer, so I'd guess the candidates also like it. I've even had candidates tell me that they think the interviewing process I've designed is really good and they learned how to be a better interviewer.
It's not a really a grievance process. It's a simple "Here is why we don't think you're qualified (e.g you don't actually know any JS in reality) but if for some reason you think we got that wrong, please let us know." Once in a blue moon someone will come back with a legit response.
> The purpose of an interviewing process for any serious business is not to find the best candidate,
No true Scotsman? I sometimes have extremely specific hiring goals. I may for example want to hire the best JavaScript person available in town because the company needs to be a thought leader in that area. Every company is different. Don't assume your hiring needs are my hiring needs. For sure when my hiring needs are to find any qualified candidate, that goes pretty quickly.
I follow up with rejected candidates from time to time with a phone call. Often I'll follow up with a phone call and talk about the areas I think they could improve in and recommendations about how they could do that. They seem to really appreciate it.
I'll close the call asking what they think about our interview process. They are mostly very happy with it and disappointed they didn't make the cut.
I've unfortunately never had much constructive criticism about my interview process from a rejected candidate. The only criticism has been the rare few times someone thinks they are qualified and kind of lashes out because they didn't make the cut. "Well I didn't want to work for your company anyway" type of thing. Not much valuable info to go on.
The valuable criticisms I have had are from colleagues over the years who have been in interviews with me participating themselves. Some of what I've improved over the years has been related to those criticisms.
How about yourself? Do you ever follow up with rejected candidates about your own interview process? What techniques do you use to measure your hiring effectiveness and what steps are you taking to improve your process?
I take a different approach; I've zoomed out from the problem, and looking at it from a macro level, I've concluded that the system is what's corrupt, not the candidates, recruiters, or hiring managers, and all of the individuals are victims toiling under a broken system.
It's a business opportunity, and it's one I'm going after once my current project gets to a stopping point.
Facebook's guidelines were 25% confidence they could pass an onsite and they would be given one.
I'd personally rather give the candidate AND the company a chance at hiring a potential candidate than missing out on one. The longterm impact on your company is huge.
This fails to consider the fact that with 25% confidence, 3 out of 4 candidates have no chance, and you're just wasting time, theirs and yours.
There are two kinds of hiring managers- those who think the system works and they're skillfully picking the best talent, though at the last minute, without formal education on the subject, without metrics, without reproducibility, without accountability, and with minimal prep time; and those who recognize the system is deeply flawed and go along to get along.
That is only wasted time if there was next best opportunity. People are looking for work so going for on-site interview is not waste of time.
What would you do, go to the beach, hang out in a bar? So your time would be spent better. Maybe you would go to other interview where you would be hired but you cannot know this in advance.
The same with company they don't know if other candidates would be better until they actually do on-site.
If you can predict future then yes. But most don't.
> That is only wasted time if there was next best opportunity. People are looking for work so going for on-site interview is not waste of time.
What's the unemployment rate for top-end talent in Silicon Valley? 2%? 1%? This foolish strategy of candidates-time-is-free is designed to turn off people who already have jobs and want an upgrade. Not only are you _not_ hiring the best, you're limiting your applicant pool to the worst.
<sarcasm>Why, sure, I'll blow 7 hours including 2 hours' roundtrip commute to audition for a 20% chance of a $10k raise.</sarcasm>
this! At Google, saw this and signed up for 100% phone screens, where I could have much more impact on improving the process, esp for experienced engineers with 7-10+ yrs experience. Conveniently, most people hated doing phone screens, so HR was appreciative as well.
Given standardized questions (with depth - impossible to memorize), you can quickly calibrate to the organizations expectations by following candidates that you approved for interviews.
1:5 and 1:10 do seem atrocious, but looking back on my tech career, I probably applied to about 100 companies for every 10 interviews I got, for every 1 offer, so it sounds realistically atrocious.
He says he interviewed X candidates to hire X/5 engineers. That means they probably selected more than X/5, but that’s the number that accepted offers. Maybe the problem is their offers aren’t competitive.
I totally agree that seems like an excessive amount of interviewing for engineers on the team.
Let me take you further in this direction of thought- many offers aren't accepted because the candidate, after having been through a dysfunctional interview gauntlet, realizes a different opportunity is better (sometimes after waiting weeks to get the start date).
If interviewing has been skillfully conducted, and the candidate gets an offer in 24 hours, there's substantially less likelihood they'll turn it down, because they won't have had as much time to interview elsewhere, or wonder what organizational deficit is causing the delay.
That is certainly true. Since you had already jumped to implicate the interview process as the reason it took so many candidates to make a hire, I was simply offering an alternative. In fact, it could be that both the interview process and the offers are flawed; there can be any number of reasons why a candidate doesn’t convert to an employee, and you’re solely focused on one.
Yep, absolutely horrendous and totally unreadable on iOS safari. First a modal pops up with the “no thanks” beyond the fold (and weird scrolling behavior). Once I finally figure out how to close it, I’m at the bottom of the page. If I scroll up, I see the first few sentences and then an email form. No thanks.
Funny that the site is from a VC firm. Maybe their business judgement ain't so hot? Or their funds aren't doing well so now they have to resort to begging for emails?
On iOS the “Unobstruct” app ($0.99 - no connection to me) removes the popover nicely. It’s also great for removing dickbars that block a horizontal slice of the page.
>So if you come on site with us, you're going to have four sessions. That’s our ‘right’ number, because we don’t want to ask too much time from candidates and be grueling, but we also need to have enough data points. The four sessions include three evaluations sessions with the team, each an hour long. Then there’s a session with the hiring manager, usually me, so I can answer any questions about how the team works
Did I read this correctly? Four hours? Is this over one day or several?
Most people I know won't even touch a job listing with more than two interviews.
When you say most people, who are you referencing? Day long loops are the norm for my social circle. As in, get in at 9am, leave at 1-2pm. That includes lunch for an informal round.
I meant experienced programmers in Moscow, Russia. For the most part it's either one Skype call (<= 30m) and one personal interview (1h - 1h30) or two personal interviews. There are exceptions though. I've heard Yandex requiring at least four interviews, but my colleagues all tell me that they wouldn't agree to proceed on such terms.
Apparently, looking for a job is a much more tedious process in the US.
I once interviewed at Apple many years back. Personal interviews with 19 people over 2 days! Well a few of them were in pairs (I presumed they were training their engineers to interview also.) I didn't make it through however not for technical reasons but cultural fit (maybe I didn't act fanatical enough about Apple products?)
Interviews here are normally 30 mins on phone or Skype and around 90 mins face to face. There no way in hell I'm doing hours of interviews for one company. I also know very few developers who would put up that sort of process.
What? These aren’t on different days, these are on the same day. It is normal even for very good companies (big 4, top quant firms) to have around this many interviews
I would love four hours and the chance to talk to that many tech people on the team. I'm going to be spending the large majority of my waking hours with these people five days a week. I want to get to know the team as well as possible before making a decision like that. A good interview process isn't just about seeing if I'm the right person for them. It's also about me deciding if they are the right people for me.
How is this even physically possible? I don't mind interviewing usually but 2 or more a day, every single day, sounds like pure hell. Not to mention that after not touching any code for a whole year I'd be questioning your ability to be doing technical hires in the first place.
I'm very skeptical also that this ultra-standardised approach is getting anything like what I would call "top 10%" programmers. Top 10% of people who happen to do well in this particular interview setup, morelike. Teams I like to work for have a healthy mix of abilities, passions and experience. This sounds like a recipe for a room full of dull clones, none of whom have any knowledge outside a strictly specified little box. What next, psychometric screening?
I do agree with one point though - 3 people in the room is probably the right number.