We look for a time the candidate wanted something so badly, they were unstoppable in pursuing it. Or a time they overcame an obstacle,” Hamilton says. As you listen to the answers to those questions, pay close attention to both the tasks and the duration described. “Try to get a sense of how long that person can stick it out. How long are they going to beat their head against a stats problem?”
Good qualities for an academic (or inventor type), but doesn't seem to have much correlation with the kind of "grit" needed for most commercial development work.
In which, the vast majority of the time, we benefit the most from "triaging": that is, discerning when not to go down rabbit holes, and when not to "bang our heads" against problems when it may be better to simply... route around them, or leave them be until some sunny day arrives when there's a legitimate business need to properly resolve them, and we have appropriate resources available to do so.
If a candidate actually had any of those attributes, never mind all seven, they wouldn't be interviewing at your company. Companies need to realize 99% of candidates are B players, and 99% of their own employees are too.
Interviews are not standardized tests. They are closer to a date/sale, than they are to a test. Same answer in an interview is likely to be interpreted differently by different people, very much like you're on a date.
The more you think of interviews like a test, the more you'll agonize about it, and the more you'll feel writing and believing such gospel-ic articles like this. True, there are a few basic common things, but beyond that, it's subjective by definition.
Whoever YOU think fits your values and ideas, is an A player. Everyone else is "B", because they are not exactly "compatible" to your thinking.
At http://InterviewKickstart.com, that's what we find. Every company has a different viewpoint of what a great engineer is. After a generic training for interviews, it's about compatibility with the companies.
My current gig asked me a bunch of questions about tools and techniques... that I am unable to use to do my job.
Suffice it to say that there were some hurt feelings when the contract actually started and I saw how they were actually doing their work. If I knew then what I know now I would have continued looking.
Hmm, this is interesting: I usually share your sentiment when I read articles like this. I think companies are overly obsessed with hiring "the best of the best" and should instead focus on finding great employees, which is a different thing. Having said that, I thought this particular article avoided that trap and enumerated attainable attributes of good employees. As a B-player myself, I believe I (and many others) have nearly all of these attributes, never mind none.
I think the "A-Class" employee idea is a myth. Obviously I've come across individuals who are more talented than others but it's very, very difficult (maybe impossible) to put people in a box.
Would you prefer to work with a genius engineer who is an asshole to deal with or someone with a great personality who takes somewhat longer to learn?
I have a sneaking suspicion that hiring is like stockpicking: after an initial filter, monkeys with darts would do just as well.
The problem is: if the interviewers are "constantly learning," (and they are, just ask them), then they are consistently making mistakes that the future, more learn-ed version of themselves wouldn't make. To take an example from the article: interviewers are to ask candidates about some time that they used data to make a decision. What if a candidate responds with a story in which they realized the data were pointing the way to a local optimum that the candidate cleverly avoided? To you, smart HN reader, you are perhaps impressed, and tempted to slap the candidate on the back and rejoice in your shared membership in the club of those who've discovered ambiguity ("Complexity, such a bitch, right?"); but if the interviewer but dimly remembers calculus but is proud of themselves for using a sentence with the word "data" in it, then the candidate's response will come off not as an indication of the ability to "ask the right questions," but a sign of non-cooperativeness, "irrational refusal to let decisions be driven by the data," and perhaps intellectual arrogance.
For interviewers, I have no idea what to tell them. For candidates, I can only say: Lie, and then follow through. If they are looking for "grit" but you had a privileged upbringing, make something up and commit to yourself to be grit-ful[0], rather than saying what's in your head, which is "I haven't had much hardship but I get the importance of resilience." If they ask for your greatest weakness[1], say anything but your greatest weakness; instead talk about the difficulty of coping with other peoples' weaknesses ("I don't deal with my father's alcoholism as well as I ought," or "I get impatient with lazy people.").
[0]This is basically what the sales people in your org do, which is why you get those requests.
[1] This is a dishonest question, because there are wrong answers. The only acceptable answers are weaknesses that the interviewer shares; but by making that the only right answer the interviewer is implying that they have the "best" weakness and therefore have no "real" weaknesses. If they don't introspect honestly, why the hell should you? Or, more charitably, which is a better approximation of reality, that you are both about equal on your paths to enlightenment, or that you are an imbecile? Is truth best served by communicating the former through omission, or the latter through facts?
"My wife thinks I work too much, but I love writing CRUD apps. I stay awake at night thinking about the most efficient way to get those little pieces of data into webforms used by by a couple dozen people. So yeah, working too much and lack of sleep is my greatest weakness."
-->I have a sneaking suspicion that hiring is like stockpicking: after an initial filter, monkeys with darts would do just as well.
I couldn't agree more. I have probably had more interviews / initial conversations in the last ninety days with tech recruiters. I would say about 95%, knew absolutely nothing about what they where doing, they ask me the generic questions... What are you strengths, weakness? What was a time you where in a bad situation... A lot of times I could figure out that they hadn't read to the bottom of my resume.
Then there is the Tech guy questions: What is OOP? What does it mean when you get this error? What is the difference between PHP 5.6 and PHP 5.5, tell me right now!
I did run across some recruiters that actual read to the bottom of my resume. Usually, they where the ones that figured out I was over qualified for most positions.
The best job I ever had was in the Marine Corps and they never asked me any questions, they where happy to have me sign the contract!
Clearly I'm not the target audience (I'm not in a position to hire, in fact, I'm also looking for work), but I'd be more inclined to read to the end if a) It rendered anything before allowing js, and b) if there was a link (or preferably content) at the top, just listing the bullet points.
Looking at the html-source, I see there's a link to a doc-document buried at the end of the rabbit hole -- I humbly suggest whatever hides in there should also be exported as html, and linked prominently from the landing page.
That said, I'm assuming your reference to people not reading your whole resume is from applications where you've submitted a cover letter + traditional CV?
I should also add, that your resume-site has some impressive points, and good visuals -- I just don't think busy, technical people would want to waste time on that, if they could just see a short list of some of the good things you've done. If those details means nothing to them without pictures, then they're probably not technical enough to be able to make good decisions regarding hiring for technical positions. Personally, I don't think being overlooked by companies that have bad management/HR is a bad thing. I guess it depends who you see as your audience?
I made the visual resume, mostly for tech recruiters and non technical people / business owners. (Definitely not CTO's), I also use it to see if the tech recruiters actual view my resume, which a lot of time they don't look it at.
Your right, being overlooked by bad management is a blessing in disguise.
I'm not sure many would agree that the Marine Corps is the best job they have held :P ...although it certainly teaches you a lot that most people don't get grounded well on & that the tech industry needs a lot more of (handling stress, good leadership, etc.).
Oh that's for sure. But based on my experience, teaching is also part of leadership. I don't see a lot of leadership in corporate America. I see a lot self interest, hence the outrageous CEO salaries / golden parachutes.
To respond to your point on being asked about weaknesses: I've always taken the honest route. I've given reasons like "hard to find motivation if I don't find the project interesting" and I've found they've gone done quite well (I think I've only been turned down for < 5% of interviews I've done in the past 10 years).
The caveat though: explain to the interviewer that you're being vulnerable. I always make it clear that I hate those "strengths presented as weaknesses" bullshit and then explained that I'm going to be 100% honest. People seem to connect with it.
TL/DR: Give the candidate an IQ test and a work-sample test. The next best is an "integrity test", which actually measures mostly conscientiousness. You can totally ignore years of experience and education.
Reflecting a bit on this, and on my comment above re: CVs -- I'm inclined to just start applying to jobs with something along the lines of: I'm above average IQ[1], you can send me a work test at <email>. If the test goes ok, lets talk.
Actually, in this light, codeval.com makes (even) more sense.
[1] Sadly, not impressively high. I forget what score I got last time, IIRC around 130 or so. Then again, I also have pretty high "Social IQ" (self-assessed ;-), and the tests don't cover that.
Yep, and I would go further and say that interviewing is something which has NOT been (and may never be) objectively "figured out" even though industrial psychologists have floundered for decades on how to best evaluate candidates.
That information is out there, but it's much easier to focus on changing or improving things that don't involve management practices and organizational behavior, especially focusing on people who aren't part of the business yet.
There is a common flawed view that if you can focus on hiring the best people then you won't have problems with performance. It can also result in high turnover, and hinder performance when the rest of the equation isn't weighted heavily.
Focusing on things other than hiring also comes with politics, because it involves admitting that something isn't perfect with the organization.
> There is a common flawed view that if you can focus on hiring the best people then you won't have problems with performance.
I think, more specifically, there is a common flawed view that hiring the best engineers means you can ignore management and have them "self-organize". Related to this is the false idea that a room full of really smart people don't need dirty things like "process".
Nearly every time I get someone to give me a concrete example of why a false positive is dangerous I find a management flaw that is the root cause of the damage.
This suddenly makes me think of the hiring process -- at least the much-criticized and apparently prevalent form -- as a primary example of premature optimization.
Perhaps a bit obvious; nonetheless, I find it useful to apply those specific words to it.
This just in: "How to do X by doing Y". The only way to get good candidates is to optimize your hiring pipeline to the point that you can hire someone in less than two days and fire them within 2-4 months if it becomes clear things are not working out. Doing anything else is just a waste of time, money, and effort. There is no proxy out there that will give you only the candidates you're looking for.
The "First Round Review" is an embarrassment, they push out content-free fluffy valley conventional wisdom every day or so, an unfortunate number of which make it to the HN front page for some reason.
Here's what all these HR sourced "top performer" filters are missing: top performers are a nurture not nature thing i.e they are a function of the environment.
What's the incentive for a top performer to work for you? We hear a ton about how the best people "take ownership", but a lot less of why they should build your company for you (rather than starting their own).
The most important thing a manager/CEO can do is to align the company's success with the individual. That's how you make top performers out of people you already have.
I agree, if you want good people you need to create an environment in which they can thrive. They need challenging problems, stake in the company, and most importantly get out their way and let them build stuff. Great companies are built by engineers and not the HR dept. or CEO's with an accounting background.
Yep. Nearly every 'article' about how to hire suffers from selection bias.
Here's what I'd like to see: Find some super-talented folks who are critical to the success of whatever they're currently working on. Have them name names of companies that turned them down in the past. Then interview HR from those companies to find out what they did that filtered those people out of their organization. This kind of study would probably be impossible for a number of reasons, legal and otherwise, but I bet it would be a fun and informative read.
The big problem with this idea is that many will assume, rightly or wrongly, that in spite of someone's success at $COMPANY_X that they still would have been bad for $COMPANY_Y. I am personally of the opinion that a good engineer is a good engineer, and a bad engineer is a bad engineer. Organization and culture can amplify or attenuate the "goodness" or "badness", but cannot make a good engineer bad or vice versa. I also think way too much emphasis is put on culture fit, however it gets defined.
The problem is that people do change. I was an outright terrible employee for years and mediocre in engineering skills and am now fine as an employee but despite knowing the general means to not be a bad engineer am probably just middling or below for various reasons not related to capability as much as personal circumstances. The job market changes over time as well.
Special case of the general class of 'business advice' writing. Any book that says 'here's how you build a great company' or 'here's how you close deals' or 'here's how to manage people' will be chock full of examples of how following the advice works, and how failing to follow the advice doesn't. But there's never any reflection on cases where following the advice didn't work.
Specifically, I'm not sure these characteristics are the perfect set for all interviews. I only spent 5 minutes reading the article, after all :)
But, I like that these are all personality traits with no easy answer. I get the feeling that an interviewer trying to craft questions to test "grit" or "impact" will discover good stories about the candidate. They'll probably push the candidate to talk about real challenges, or explain below-the-waterline projects they thought were important. Assuming there's a technical screen that validates that they're woefully uneducated (I'm thinking FizzBuzz, not 3 hours of red-black trees), it feels like an interview designed to capture these attributes would be pretty probing, and if the topics were divided among different interviewers, they'd all get different angles on the candidate.
That seems good. Interviewing for the nominal job description is easy to game. This checklist, if thoughtfully prepared for, seems harder to game.
As a person that has interviewed for and gotten jobs at multiple Fortune 500s (whose tech departments varied from abysmal to impressive effective) and as a person that has been a part of the interviewing/hiring process several times, I appreciate the attempted formalization of interview metrics and see more than a little value in the metrics described in the article.
To me, though, there is one "metric" that trumps all the rest, and that is personality fit with existing employees. If you have two brilliant software engineers that just absolutely cannot stand each other, nothing will get done. The best interview experience I've had, personally, has been to be placed into a technical interview - "start programming application X that does Y" - with two existing employees looking over my shoulders asking me why I'm doing what I'm doing and why not something else.
The technical justification for my chosen solution to the technical interview question didn't matter much once it was clear I covered a base level of technical ability, but it was incredibly valuable that I and the employees looking over my shoulder didn't want to get into a fistfight after an hour - it meant that we could amicably disagree, support our reasonings for our arguments, and then not have sticks up our asses when the other person's proposed solution ended up being the right one, and then gracefully defer and say "thank you for the conversation and debate." That meld is immensely important.
That said, I do recognize that it becomes increasingly difficult to gauge this sort of interaction with increasingly large teams. My experience interviewing and being interviewed is limited to cases where it's been a team of 3 - 8 people. For larger teams, perhaps such a formalized approach makes sense. I do not have any experience that informs me as to whether it is or isn't.
But then I have to ask whether there should be a team of more than about 8 people without a dedicated manager. I don't have anything more than a gut answer to that question, but my gut says no.
I will just say again like I always do, there is no solution to the problem. You hire, you try, and if no good, you fire.
I have some thoughts regarding possibly giving the interviewer some more leeway (like for engineers, don't start off with solve this first) but haven't thought enough about that.
Interestingly, this rubric outlined in article applies the advice Daniel Kahneman gives in his section on interviews in Thinking Fast and Slow. As he acknowledges, it isn't perfect but in studies it generally shows an improvement over the sort of gut-level hiring most people end up doing.
So yeah - if you think the technical interview process is broken, take a look at this and realize that at least in our business there is hope. Hiring someone for non-technical jobs? This is the best you can do.
Given a person who knows nothing about
medicine, then might sell them on
leech bleeding.
Given a person who knows nothing about
technical work, might sell them on
the OP.
Bet if take five interviewers
and have each of them
read the OP and apply it to interviewing
100 people and compare the results, will
find (1) validity, the interview results
will on average be a poor predictor
of anything important and (2) reliability,
the interview results will scatter
with huge standard deviation. That is,
it's throwing darts where the scatter
is huge and the center of the scatter
is far off the center of the target.
Uh, didn't see where there was some
good double blind tests of the OP
for such reliability and validity.
Uh, the OP was big on analysis -- good,
now have them tell us about their
7 devices and reliability and validity,
that is social science 101.
Let me just guess, from reality
and common sense, what is going
on with the OP:
The whole OP is essentially
just plausibility
psycho-babble that HR interviewers would
like to entertain. It's aimed
at HR.
The OP is yet more evidence that under
no circumstances should anyone in
HR have a substantive interview with
an important candidate for anything
important.
Instead, HR should smile, be nice,
offer water, coffee, tea, soda,
smile, be nice, give directions to
the rest rooms,
help with the interview schedule and
the organization chart, hand over the
benefits packet, smile, be nice,
help with the travel and lodging
arrangements, help with expense
reimbursements, smile, be nice, etc.
E.g., the OP wants to ignore
school work but wants to emphasize
ability with, what was it, analysis?
Okay, I can understand why an HR humanities
major would get confused here, but
a good solution is to look for majors
in math, physics, electronic engineering.
The OP wants to give as a test
of ability with analysis something
with Excel. Upchuck: I avoid
Excel because it is nearly always
way beneath me -- I use much
more powerful tools. Excel will
do Lagrangian relaxation, the fast
Fourier transform, Wiener filtering,
least squares, multi-variate spline
interpolation, L. Breiman's random
forests or even CART, stochastic
optimal control, solve some
differential equations? I don't think
so. I don't waste time with
Excel. That's like testing
a cardiac surgeon on cleaning
bedpans.
"Top Performers"? Right, maybe
for toilet cleaning. Even then
they'd be at risk of mixing the
Muriatic stuff with the
Drano stuff -- don't do that!
HR tries again, fumbles with the ball,
drops the ball, trips over the ball,
falls on the ball,
loses the ball, is face down in the
mud.
"For more technical roles, you can even build a timed Excel test with some practice problems or logic tests.", what? is the Excel page for the interviewer or the candidate?
In any case, you can use whatever questions you like, they are very trainable and people good at BS and making up anything quick will do well at those.
I'm an MBA type with a lot of compsci background, and I get this bullshit all the time. These advice were poignant and to the point of actually suggesting to me how to think and how to answer those incredibly silly questions. That's what it's like over at the Corporate Overlords in Scandinavia at least.
This would be very politically incorrect, but I'd wager the best way to find top performers is to ask "What do your parents do for work, and how much money do they make?" There is a very high correlation between the success of parents and their children.
Certainly above average parents are likely to produce above average children. But most the top performing parents' children are likely to regress back towards the mean. Getting the highest umpteenth percentile in anything is going to be crapshoot if you only consider one factor.
Attention is your scarcest resource at any company. You hire people to add to this resource. Bad hires do not add, but subtract from this pool, by forcing you to pay attention to them rather than on running your business.
I'm an engineer; one always needs data to support something.
> Attention is your scarcest resource at any company. You hire people to add to this resource.
Not sure what you mean by "attention", but I think it is the same thing I would call "throughput": the rate at which the company can turn customer requirements into engineering products (and thus money). If so, I would agree, as pretty much all resources of the company combine to affect this.
> Bad hires do not add, but subtract from this pool, by forcing you to pay attention to them rather than on running your business.
Still on board with you. I would agree that someone whose net throughput contribution is negative is a bad hire.
The problem is, this has nothing to do with the statement I was originally reacting to: "Avoiding false positives is much more important than eliminating all the false negatives." That statement needs hard data about:
- The expected number of false positives
- The expected magnitude of the throughput loss from each false positive
- The expected number of false negatives
- The expected magnitude of the lost potential throughput due to the false negatives
>The problem is, this has nothing to do with the statement I was originally reacting to: "Avoiding false positives is much more important than eliminating all the false negatives."
You don't quantify the impact of false positives the same way you do the false negatives. A false negative simply means you take more time to look for a candidate. A false positive's impact is so huge that you can't come up with numbers large enough for false negatives to outweigh it.
To make a broader point, not everything can be tracked and measured. Things that can be tracked and measured often can't be compared to each other usefully.
> You don't quantify the impact of false positives the same way you do the false negatives. A false negative simply means you take more time to look for a candidate. A false positive's impact is so huge that you can't come up with numbers large enough for false negatives to outweigh it.
I vehemently disagree with this.
A false negative means either work isn't getting done or current people are overworked. This is costing your business. If it isn't, why are you hiring in the first place?
As for the cost of false positives being "huge", how? All you have stated is that a bad hire brings down the net productivity of a team. How much is very relevant and something you can get a rough feel for. If I hire someone who has a slight net negative impact on team productivity, and they show no signs of improving, I fire them and move on. Is it a worse outcome than not hiring them or anyone else for that period of time? Yes. Is it a huge impact? Not at all.
You can come up with a hypothetical worst-case, company-destroying false positive, but that would be comparing apples to oranges based on your position concerning false negatives. The worst-case false negative situation is also company-destroying.
Didn't listen to the podcast, but if the thesis is borrowed from Judith Rich Harris[0], then the GP's claim is still valid. To wit, parents' parenting practices have limited impact on their children (vs. peers and genetics) but their success and income do (via genetics and the influence of wealth over peer environment).
We look for a time the candidate wanted something so badly, they were unstoppable in pursuing it. Or a time they overcame an obstacle,” Hamilton says. As you listen to the answers to those questions, pay close attention to both the tasks and the duration described. “Try to get a sense of how long that person can stick it out. How long are they going to beat their head against a stats problem?”
Good qualities for an academic (or inventor type), but doesn't seem to have much correlation with the kind of "grit" needed for most commercial development work.
In which, the vast majority of the time, we benefit the most from "triaging": that is, discerning when not to go down rabbit holes, and when not to "bang our heads" against problems when it may be better to simply... route around them, or leave them be until some sunny day arrives when there's a legitimate business need to properly resolve them, and we have appropriate resources available to do so.