When I hired a plumber for my heating firm, I put her through a hazing interview, tested her knowledge of Maxwell relations and thermodynamic equations.
Oh wait, no I didn't. As she has been a plumber for years, so we recognised her skill and experience and cut to the chase.
When I hired a programmer for our doomed to die VC life support funded fart of a project, we put her through numerous rounds of comp sci bingo and algorithm hell. Why? Not sure. Everyone else does. I mean, hiring in IT is shit, why should we break from the industry norm? Why should we want to care about the candidates? Right? I mean if Google can do it...
I frequently hear this basic point: "Software interviews test algorithms and data structures that are so clearly not relevant to the work being done and soooo theoretical I don't understand why interviewing isn't better???"
I used to believe this myself, mostly because I didn't have a strong CS background and was still successful at carrying out software projects at a relatively high level.
The truth is, as I learned more CS I learned that these things are actually pretty crucial. Not necessarily for executing one single project, but for building a large system that scales reasonably? Absolutely. I spent years reinventing the wheel of basic data structures and algos simply because I did not know them or how to use them.
Bottom line: I would not want to work with the me of 5 years ago because I wrote shitty code while proudly proclaiming that CS was theoretical bullshit that didn't matter in web development, and I think many people who make these types of claims really just don't understand how applicable a good CS foundation is.
I'd agree that ideally you want some kind of work sample as well. Either an OS contribution, github or even small project written specifically for the interview. You can't ask someone to write enough code on an interview that you really see what happens when they have to structure a real project (ie: thousands of lines) or work within a real system (ie: millions of lines). This is where CS foundations in interviews have their relevancy.
I think you are both taking extreme sides of this argument when there is a very reasonable middle ground
Should a programmer you hire be able to reason about CS problems? Yes. Do they need to be able to write out the best possible solution to a problem on a whiteboard within 15 minutes? No.
The white-board sessions only really serve to intimidate people who aren't good public speakers, think better with some time and peace and quiet, or simply haven't bothered to memorize code they never have to write from scratch.
Legitimately bad candidates could just as efficiently be eliminated by having a casual discussion about a CS concept that relates to the job description.
I totally agree with this. In one of my other replies I mentioned the concept of having multiple paths for a candidate to be successful through, and I agree that "whiteboard coding" should not be the only way to prove you are smart and competent.
It's not that they're irrelevant, it's that the problems can't be done in a disruptive environment with $100k on the line, by asking questions that must be memorized by chance first.
Well the idea is that they aren't memorized by chance first, they are known pretty well because ideally you have a track record of applying them to real life problems (even if they are just school projects or side projects). I can describe times I've used queues to solve various problems, for instance. The other thing is, as others have pointed out, seeing how someone breaks down and approaches and unfamiliar problem is very valuable.
It sucks that we have to do it under the pressure of a 45min-1hour conversation, and I totally get that the stakes are high and so nerves are a problem. In general I think the best way to handle this stuff is to give candidates multiple paths to succeed.
I think the nerves problem is slightly overstated though, because I do think many companies do give candidates multiple paths to succeed. For instance, if you have trouble performing well in an interview setting it's pretty crucial that you have a wide array of work on github proving you can code and that will give you an opportunity to have some complicated but familiar code that you can talk about in the interview. If someone comes in with this but is shy or gets frustrated or something, as an interviewer I'm really going to try and pull the magic I see in their github out of them in that interview. Keep in mind that people hiring engineers really want the candidate to succeed in most cases. It's so difficult to find good talent that while I'm never going to lower my standards to hurt the company, I'm definitely on the candidate's team in an interview, trying to get them to shine as best I can.
Very few people are working as comp scientists, but rather software engineers, making BFS=Queues a random trivia question that must be prepared for in advance.
BFS queues are not particularly difficult to describe, and deciding whether or not to do breadth or depth first is something I think about all the time. Frequently, for instance, in data access patterns, that decision can have a very big effect on performance. I can't agree that it's a random trivia question.
Edit: Ok, let me add some nuance here. I think we both probably agree that implementing a shortest path algo perfectly in an interview setting is beyond reasonable or useful. But I would expect an engineer to be able to have a discussion about it that shows me they understand the basic concepts and given google and a bit of time could relearn that. But without the complexity of shortest path I would not expect basic BFS to be outside of the scope of an interview. I guess that's just where I draw my lines.
Hmm, their difficulty is irrelevant. Your experience (like everyone's) is anecdotal and not representative of the immense fields of CS and SE that no one person can know all of, but could certainly look up.
If you are looking for a computer scientist, ask for one in the job req. No shame in that, it would save everyone time.
I added a sneaky edit (sorry!) that may address some of this, so I think we probably largely agree. I'm responding to a relatively specific claim, frequently heard by bootcampers and inexperienced engineers in web development, that foundational CS isn't useful, which my experience is somewhat representative of, though you are correct it's still anecdotal.
I didn't finish the problem in my interview, and got hired. I asked the PM later why, and he said "we wanted to see you try to solve a problem we know you couldn't, so we could see your problem-solving process and how you handle stress."
The author of the article comes off as defensive and blame-shifting. I'm not surprised he didn't get hired. Since when is it ok to say "why?" when someone asked you to perform something in an interview? Do you really think the multi million dollar news agency will need you to program a maze at some point, so they're testing your ability to? Just how dumb do these engineers think recruiters are?
Steve Yegge thinks people disregarding CS questions in interviews is the Dunning-Kruger effect in action. I think he's right, but I have no way of knowing - due to Dunning-Kruger...
I don't object to an employer wanting to see a background in data structures and algorithms, sql, binary, and a few branches of math. This is why I agree that hiring is broken but also understand that it isn't easy to fix.
Some of it stems from something very wonderful about our profession - we are flexible about how people can acquire this background. In law, you must do a 3 year JD[1], and you must typically pay way over 100k for it. Alternate paths toward learning this material are pretty much illegal (as in, we'll put you in jail if you try to practice law if you haven't done the 3-year degree, regardless of your master of the material).
For instance, I was a math major, and I only took a bit of formal CS. However, when I did self-study to try to plug some of these gaps (partially for interviews, partially for my own education), I was surprised with how much I actually had covered from a different angle. Many of my math (and later, in grad school, Industrial Engineering) professors ran "labs", some for a bit of extra credit, others as an optional set of assignments. I took graph theory through a math department and stuck around in the lab and so forth to implement some of the algorithms, to learn about how some types of proofs actually contain an algorithm. BFS, DFS, minimum spanning set. That all involved building trees, lists, using pointers. In numerical analysis, I did a lot of matrix algebra manipulations. And so on. If CS were run like law, it would be illegal for someone like me to work as a programmer, because my degree isn't an ABET accredited CS degree.
The downside to all of this, though, is that there is no recognized credential. I don't object to being taken through my paces, I object to how random, capricious, and redundant the process has become. I read a blog article about a programmer who took the bar and studied for 100 hours and passed. Think on that for a second and compare it to the amount of time people spend preparing for and taking what are essentially white board exams in technical interviews. The bar, an exam that is considered one of the most brutal rites of passage for a learned profession? In some ways, we go through that every time we do a new round of interviews when we change jobs!
Actuaries have to show understanding of relatively advanced math, but senior actuaries don't (to my knowledge) get grilled on integration by parts or some specialized set of partial differential equations when they interview. Why? Because there is a proper exam for their field (and unlike law, actuaries are free to obtain this background through multiple educational paths, they often major in math, but hardly always).
Another problem is the capriciousness of the tech exam/interviews. I believe that a "bill of rights" so to speak slowly evolved between examinee and governing board over time in true professions. The nursing and medical boards, the actuarial exams, the bar - these fields have a tough exam (or series of exams), but they are consistent, there is a clear study path, there is a commitment to grade them fairly, there is a clear study path, if you fail, you get feedback or at least a score (the bar doesn't write you back and say they "decided not to pursue your candidacy further at this time"), there is often a second (or additional) change at the exam, and, most importantly, when you pass you get a lasting, public credential respected by your peers in the field.
People often describe these exams as the most brutal, anxiety ridden, stressful events of their professional lives. This is why, I believe, they slowly evolved that "bill or rights" that protects the examinee.
Unfortunately, in tech, I believe we experience these exams over and over, but without any of those benefits.
I am a-ok with requiring people to show competence, but the way we go about it is, I think, badly broken. I believe that this process accounts for a great deal of attrition in the field, as well as people deciding not to enter the field in the first place.
[1] yes, a bit of handwaving, it's not quite that simple in some states.
Interesting post, a lot I agree with, but I think if you aren't working on bridges or things that can kill people when they go wrong credentialing would probably be a negative.
A plumber, sure. What about the guys who design your water heater? Should they just google the thermodynamics as they go as well?
My point being, it depends on the role, and the expectations you have for the future of a candidate in your company. Are you hiring a guy to bugfix or implement a couple of features on your web app? Or a guy to design and implement from scratch a app that should have some hope of scaling in the future? Or a guy that can tackle a tricky performance problem you're having?
I'm not claiming the current status quo is alright, but hiring the right people is hard. Knowing CS basics and generalities of the field doesn't hurt either...
Sure, it makes sense to ask a compiler engineer the various ways in which to reverse a binary tree. That's going to be a problem that you're probably going to encounter on the job.
The fact is that most developers spend 90+% of their time futzing with NPM, dealing with Apple's review process, or renewing SSL certificates. Low-level algorithm work is something very few engineers need to know.
But still, if the other 10% of the time is designing the system, it can wreck the project if you do a bad job at it. And no amount of futzing with NPM is going to save you, no matter how good you are at it.
It also depends on if you want the guy to be more than a front-end bug-fixing guy in the long run.
It comes from the point that the candidate is a lair (when we talk about hiring experienced pros). If someone in your company who is in charge of tech hiring is unable to tell if the candidate is laying about her past experience and projects - you should fire him in the first place (or at least keep him away from the hiring).
So if you want to hire a "guy" to "design your water heater", ask him about the water heater he designed in the previous company, or a water cooler he designed for the neighbor community...
>A plumber, sure. What about the guys who design your water heater? Should they just google the thermodynamics as they go as well?
Honestly, regularly referring to the reference material as you need it rather than trying to remember or derive every single pressure equation off the top of your head sounds like a good idea when designing a potentially explosive boiler.
>Shifting to CS, how do you even know a specific algorithm/data structure might help with your problem if you don't have a rough idea of its specifics?
In general, these things are well implemented in either the base language or a library, and thus both easily searchable and well covered in practical examples. It's unusual for 9-to-5 non-R&D developers to be solving a problem which hasn't been solved before at the algorithm level, the job usually revolves around connecting pre-made components together and specializing them. As such, you really only need a loose knowledge of the fact that things like binary trees exist. Being able to hand-write one isn't useful and may be a liability depending on the temperament of the developer.
Your hand-rolled implementation is unlikely to perform better by any metric than the pre-made one, and it'll probably be less platform independent and more buggy simply because it's newer and looked at by fewer people.
In 95% of all software development today, rolling your own version of any off-the-shelf algorithm is a mugs game and actively bad practice because it places the foundations for technical debt. You really don't want to be hiring people who roll their own just because they can.
Now, if you're going to be working in the R&D or experimental areas of software development, kind of where the Oculus driver crew are at the moment (as an example), then sure, you definitely need to know the inner details of these things.
The problem is that we're interviewing everyone as if they're going to be writing high-performance drivers, when most of them are never going to work at that level and wouldn't want to.
> We recognized her skill and experience and cut to the chase
So, to clarify, you're advocating that programmers be hired based on resume, references, and a conversation about prior projects? I'm not sure I agree with this. It seems you'd be selecting for ability to present yourself as a good programmer over actually being a good programmer.
What is your objection to attempting to verify someone's programming skill directly? It's certainly not an easy thing to do, but it doesn't seem apparent to me that it's impossible or not worthwhile.
If the standard "whiteboard algorithmic brainteaser" is not a high quality signal (which I'm not sure I grant, Google seems to think it is, and they claim to have the data to back it up, though they're probably much, much more false-negative tolerant than most), it seems like there's plenty of dials to twist to try and increase the signal-to-noise and cost/benefit.
a) Length of interview (one hour? full day? trial run of several weeks?)
b) Topic (algorithmic vs. brainteaser vs. some attempt to simulate the "real world" to some level of fidelity?)
c) Interface (Whiteboard vs. on computer vs. something else?)
(I'm being a little disingenuous with the whiteboard vs. computer thing. I don't get the value of writing on a whiteboard. I'm kinda hoping someone might be able to elucidate that.)
There's probably not a one-size-fits-all. Depends on your tolerance for risk, how senior you're trying to hire, how tight the job market is, etc. But I really would hesitate to work at a place that just took people's word on their programming ability.
> But I really would hesitate to work at a place that just took people's word on their programming ability.
This is how I've hired for every open position in my company. You'd be amazed how easy it is to weed people out based on a conversation. It's very rare that someone can talk the talk without actually knowing what they're doing. There are too many obscure concepts and it gets clear really quickly if someone is bullshitting or has no idea what they're talking about.
…and even if you get a bad apple it is simple enough to let them go in a week or two. If you are really hiring, your recruiting process should be running continuously so the loss is not fatal.
> So, to clarify, you're advocating that programmers be hired based on resume, references, and a conversation about prior projects?
I think he is, considering this is the way every other engineering discipline does it. Please don;t fall back on the licensure issue, either. The vast majority of engineers in industry are not and do not need to be licensed. They still get interviewed this way.
> When I hired a plumber for my heating firm, I put her through a hazing interview, tested her knowledge of Maxwell relations and thermodynamic equations. Oh wait, no I didn't. As she has been a plumber for years, so we recognised her skill and experience and cut to the chase.
When I hired a plumber, I hired someone that was insured, had multiple good reviews online, and was able to give good, descriptive, and in depth details of the job he was going to do. Generally, a programmer will not insure their work, and usually you aren't hiring for a single job that can be designed and discussed in detail up front.
Even so, some of the contractors were still pretty damn shoddy, and I've had to fix their work (or hire someone else to) in the past.
Part of the problem is that I don't know enough about construction codes and the reasoning behind them to ask, eg, what kind of slopes and diameters pipes need to effectively carry sewage, when it's appropriate to use one type over another, etc. If I could, I would.
As she has been a plumber for years, so we recognised her skill
Because being a programmer for years doesn't guarantee the person will have any skill. I met a programmer who had been working for 10 years, and couldn't write a program to swap two variables. I don't know why, but for some reason experience is not equal to skill in the programming industry.
It's really a question of what kind of experience you've got. Many programmers with "10 yrs of experience" really just have 10x1year of experience; they've just done the same thing over and over again for a decade without pushing out of their comfort zone.
On the other side of the coin, that necessarily means that most companies asking for 10 years of experience actually only needed to give a person with zero experience 2 weeks to ramp up on the proprietary CRUD apps and uniquely broken reporting systems, and they'd do just fine for the next ten years without learning even one more thing.
The ridiculously over-the-top interview process is one way to signal to hiring candidates that they will actually need some form of skill or experience to work at that company, and that it will therefore be an exciting, resume-building place to be for the next two years. Unfortunately, since the boring, stodgy companies slavishly copy cargo-cult practices of the young, hip, trendy, and popular companies, that signal was already being swamped by the noise by 2004.
Maybe more tech companies should start using some of their revenue and funding to invest in training for people with the will and potential to get past that first year of experience.
You had to put the programmer through her paces because you had so many to choose from. If selection was limited, you'd be happy with just about anyone willing to work for you, but with the overabundance of programers out there, selection becomes a process.
My thoughts exactly, although that could be a regional thing.
In Portugal I was very amazed by the large amount of women driving buses, taxis and garbage trucks (for instance). Back in Brazil I must have seen at most five or six but in Portugal it is a very common sight.
Now for this kind of specialized trade jobs like plumbers, electricians and auto mechanics I personally never seen a woman doing it in both sides of the pond.
That's not how that works and there are whole websites for figuring out if the plumber is useful. Angie's List, Yelp, etc... Plus plumbers normally have to be licensed by the state.
Instead this process is left to the individual corporations. So, you get mixed results in judging quality, because there is no standardization.
When I hired a programmer for our doomed to die VC life support funded fart of a project, we put her through numerous rounds of comp sci bingo and algorithm hell. Why? Not sure. Everyone else does. I mean, hiring in IT is shit, why should we break from the industry norm? Why should we want to care about the candidates? Right? I mean if Google can do it...