A valuable practice, which surprisingly few tech companies do, is to ask candidates about diverse and orthogonal problem domains, or alternatively, allow the candidate to pick from a diverse set of problem domains. In the former case, you often find that candidates have difficulty writing even simple solutions for some problems but on others they instantly and fluidly can code up a good solution. Because they've had to do it in recent memory and still have the "muscle memory" from doing it previously. You often see bipolar results across the problem set this way.

For senior engineers with deep domain expertise in an area, there is an additional trap in that they use more sophisticated and often very different algorithms in their day to day lives than are applicable to the toy problem domains. Graph algorithms are a good example of this, and they are popular in interviews. The representations most engineers know (e.g. adjacency list or matrix) don't scale but the extremely high scale algorithms operate on a different set of principles that aren't trivial to code and aren't relevant to non-parallel cases; coding up an adjacency list graph traversal algorithm is going to be very unnatural to a software engineer used to doing the same on trillions of edges in real-time. It would be crazy not to hire an engineer on this basis but I've seen it happen, ironically because their expertise caused them to show poorly on the coding exercise.

Interviewing for technical skills is intrinsically difficult but I think that as an industry we are much worse at it than we could be. For all the claims of companies that they only hire the "top 10%" or whatever of engineers, the interview process is often optimized to the benefit of the median engineer.

So much focus has been put on 10x this and high performing that. Companies waste lots of engineering talent and company money looking for 10x when 10x is mostly situational in nature. Set the scene. In one scene the engineers are not producing. Change the scene and now they are 10x.

Instead of looking for 10x, that time would be better spent making incoming and current engineers better at their jobs and creating 10x situations.

I can relate to this. I have pretty good resume - good schools, advanced degree, impressive sounding projects, a long list of publications. My track-record suggests I am at least a 3x engineer.

So when I get into a coding interview, I'm expected to perform as such... but I usually come off as a 0.3x engineer. Like really bad. Forget syntax bad. Forget algorithms bad. I've bombed so bad in a technical interview, the interviewer questioned if I even knew basic trigonometry. They had spent 10 hours interviewing me prior and felt good, but got me in person and suddenly thought I was an idiot.

In each of my jobs, I've had a considerable ramp-up period. I'm talking 6 months or more of feeling completely lost and unproductive. I get in my head and it makes me perform worse.

If I'm not fired in that first period, I'm lucky. Only after about a year on the job I can perform at the level my resume would suggest.

I regularly look things like that up. I am a builder and a problem solver, not a reference manual.

"I'm your Google, anything you'd need to look up just ask me and I'll give you the answer".

Great way to take the pressure off plus it gives you a good idea on how well they ask questions/are comfortable leaning on others. It's a huge plus if I can get a reasonable candidate to admit that there might be parts of the problem at hand they don't understand.

I've written code tests where I'm expected to read the code, find the issues and fix them on paper. But when they take it too far beyond the spirit of what I believe to be a reasonable test of understanding and fixing code, then I write that I'd just for example compile the program and look at the output.

The companies that are most into questions like these seem to be the classic (failing agile) body shops with ridiculous turnover of their talent, which causes them to have to interview even more, which causes in turn for their tests to become even more abstract and worthless.

6 months to get into things is kinda long though. Unless it's super complex things, it should be less. Are you sure you are not exaggerating?

I respect your opinion but I find this strategy very strange. (Or maybe I don't exactly understand what you're saying.)

In my view, it would be a dream scenario if the next 10 programmers I hire were all superior to me. If I was the worst programmer on the team, that would be an ideal outcome. Yes, I've been programming for 30+ years but I'm also self-aware of my limitations. This gives me the ability to recognize better programmers and learn from them. If I'm not hiring interns, I do expect (or at least hope) that they are better than me.

Am I competent enough to write a "do/while" loop and knock out FizzBuzz in 5 minutes?!? Sure, but that doesn't mean there aren't better programmers than me. I would consider it a great business skill if I'm able to consistently attract superior programmers -- either to work with me or for me.

Larry Page attracted programmers better than him (e.g. Jeff Dean). Yes, Larry earned a masters degree in computer science but his java programming (the rudimentary 1996 crawler) wasn't considered very good. His employees rewrote the whole thing in C++ and made it scale for billions of pages. Bill Gates hired superior programmers. Mark Zuckerberg hired superior programmers. Etc. etc.

It doesn't seem like it. The next 3 sentences from the thegayngler:

> I have but one qualification. Can they do the job? Are they strong enough that I can guide them into the position I need them at if it is required.

In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them. They can come up with programming solutions I could never think of. It's very realistic that "the qualification to do the job" essentially means they are a better programmer than me.

Presumably, in your mind, "programming" skill is along one spectrum and one focus, as well? I think that's part of what's being talked about here. There are many, many aspects to software engineering/programming (depending on whether you want to classify them differently), and many roads that could have been taken to get to the point someone is at. I see no reason to think that someone that is better or worse than me in one aspect will also be so in every one of the other aspects of the job.

Let me try to restate that in more general terms and you can tell me if I interpreted it correctly: Since programming skill is multi-dimensional / multi-faceted, and it is unlikely for a programmer to be superior in _all_ dimensions compared to another, it is therefore not possible to conclude if one programmer is superior to another.

Is that a fair rewording?

My response is... sure, programming skill can be looked at as multi-dimensional. But so can other real-world comparisons. We prioritize the dimensions that are more important and collapse all aspects into a composite score.

Multidimensions such as choosing a new job: Company X is a 5-minute commute with 4 weeks vacation but Company Y has the newest technology and has lucrative stock options. Yes, it already goes without saying that there's more than 1 aspect of "desirable working conditions" at each company and no single company can claim a superior score on _all_ aspects. We can still weigh all aspects and eventually pick one (and only one) which is superior.

If I hired a "Jeff Dean" type of programmer, it would be an upgrade to my team and my company. Could I search for an "aspect" that I'm better at than him? Well, looking over his expertise[1], it looks like he doesn't have much frontend GUI programming experience. So yes, I guess I could claim superior skill on writing event handlers in C# language for buttons. That's not important though for my assessment. My point is I don't need for programmers to be superior to me in every aspect for me to determine that they are superior overall. (Same as determining which company is "superior" to work for.)

If I was founding a startup and I was the "ceiling" of programming skill instead of the minimum "baseline", my company would fail. I think I'm pretty decent at programming but I have no Dunning-Kruger delusions that I can "train" anybody that can pass FizzBuzz into a "Jeff Dean". I know of no 6-week bootcamp or even a 6-month corporate training program that can claim that feat. What I can do is recognize the programming skills in others that are (overall) superior to mine and hopefully convince the candidates to come work with me.

[1] https://scholar.google.com/citations?user=NMS69lQAAAAJ&hl=en

Well... no. But I think that's my fault, for the most part. I read the part of your original comment that said 'In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them.' as implying all their abilities as being beyond you, but it doesn't specifically say that.

I wasn't trying to imply that it's foolish to try to ascertain if one programmer is generally better than another (as long as some common domains along which to measure are involved), just to note that it's possible to hire someone that's far beyond you in some areas, and behind you in others, with the intention of specifically helping them build in the areas they are lacking (which have application to the job). The result should be a programmer that is, generally, as good or better than you in most areas, making them a more obviously superior programmer, even if that wasn't entirely obvious at the time of hiring.

In short, you can hire for the things that are important that you can't easily improve while allowing deficiencies in the areas that are less important which you feel you can help them improve in.

As a strategy, it has it's pros and cons. As a pro, it's probably easier to find candidates since your criteria is less stringent. As a con, you might find the candidate you hired has problems coming up to speed in those areas they are deficient in (which may be why they were deficient in the first place, instead of lack of experience).

I didn't really express the point very well originally, and just stopped at the opening, probing question.

Ive seen companies literally turn good engineers into underperforming engineers...stifling them with process, tech debt, politics, etc... Then they get angry some people can perform and others can’t under the circumstances and conditions the management created.

I’ve watched perfectly good engineers get fired or forced out for not being miracle workers only for people to later realize there was nothing wrong with the engineer in the first place.

You don’t know who is going to be the best until you work with them. I would advise people to pick people they will like even under the worst of conditions and train them to the way you want them to be.

I also find it remarkable how quick people are to blame the engineer, rather than considering the whole picture (process, tech debt, politics, etc). It's a complicated equation with a large number of variables.

In my experience the manager cares about fit and attracting you to the team, whereas the peers are all looking for reasons NOT to hire you.

Jokes aside; I agree with the advice of hiring people smarter than you in the domains where you won't be paying attention should the company actually take off.

If you're looking to be a founder/CEO level type, doesn't seem to matter what your skill in Java and OOP are, yes? No need to be a water walker.

That's the "genius" of management; they sell an idea and get someone else to do the work.

Imagine a programming ecosystem with e.g. 5 approximately equally popular frameworks. Suppose that someone with this attitude is familiar with frameworks F1 and F2, but never used F3, F4, F5. Put such person in charge of an interview, where equally skilled candidates (i.e. knowing 2 of the 5 frameworks) apply. They will only classify 10% of them as "senior-level" (those knowing F1+F2), and additional 60% as "somewhere between junior and senior level" (those knowing one of F1,F2; and one of F3,F4,F5). So they would hire literally themselves, but probably not someone on the same level.

IIRC a Google recruiter once sent a set of packets to a hiring committee and all of them got a "no hire". Then the recruiter revealed the packets were lightly-anonymized versions of the actual packets of the hiring committee members, from when they'd all originally interviewed.

I wonder how much of that comes from being exposed to all of the consequences any poor decisions they made shortly after they were first hired.

This is so correct.

Productivity follows when people are working for a worthy cause. In fact the core of productivity isn't lists, management techniques or whatever, it is a thing called 'constancy of purpose'

There exist no such a thing called 10x burger flipper.

If you don't encourage a culture where everyone is asked to hire people better than themselves, it can easily degrade where each progressive hire is slightly worse.

I was shocked doing an interview in my last round and got so many test questions. I bombed so hard because it wasn't what I was doing.

The real key to me is, we treat all of these jobs as more or less the same, when in reality they are all vastly different. There is no singular senior engineer skill set, but a variety.

I wrote a whole blog post on that: http://www.mooreds.com/wordpress/archives/2812

I just wish companies would be more upfront about what they really needed, as opposed to the wishlist.

I was declined for a position at a trading shop for a C# backend role because I used linq to do something. I didn't have unlimited time for the tests, so I went with expedient over performant, for which they did not care. Fuck that, set clear expectations for interview "tests".

If your priorities aren't aligned with the environment you set up, you are the one that must say so. On a real world project that means once in a while I'll take a project back for small fixes, instead of wasting unending hours thinking about every possible problem up-front... if you expect something different, well, that's not optimal.

This really seems out of necessity. There are going to be a lot more qualified median engineers rather than superstar coders, so it makes sense to optimize your hiring pipeline to assess these folks.

In my experience, most of the "top 10%" hires are either through interpersonal contacts or a fast-tracked interview process for a specific candidate.

At least when middle managers do it they're only interested in who gets credit for the work so they leave the average tech guys alone long enough to get the work done.

In that case, the key was that clumps of them had already worked together in earlier jobs, so the incompatibilities had been pruned.

If you got an interview, and did well you got another challenging programming problem to solve on-site. You were given a computer with the programming environment of your choice and as much time as you wanted, as well as someone to ask questions from.

I think it was also important that the company solved hard graph problems in Lisp, with a lot of macro-driven code. If you were a Lisp programmer, you generally wanted to work there.

You can pretty easily hire the top 1% of applicants, and still get only mediocre results. E.g. at one job, when looking for a senior-level eng position, clearly stated in the description, we easily weeded out a couple hundred applicants who had effectively zero experience whatsoever. No prior programming-related jobs, internships, side projects, open source, nothing.

Which is a waste of time for everyone involved. Hence the heavy reliance on connections :(

In reality, the applicant pool is heavily biased towards very bad engineers.

Exp: You must code in an unfamiliar language (Fortran 1988, for example) and do something simple in it. At least then you'll know that people are all starting from scratch. Time it, make sure it doesn't last more than 2 hours, see how laughably slow we all are, pick those that get the furthest.

Even being upfront about our low expectations and providing very friendly and helpful people to pair and guide, it lead to record numbers flipping the table and stomping off premises before the completion of the interview.

I don't use this approach now, as I don't think its a good use of time (and I think an interview should be a positive experience even for those that fail), but it did provide, I believe, a fairly strong signal of the attitude with which a candidate approached learning new things (but not really anything else).

Really, please, tell us more!

It's not that you're wrong, it absolutely is a really good way of finding out how someone reacts to being asked to learn something new in a stressful situation. And that is a valuable thing to know - I love working with people who get excited about an opportunity to learn something.

It's just that you have to be prepared to give so much help and the question has to be so easy because it's all completely new that you can't test competence in any meaningful way. Which means that at the end of the 90 minutes or whatever, you know exactly one thing more - their attitude to learning under pressure, but you have relatively little time with them, and you probably need to learn multiple things per session.

You also have to be careful when in the interview you throw this. A bad hit like this can throw a candidate off for the rest of the interview even if they are quite acceptable.

The Kobayashi Maru test.

1) Some people may find Fortran easier to understand than others even if they haven't touched it before. This was the case when my University decided to use Haskell* as the first language to teach students in order to level the playing field. It did help somewhat, it leveled the field for those with little and those without any prior programming knowledge. But to those who had already come to terms with recursion etc it was a very tiny stumbling block.

2) Some people need to dive deeper and gain a greater understanding of things before they feel at ease using them, but once they have reached that level, they will often outperform most other people.

* This was 20+ yrs ago and Haskell was not so well known back then.

nice to see it's more common

Oh wow, that would definitely get me sweating. All the FORTRAN i learned was 77!

https://en.wikipedia.org/wiki/Fortran

* Predictive of job performance

* Either (1) difficult to optimize against or (2) easy to optimize against but in a way that doesn't reduce predictive value

* Low cost to give

* Low cost to take

How about "Google is giving tests in these 25 languages"? They could even choose languages that are purposefully dissimilar from any language that you've claimed familiarity with.

After that he said "Okay, I know you know enough to handle the job, now I'm curious how much you really know. " (This is key, I feel. He specified I wasn't going to get dinged for not knowing the answers, so I didn't feel the pressure in the questions he asked afterwards, when I usually feel intense pressure, as I know I've been passed over for getting a single question wrong in multiple interviews in the past.) He then asked me some pretty deep questions about low level memory and other things, and if I said I didn't know he turned into teacher mode and taught me the concept.

I actually walked out of that interview having learned something new and useful. I then got to chat with the President very casually, and I received a job offer a few days later.

The interviewer new his stuff, too. He'd been programming arcade games in assembly for decades. His games have sold millions of dollars, and two of the series still get made today.

Ever since, I figured if a freaking legend could be satisfied after reviewing some code and a code sample, the crap I've had to endure everywhere else is completely and totally unnecessary, and it's just frustrated me to no end.

After several questions that he was sure I knew the regular stuffs and was qualified for the job, he said he was going to ask some really hard questions to see how much I really knew and what was my thought process. The questions had no simple answers. Each of them required knowledge and experiences from a few technical fields. I didn't know much then. He guided me how to solve the problems, discussed the pros and cons of each approach. Then he told me how they did it, how other products in the same category did it, etc.

The half-hour interview was prolonged to one and a half. At the end, he said, "You have a very intuition on what are the right directions to go when facing unknown complex issues", and gave me an offer. It is my best interview experience. (I didn't join the start-up due to other reasons.)

The technologies have been advanced in a blazing fast speed i the last 10 plus years. Technical interviews, on the contrary, regress in a similar speed.

One company I interviewed with gave me a program that had to be written in Scala, with the full knowledge that I would have to learn Scala to do it. Please.

I actually really like homework assignment approach when done correctly, both as an interviewer and as an interviewee (though seanmcdirmid's suggestion of prep material, rather than a homework assignment, is quite sound), but this is a separate issue, independent of how skills are assessed.

It's really about the company/team having a preconceived notion of what skills they are looking for vs. looking to see if a candidate has a distinctive skillset that could help round-out the team. How do you, as the candidate, market yourself and the things you are good at?

Obviously, the resume is a good place to do such marketing (and I'm sure you already are) so if teams are turning down applications based on the homework without looking at the other skills the candidate has to offer, then that might be an issue.

"The IT industry is committed to increasing the number of women engineers. But if they're too busy caring for their children to spend six hours doing our silly test, that's their fault for being women. We are forced to hire 20-something white males who have nothing better to do with their time."

For her at least, this worked out better for her. She does design, which is even more difficult to evaluate in 30 minute time slots.

Not that I’m arguing this is the way to go, I was just wondering if interviewers could give some hints on what they would cover I the interview, so we could prep like studying for a test.

I hope your not impugning the motives of those who issue take-home assignments. Existing practices for hiring are objectively bad at selecting qualified candidates, and most people are just trying to do better. There is some evidence that take-home assignments are an improvement above the norm.

Also, consider that such assignments can serve as a "blind-audition" and can help eliminate bias from the interviewing process. And there are lots of ways in which loading everything into the interview unfairly penalizes certain types of otherwise qualified candidates. What's left, referrals? That's definitely biased, even if it also tends to have a lower false positive rate.

An easy switch would be to convert take homes to onsite work. One of the more recent interviews I had consisted of putting me in a room with some data and a stub of code and asked me to fill in the stub over a few hours. Every so often, they’d come by and ask if there were any problems. (More frequently at first, and then later less frequently.)

It’s basically the same, but it’s timeboxed and well defined with enough back and forth, to not feel like they’re just throwing a problem over the wall to you.

They are relevant enough that the prior poster felt the need to smear them.

> if the effect is the same

The claimed effect is, at this point, purely hypothetical and indeed runs contrary to my experience.

> An easy switch would be to convert take homes to onsite work.

Generally speaking, an onsite task like you describe seems like a reasonable choice for a company to make. However, this strikes me as worse for the affected groups that reaperducer was concerned about:

- It requires scheduling an even larger fixed block of time outside one's normal job/life responsibilities. A take-home assignment at least allows for flexibility for when you work on it. It strikes me that time-poor candidates would appreciate the flexibility.

- It would no longer be a "blind audition". When I've reviewed take-home assignments, I don't know anything about the candidate except that the code. It strikes me as a good way to resist unconscious bias from seeping in.

The the on-site solution remains a blind audition if the people doing the evaluation are not the people administering the test. This is exactly the case for a take home assignment, only now it is timeboxed and with access relatively timely help if needed.

The purpose of the blind audition is not to hide the identity of person being interviewed, as much as it is to enforce an objective standard and remove bias. If the rubric is, “Accept all that got automatic evaluation score of at least X”, then that achieves a similar goal.

As far as time-poor people, never underestimate the desire of having a clearly defined block of time. Of course, onsite or not can always be optional as well.

You will never be unbiased in a hiring process, because the bar you set is not universal, and always be biased in some way. Being "as blind as possible" isn't the right metric, and we shouldn't hire based on it.

Yes, that's exactly what I'm doing.

I wrote a thing about it and I got someone who responded to me with a rather scathing rebuttal. I don't know, it's very conflicting.

Post in case you're interested: https://dev.to/spirodonfl/should-i-accept-coding-challenges-...

Edit: better wording

However, don't see how well take home can work for senior engineers. In fact, its unfair and delays interview process to expect take home assignments from working folks.

How does a fear of firing impact hiring procedures? If you are afraid of firing, you're afraid that you won't be able to get rid of a toxic individual; that a single individual will act as a poison to the morale of your entire workforce. Thus, you want to make damned sure you don't make a bad hire in the first place - even when it means you leave a position open for a really long time.

This fear of firing has gotten so bad that FAANG-alike companies directly espouse how bad the impact of a bad hire is, how they would rather give up on 99 good candidates than hire one bad candidate.

But that's bullshit.

That's letting the fear of interpersonal interactions drive business decisions. The impact a bad hire can have on a work force is pretty minimal when caught and addressed quickly; it can even provide an overall boost to morale to know that the company is willing to address real problems in an adult way. Even a perfect initial hire can turn toxic after a few years: what will you do then, if not fire them?

Do you have 10 positions and 10 potential candidates? Hire all 10 folks, and fire the one bad person; your company will be better off for the decision. Much better off than it would be if you instead overwork your existing team because you haven't found your unicorn hires yet.

The simple explanation is that it's really expensive to fire someone without cause. It takes months because the company has to produce a paper trail proving they fired the person for a valid reason to protect themselves from litigation. That's months of paying an individual who may be incompetent or toxic, months of paying people who put together the paper trail, all the morale damage the individual causes, the damage that might arise from firing someone, even if it's someone no one really liked.

All of those things are costly, probably much more costly than not hiring the right candidate. At least so the reasoning goes.

1. As another commenter pointed out, most jobs are in "at-will" states.

2. Many companies require arbitration instead of litigation to settle any disputes as a requirement of employment.

3. It's just as expensive for the fired employee, probably more so, since if you litigate, you're unlikely to ever work again.

The cost is also not that high in non-at-will states, since there's typically a 3-6 month "evaluation" period where no paper trail is required.

There’s a lot of political bodies focused on playing up the idea that women and minorities are this big bad wolf suing legitimate businesses left and right over nothing, and winning. But it’s a largely fabricated narrative created by people trying to turn the leftist identitity politics arguments into a quagmire by “both sides!”ing the situation.

In reality there’s a substantial burden of proof for someone to win a wrongful termination suit.

Even if you hire two do-nothings and a negative-performing person (consumes half an FTE's day in questions), you still end up with 6.5 productive people. And frankly, a 65% productive workforce is pretty damn good. I've seen lots of companies where <65% of people are productive.

Plus, some of the do-nothing people can be alright with some direction. Sometimes people don't get anything done because they just don't know what to do. I've been on so many teams where the manager said, "we need a person to do X" and they end up with some random person that doesn't really know how to do X. The solution that I've found is to find something they can do and have them work on that. Most projects have no shortage of work that's more time-consuming than difficult.

For negative performing people, I usually give them something kind of busy-work that doesn't involve the rest of the team. They might not be getting anything done, but they also (mostly) aren't weighing down the team by asking them questions all the time.

Why waste time? Why not just fire them?

a) Protect a vulnerable class of people from being fired (say over 40 people)

and not

b) Require 30 forms when firring someone in that class for perfectly valid reasons

Seems really really hard to hit both. You need to err on one side or another.. which way do you go?

There are plenty of people in non-vulnerable classes who regularly do indefensible things because they know their companies won't act on it.

I've also explicitly called out the FAANG hiring practices, which are frequently held up as the best practices in the industry by those doing the hiring. After all, who wants to be even partially responsible for hiring "that guy" when you can not or will not fire them later?

But, here's my point: I've been part of the process that hired "that guy". I was also consulted on the decision to fire "that guy" a month later. I was wholly on board with the action in both cases; and in the process we got two other great employees that we never would have hired if we were afraid of taking chances.

To me it paints a terrible picture. My suspicion is that the dangerous bad hires are not the utterly incompetent people who will flail in place until removed. The dangerous ones are going to energetically do all sorts of peripheral, irrelevant work, try to insinuate themselves into a lot of 'process' type setups, and move around the company in a way that stops anyone from ever noticing that they can't actually do anything. I suspect a lot of BigCos have very weak immune systems against such mobile 'attacks'.

Is there not also a fear that an overworked person will become toxic?

Apparently not at my employer. My team was 10 people 4 years ago, and we're down to 6 now, without much dip in workload. We had someone leave about a month ago, and we're still fighting to get a job rec to replace that person.

[1] https://changemyview.net/2018/03/28/thats-bullshit-rude-enou...

They miss basics of OO programming etc. And then what? you take him in, which has costs time, money, hardware to let him go within a month? On the other side I think you can weed these out quite fast with 1 personal interview.

Looking back, often no talent has been better than poor talent (or those that actually decrease group productivity). Huge opportunity cost with bad hires, and even more challenging to move them out to somewhere that they would be happier/more effective/less disruptive.

In those circumstances, it's easy to imagine the mental math: Firing someone could cost many times their salary, but a screening service costs 50 bucks per candidate. So the chance of weeding out one bad candidate pays for the service, right? Where's the fallacy?

I think the fallacy is that the service doesn't actually improve the hiring process, and the ROI is zero.

That in itself isn't unreasonable. While much of the US has at-will employment, most of the rest of the world does not, and hiring the wrong person can be extremely expensive.

However, I suspect with current hiring trends we should not attribute to malice that which can be sufficiently explained by incompetence.

Interviews are imperfect, and damaging candidates will slip through the cracks. Are you seriously suggesting that a business should decide not to fire someone based on their immigration status, rather than their performance and integration with the team?

Plus now there is a 60 day grace period for H-1Bs to try and find a new job.

It's surprising to see the number of people who interview for lead technical roles that cannot code, or whose work is exceptionally sloppy. Incompetence is more commonplace than the author believes, even at the highest level.

I can try and come up with reasons why that might be the case, why a simple OO modeling problem + a couple of trivial algorithmic problems (like, take an ascii string, return me a dict mapping characters to character counts) would trip someone up...but that isn't sufficient justification for me to want to continue to an onsite.

And that's the people who get past the verbal screen. Plenty drop out at that point. "I see you spent two years as a Senior DBA. Can you tell me what the acryonym ACID stands for?" "No, I'm not familiar with that" "Okay. Well, the 'C' stands for consistency. Can you tell me what one means when we talk about data being consistent in the database?" "It means when you write something it stays written (or some other made up twaddle)" "I see."

That said, I agree that coding tests that are completely unrelated to the job, and are geared toward college grads rather than long term developers (i.e., "Implement a (data structure)" or "Remember/discover an algorithm that was a PhD thesis 30 years ago to solve a contrived problem in a theoretically optimal way" rather than "Solve a real problem of a kind similar to what we'd expect you to do here") are dumb.

For some reason that company insisted on only doing interviews at 7:00 AM, and my brain doesn't come fully online until after 9:00 anyway. I felt annoyed later when the answer came to me that afternoon, too late: put the list in a hashtable, add all pairs of numbers for O(N^2) complexity, and check if each answer occurs in the hashtable.

Much later after that, I realized that all the other red flags I'd picked up on while interviewing there added up to an impression that their corporate culture was seriously f'ed up, and that I probably dodged a bullet by getting passed over quickly in the process.

They were probably looking for the answer of, "Lets talk through it and figure out a better answer". Coders are often under a mistaken impression that interviewers care about your answer. They don't. They care about how you approach it, how you think through it, what you do when challenged, etc.

So from their perspective, they asked you to try harder on a problem, and you just said, "No." And they dropped you for it.

FWIW, I had similar questions in some interviews where we hashed through similar problems, and together collaborated down to a "No" answer... but we spent 15 minutes exploring the problem and seeing how well a couple coders can work through it before saying no. And I got the offer despite an otherwise rusty performance.

This isn't always true. Some interviews are really about solving the problems as fast as possible. And lots of interviewers are looking for exactly the answer they have on hand, and will think you're doing it wrong if you come up with a different, but equally valid (or better!) solution.

I was asked to sketch the proof for the irrationality of sqrt(2) in a quant programming interview. I kind of froze, and explained that though I had learned it, I could not recall.

He prompted me with "Well, what does it mean to be irrational?", and with that little hint I did the rest. Though ultimately I did not end up working there, it was very satisfying to have answered it.

Well, in one case (me), all the information the interviewer had was "this guy happens to know how to implement a quick sort by heart. K.".

In the other case (my friend), he got "this guy is able to understand and implement an algo he never saw before in a reasonable time", which is much more impressive

> We now show that the equation (1) p^2=2 is not satisfied by any rational p. If there were such a p, we could write p=m/n where m and n are integers that are not both even. Let us assume this is done. Then (1) implies (2) m^2=2n^2. This shows m^2 is even. Hence m is even (if m were odd, m^2 would be odd), and so m^2 is divisible by 4. It follows that the right side of (2) is divisible by 4, so that n^2 is even, which implies that n is even. > The assumption that (1) holds leads to the conclusion that both m and n are even, contrary to our choice of m and n. Hence (1) is impossible for rational p.

Assume sqrt(2) is rational and has the reduced form (x/y). Thus we are assuming that x and y are integers and that gcd(x,y) is 1. You square it and get x^2/y^2 which are somehow simultaneously coprime integers and also reducible to 2/1...

Huge asterisk that as long as you arrive to the correct solution with minimal help. I've had plenty of algo/ds interviews and it seems like needing help to see the trick in the question pretty much means you're out.

In many of the places that do claim this, even if you solve the problem "correctly" you still get more brownie points than someone who got a less efficient answer or didn't get an answer. Meaning that they /do/ actually care. I do get that some places are smarter with regards to this than others but it's depressingly common in my experience. "You didn't get the O(n) solution that has weird edge cases and was published as a paper in the 70s? Too bad, because some other person did, because they remember the solution from some programming interview book!"

At my last company, we would sit a candidate down at a pairing station. We'd offer them about 5 problems to choose from, and together we'd pair program on the problem for 2 hours. Using the internet, the IDE, anything they want was totally fair game. No system is perfect, but this was by far the best one I've ever experienced.

My immediate solution: Sort the two strings and then compare them:

  (defn anagrams? [x y] (= (sort x) (sort y))) 

"Ok, what's the efficiency of that solution?"

"Well, assuming the library sort functions I'm using are sane, I'll take a guess and say O(n log n)"

"Can you come up with a more efficient solution."

Off the top of my head, on the spot, I couldn't. Later, during the plane ride home, the obvious occurred to me: Don't sort the strings, just scan through each string once and build a hash table, keeping track of the number of occurrences of each letter in each string. Then compare the occurrences for each string. Not as elegant to express in code, but faster.

As it turns out, they later declined to hire me, giving me an excuse that made it clear they weren't really serious about hiring anyone for the position (as is the case at least half the time, it seems).

And thus ended my latest round of failed interviews. I cancelled the last one I had scheduled (probably another fake) and decided to take a break for a while (I mean, I do have a job right now, so it isn't urgent).

You’re right.

So I think something silly like: character_code = len(string)*len(alphabet)^character_index should work.

ac = 1 + 3 = 4 bb = 2 + 2 = 4

Still, consider the word ‘abe’ with a = 2, b = 3, c = 5, d = 7 and e = 11.

abe would then be 2^1 * 3^1 * 5^0 * 7^0 * 11^1. ‘abba’ would be 2^2 * 3^2. Each anagram would have a distinct value.

See also Gödel numbering and FRACTRAN.

I wasn't rejected but I've been asked to retake a phone interview. Apparently, the interviewer had very good feelings about me, but found that I was "out of practice" as far as coding goes.

I wonder if my age is an issue (40+) or if they have a really high level of expectation. Are most other candidates super fast at solving these kinds of problems. I'm a bit disappointed because I'm not sure that I've got much room for improvement at that stage.

They want 23 year olds who just did their finals. You know, real stallions.

You know, real pegacorns.

The answer to that is almost always yes.

Maybe the interviewer was biased and led to think I was "out of practice" because he knew I was on the older side. But I'll give them the benefice of the doubt. I believe it's a competitive position and I simply wasn't in the right percentile to qualify.

Now, would my former self 20 years ago be more competitive for this kind of interviews? slightly faster perhaps, but I don't feel I'm at a disadvantage compared to younger candidates.

From my perspective that is only about 3% of what I expect from a software engineer. Writing readable, easy to maintain code that is well tested, collaborating with product and other developers for how a feature should work, influencing technical designs, giving good feedback are things I value much higher than how quickly someone can write a snippet of code that works.

This is _a_ metric but I wouldn't bet too much on it. I know lots of people that come to optimal solutions to "algorithm type" _much_ faster than I do. I'm pretty slow at that type of stuff. But... its just such a small part of what makes someone a good programmer. Building out a medium to large application requires balancing a lot of trade offs and figuring out how to keep things simple. I know seasoned, quality algorithm solvers who honestly just repeatedly churn out garbage applications. And they can tank the productivity of an entire team of developers in their wake. I don't know how you test for that, but I can promise phone screening for problem solution time isn't it.

How did you arrive at that conclusion? Is it based on data or does it "feel" right based on an easily-measured metric?

Translated: "the one that did it much faster is a more recent graduate and exploitable for 100-hour weeks at below-par pay, hire that one".

Congratulations, now you know how to hire at a tech company!

The skill takes a long time to practice, and quickly evaporates once you stop doing it. Yet none of our work is anything like that. a more real world situation is reading documentation or SO for the function concat_ws that will turn an array column to a concatenated string.

however, the system that is, and one that you seem to think is ok, is one that discriminates people on many levels. Its a skill you do not get good at by working, so you must practice this in your free time. Now you just discriminated against men and women with families, people from less fortunate backgrounds, and others who otherwise do not have the time in the day to dedicate to a skill that is only useful to coding interviews.

as far as a comment above, I was a DBA for 10 years and can do pretty complicated queries off the top of my head, know how to optimize my indexes, worked with both structured and unstructured data in the multi tb size, ect ect, but I have no recollection on what ACID stands for. I dont really care. Sure, you may hire someone who memorizes useless crap, but then they have no idea why the IO has gone through the roof when inserting IDs out of order on a clustered index.

I just look up joins when I need them, and it's straight forward, but ask me in an interview and I sound like an idiot.

For many tech companies where they get a lot more candidate applications than they have positions for, the goal of the interview process isn't to avoid losing good candidates, it's to not hire bad candidates.

So much more goes into software engineering than "Can solve tiny example problems in an examination system"

True that software engineering is more than that, but those other things largely depend on in-house company processes, tools and policies so you would learn them afterwards.

It annoys me that the good answer if you actually encounter the problem during the job is never accepted during an interview.

They aren't looking for people who don't know to start by doing some research, they're looking for people who won't be completely stuck when that research doesn't turn up the answer.

Which means what they're really saying is not "we want problem solvers". What they're really saying is "the bare minimum for entry level work here is being able to, in 30 minutes and on the spot, out-perform top-tier theoretical CS researchers".

Which in turn really boils down to "be a recent CS graduate who memorized this algorithm in advance so as to 'derive' it later on command".

So then I try and give them a O(n^3) solution (or something).

Of course this isn't accepted as there's a better O(n^2) solution, which can always be found by googling, and I fail the interview.

The "turtle and the hare" problem of finding a loop in a single linked list in O(n) time and O(1) memory is the perfect example. It's easy if you know the answer (or can google) but basically impossible if you don't.

What did the interviewer do when I said I couldn't do better than a hashmap? He laughed and said "not many can".

The gist of it is: walk the list and after each step, multiply the previous node’s pointer by negative one.

If the current node’s pointer is negative you’ve found a loop (because you colored it by inverting the pointer in a previous step).

Clean up by starting at the beginning and multiplying all pointers by negative one until you reach the last one you inverted. Then return your answer (true).

If you reach a null pointer there is no loop because you’ve gotten to the end of the list. Clean up by going through again and multiplying all the nodes by negative one. Then return your answer (false).

Your worst case running time is 2n (if there is no loop) which reduces to O(n) and you use O(1) memory because you’re coloring the list by using the already existing pointers in the list.

The problem that I have with those problems is, they always assume it's easy to answer after seeing the solution. But if you're working on them for the first time, it's a terrible environment. The minute that they stop asking you, as the canidate, are on a timeline to finish. (Also, to finish as well)

The answer to that item would be: (with scala)

> datasource.window(3,1).filter((a,b,c)=> a+b == c)

I mean if you knock it out of the park and know the internals of window.. then asking the bar raiser question is appropriate as a curiousity.

There are other problems as well: Those are the ones where they have an expected answer, response, and reiterate. I've seen that with tree problems. That's where they would low-key ask for the recursive version and then get you to say stackoverflow exception.

Reminds me of my final round at Amazon, which they always do in Seattle. I woke before 4 AM west coast time having flown out the night before from the East, and 12 hours later was still coding on a white board. After writing (to my surprise) a correct merge sort, they asked my to write a program to do basic math with, IIRC, binary numbers. I was like, um. I'm out.

And if you throw in distributed databases, there is yet another understanding of "consistency" (vis-a-vis the CAP theorem), which really means "linearizability".

Sorry, didn't mean to be super-pedantic but consistency as a concept may seem simple to the uninitiated but is easy to get tripped up on, especially for folks who are aware there is a deeper layer of understanding associated with the concept, but can't articulate it right off the bat.

All this is to say filtering people truly is a hard thing. The extremes are easy to tell apart, but people who hover around the average are much less differentiated.

[1] https://en.wikipedia.org/wiki/Consistency_(database_systems)...

And I have similar anecdotes related to candidates who claimed years of distributed systems architecture experience, and were yet unfamiliar with CAP (and when explained, could not tell me even at a high level whether, in the event of a partition, their system was CP or AP, let alone the specifics of how that actually exhibited itself).

One of my mentors said that about an interview he conducted a while back. I found a lot of truth in that line. I run my tech interviews by starting very green and let the candidate dictate how fast I ramp it up. I've gotten pushback from managers before that you can't start with basic questions, but I've equally gotten positive feedback from candidates (even one's who've been failed the interview). In the end, programmers want to see an algorithm that allows them to judge an interviewee. I don't believe one exists.

Once you start working on a particular database, and are working on all of the specific details of that db, how often does ACID come into things, really? It just doesn't.

If you legitimately did the work without picking up -any- of the terminology, -and- respond confidently to questions you should know you don't know the answer to (rather than admitting ignorance and asking for follow-up questions, or declaring assumptions, i.e., "Well, I would assume it has to do with the behavior that if a constraint is violated or something similar in a transaction, the transaction will be rolled back instead of committing"), I don't want you. Because it means you both did not have formal training in, AND didn't seek out information or knowledge in the domain you were expected to be -senior level- in.

And while atomicity and durability are just characteristics of a relational DB (and so adminning one doesn't really require you to understand them), isolation and consistency both have definite relevance, because the isolation level is configurable, and that affects the consistency guarantees of the system. I expect someone saying they were a -senior level DBA- to be able to talk intelligently about those behaviors, and yes, to understand the words, because just reading the docs would have introduced the words.

Otherwise, I think we simply disagree in some respect, although I think it's over emphasized due to the topic. I think both perspectives have a level of truth to them, and have their flaws. It really depends on the candidate, and what they really know. Perhaps my particular circumstances are sufficiently odd enough that they aren't useful in a broader context. Who knows.

I don’t think it’s essential to know this, but I’d be surprised if a web dev interviewed with us and didn’t know at least the basic version I’ve given above.

Just my experience, perhaps I live in another world.

I think this is partially what I'm referring to. I use these terms in conversation with others when designing systems because they are useful in describing very specific properties of databases.

There were people who were bad at (possibly some because they were under pressure), it but nobody who couldn't do it at all.

I wasn't giving out a trivial question either.

I'd suggest a widely-accepted professional certification could help a lot, like doctors and lawyers have with the medical board exam or the bar exam. Easier said than done, but what we have today, where the candidate pool is overflowing with impostors with great resumes is not working.

"i passed the bar exam!" and nothing else doesn't get you a legal job you want.

and your medical boards, if i recall correctly, just mean you're qualified to go be a slave, er, uh, resident, some place for a few years.

Find me good certifications that are actually meaningful and relevant to my career, and I’ll do the study work and get them.

I have yet to find any.