I've had similar experiences to yours, but maybe this doesn't fit the leetcode people are complaining about.

We hire with a similar setup, but we try to keep the test extremely simple and close to reality. Something they would likely encounter on their day-to-day.

The majority of the developers fail at it. Senior, mid-level, juniors, etc. It doesn't matter. Which always surprises me. People who claim to have 5-6 years of experience can't even refactor some code. Or they can refactor and can't explain why they'd do it. It's bizarre.

But I would never classify this as leetcode because these are common tasks that any developer should be able to solve and not purely algorithmic questions that will rarely (if ever) be used in real life.

If you have 0 passion for programming and land a gig at a corp that doesn't care about code quality at all, you could do that job for few years and just coast/do bare minimum.

I'm an IC and I could deliver software with all methods in a single class, nobody would care I think, as long as it worked as intended. I happen to like programming and as I am improving my skills every time I touch an existing project I look for things I could have done better. If major changes are being made, I often upgrade frameworks or do a partial rewrite to improve things.

I know people who just do stuff without trying to understand why. If I cannot explain why I am using dependency injection and what problem it solves, I am not doing it correctly.

I have had the same experience. Some things I've witnessed:

- multiple "senior" data scientists who cannot write a function definition in python

- a "senior full stack engineer" with 5+ years of web dev experience who took 20 mins to change the width of a div in a web app

- many "mid/senior level" full stack engineers who don't know the difference between sync/async, or have any clue about how the JS event loop works

There are so many candidates out there who have what looks like impressive CVs, but can hardly string a program together.

Your first two are pretty bad (possibly panic induced), but the last one might not be realistic. You can rack up multiple years of experience working on low user count apps without knowing those things. You can still write Javascript and solve business problems without knowing those things.

But also, seniority doesn't determine if someone knows those things. If I asked you for a list of things I should know by the time I get to mid/senior, someone else will have different expectations. Everyone is kind of inventing their own requirements on what someone ought to to know at junior/mid/senior levels.

> You can still write Javascript and solve business problems without knowing those things.

It depends what the requirements of the role are. For some frontend React work maybe, but the engineers we were trying to hire would be working on APIs and processes that handle high volumes of streaming data. In that context using sync methods over async would significantly impact throughput.

When asked "why are you using a sync method here" the answer would often be "I want the code to pause here before the next line executes". It's a fundamental misunderstanding of what's happening under the bonnet that IMO is required for this sort of task.

Also - I agree that labels like "mid" or "senior" shouldn't carry a specific list of knowledge requirements, but they are useful to set benchmarks for what a candidate should be able to roughly offer in exchange for the level of comp they are requesting.

If you don't know the difference between sync and async you're not qualified to solve problems with any realistic level of scale (>10 concurrent users which is a very low bar) or complexity (have you really never had to code logic that uses 2 or more concurrent network calls?). Might be excusable for a junior engineer but definitely not someone who describes themselves as senior.

you google it and understand it or re-understand it in 5 minutes after trying 20 iterations in the IDE.

thats representative of the job, so it should be represented in the interview. open book interviews test for resource finding and implementation skills.

You shouldn't have to google a basic concept that, if you truly were senior level, you would be using on a daily basis. This is not some obscure language syntax or feature.

They shouldn't be penalized for it either, using reference should be a force of habit.

The main problem with your response is that the threshold for "basic concept" is arbitrary by framework, by language, by time, by interviewer. Revert back to interviewing things relevant for the day to day job, look at what developers actually employed there did over the last few weeks, and interview on that.

Knowing the difference between async and sync is relevant to any JavaScript developer's daily tasks. If you are hiring a JavaScript developer it's 100% a fair question to ask. If you are hiring a generalist maybe not, but if they have JavaScript experience in the last 3 years listed on their resume, I again say it's fair to ask.

Asking about async/sync in JS is completely different from asking someone what obscure features like finalizers are in Java. One is used on an hourly basis when writing code, the other one is never used except in obscure legacy code.

If you do what you're proposing and just ask about projects I think you'll find it's easy for incompetent people who are floundering around and copy pasting from stackoverflow without understanding what they're copying to pass your bar. I copy from stack overflow all the time but I understand what I'm copying and why it works and when it doesn't work.

> Revert back to interviewing things relevant for the day to day job, look at what developers actually employed there did over the last few weeks, and interview on that.

`async` questions are the most day job relevant questions I can imagine for a javascript developer. It is so pervasive, a such a simple piece of syntactic sugar over basic asynchronicity, that not knowing it is damning along a number of axes.

If you have made it through multiple years of building UIs without ever needing to google that, then you must have a profound lack of curiosity.

You can certainly stumble your way to a semi-working app without knowing that, but your mental model of how Javascript works is fundamentally lacking if you don't know that.

its still about the syntax and referencing that, there are like 3-5 ways of doing asynchronous code in JS

and its also not my point about "async in JS", thats a symptom in a field where ever random person pulled in to conduct an interview has their own random threshold of competence, when simply conducting an interview that mimics the day to day on the job would be more accurate

"did you google that in 10 seconds" great!

"did the compiler autocomplete your method name or show you a list of likely possibilities?" great!

"did you post a question on a QA website, while you went to work on other parts of the task while the hints and answers rolled in?" efficient!

"did your compiler show you a syntax error immediately that you corrected immediately?" great!

"did you compile it as you went and rapidly iterate towards the write answer like everyone else working here?" perfect!

whiteboarding, and most collaborative coding platforms, fail to replicate 9/10ths of that. so it tells nothing about the candidate.

> multiple "senior" data scientists who cannot write a function definition in python

I definitely would have failed this a few years back, because I spent most of my time in R & SQL.

1: Bad.

2: Could be terrible or totally reasonable... how much of that was "fixed cost of any push to prod regardless of complexity"? I've worked at very few places where I could get a change from my editor to production in less than 10 minutes, except in emergencies. Even at my one person freelancing shop a push to prod took at least a few minutes to go through the pre-flight checklist.

3: Bad but not insanely bad, depending on what's needed for the role. Definitely something they should learn, but not catastrophic especially if they are a huge value add in other parts of the stack.

You can be full-stack with minimal JS knowledge, if you use stuff like Blazor. Lack of knowledge about sync/async would be inexcusable for a mid/senior though.

Why would you quiz on language specific syntax?

I should have clarified in the original comment: the roles all had python experience as a specific requirement and some of the candidates apparently had multiple years of experience with Python listed on their CVs.

Depends on whether the candidate claims to know Python on their resume. If they do so, I'd certainly expect some basic ability to read such code.

Almost all big companies let candidates choose their language for the interview, with assumption it'll be python/java/c++ or similar mainstream lang.

I don't mean this to be incendiary, but, do you think perhaps this reflects on your recruitment pipeline, the standards you hold interviewees during the discussions about their experience, and the way you review resumes?

There is of course a possibility that someone has rehearsed their interview happy path down to the word and they can easily describe complex technical concepts without understanding them because they're quoting a script, but an interview that can be gamed in such a way may highlight a problem with the interview.

If most of your interviewees are failing to write basic code, the issue probably isn't the applicants.

I've had a very similar experience to this, these are literally with candidates that are extremely "over qualified" based on their resumes. We're talking masters degrees, doctorate degrees, etc. They can talk like they have years and years of experiance at this place, that place, you ask about projects they'll dive into high level details about them, everything sounds good (can still filter 80% out with these questions to be fair). Get to the code question, boom fail.

I dislike leet code, but you do need to be able to code something (Even if it's syntactically incorrect) on the fly and reason with me about what and why you're doing something.

For starters having a masters or PhD implies _less_ programming experience, generally speaking. Its a huge plus for a variety of positions but IME PhD or masters candidates were always a little more raw engineering wise. The way you get good at coding solutions quiclky and on the spot is by writing a lot of code.

Particularly PhDs in Math, EE, or areas of CS that aren't engineering heavy. You hire them to solve problems that your engineers can't solve, not (just) to write code.

Masters degrees are a total crap shoot. I honestly ignore them entirely unless there's a thesis.

Not incendiary at all. I've been worried about that, but as we've adopted more skills assessments, the quality of hires has went way up as measured by first year retention. We're not losing people because they can't do the job.

I'm surprised that someone who couldn't say what type a python expression returns could do well on "show me things you've built", "tell me how this function you built works". Do you have an expalanation/guess? Were they giving you rehearsed answers to specific code that, well, they didn't necessarily actually build themselves?

Best guess is someone else did the work and explained it to them.

You’d pop that bubble in a few directed questions wouldn’t you?

You definitely can but it's not easy. These people are really good at bullshitting, that's why you have to back them into a corner. The simplest way to do that is to have them prove, live in the interview, that they can write code.

You mean stuff like asking what type a function returns?

Woah. This is a great post! Zero trolling: Can you share your interview question? We would love to have something similar!

Deeper question: Did you think about why some people are able to "show me things you've built, tell me how this function you built works", but fail miserably on a straight-forward coding test?

Sometimes, during an interview, it seems obvious the person is very intelligent, but having a bad day. Some part of interviews is pure performance to me.

Not OP, but we often include these two in our interviews:

1. Write code that will find the second largest item in an array on ints.

2. Write SQL that selects the second highest number from a DB table.

I'm floored how few candidates can do these. Many end up giving up or fiddling for 10 minutes before giving something like `SELECT table WHERE 2`.

wouldn't be surprised people miss the SQL question. Full stack devs are rather rare nowadays, and it's a bit obscure SQL trivia considering the answer is dependent on which SQL db you're using.

Modulo syntax/keywords, I think the following query would work in any modern dialect:

    SELECT num FROM 
        (SELECT num FROM nums ORDER BY num DESC LIMIT 2)
    ORDER BY num ASC LIMIT 1;

which is actually nearly identical to the simplest Python solution in terms of the computation that's specified:

    lambda xs: None if len(xs) == 0 else sorted(xs)[-2:][0]

I think the reason the question is tricky for some is because they either:

1) haven't used SQL in a long time and forgot all the syntax, or else

2) don't really "grok" SQL as a model of computation (as opposed to e.g. a data structuring language or something like that).

But both of those seem like good reasons to disqualify someone from a full stack role?

A common reason in my experience is that the things they've built were done very slowly. I've failed my share of interviews because I was having a bad day, certainly, but I have direct professional experience with many people who require hours of dedicated working time to produce even the simplest of pull requests. This isn't a complete justification (if I have professional experience with them doesn't that mean they passed our straightforward coding tests somehow?), but I do think that there's a real selection problem there people are trying to solve.

Are you US based?

Here in Europe this sometimes happens (candidates can't code at all), but _very_ rarely, maybe 1 out of 100? Especially as you move eastwards, the base coding abilities start to improve massively for the average candidate - in the UK, if you are stupid you can easily get by on the social welfare system, in Eastern Europe, if you are stupid, you starve. As simple as.

There was a time I used to ask Fizzbuzz-style warmup questions on interviews, but then I stopped because the signal it gave was pretty much useless. The only positive case I can remember was when a tester applied for a dev position.

I worked in Russia and used to think this way too. But once we managed to hire person how couldn't code. At all. He was intelligent during the interview, talked his way through system design section. But during the probation period he basically couldn't do anything, even simple stuff. After that I started to block any candidate, who couldn't provide code solution for at least simple problem.

Er, that explanation doesn’t really hold, considering if you’re stupid you starve in the US too, and we get tons of idiots applying for jobs they’re completely unqualified for.

Agreed you need some tests but what you're asking is a different order of magnitude than leetcode problems.

Not really sure, but it sounds like some of things you are testing might be syntax - which makes sense if it's critical to your hiring need - ex. hiring senior C++ folks to start a new project - otherwise every new job I've started in this industry thrusts me into a new language and I have no problem sending CRs almost immediately (given Google and an IDE).

I'm generally more a fan of keeping skill-based examination too, but making it very easy and language agnostic. Harder than FizzBuzz of course, but nowhere close to dynamic programming.

Ideally, if a candidate has never seen it before, a basic tree traversal is enough. But since 100% of people have practiced that before interviewing, I invent a problem by ripping out a little piece from the codebase I'm working on. It usually amounts to.. Show me some clear control flow, factor that code into a subroutine, iterate over a collection and call that subroutine, contemplate the performance characteristics in a discussion (not bigO complexity, more so consider the I/O device..caching..)

Well, a python return type doesn’t really mean anything except to a type checker.

That was my first thought as well. If someone asked me that during an interview, I might assume the type annotation for the return type was something meant to throw me off.

The value being returned in the correct type is especially important in languages without static type checking. The next line may try to add the dictionary you returned to an integer or something crazy.

Run mypy on it and see if it complains. If it does I think it will even tell you what type is being emitted instead.

The question wasn’t what mypy will do. The question was what type will the function return.

mypy error: function 'x' returns type 'y' expected type 'z'

I don't recall the exact message but I think it will emit something along those lines.

The question wasn’t what mypy will do. The question was what type will the function return.

One common question of this type (I've had it, and heard a few others as well) is: implement a function returning the factorial of its input (so n!). Basically FizzBuzz.

I recently got that question after a homework with about 200 lines of python written. If I had failed at this question, the company would have concluded that I had somebody else do the homework for me, so I understood where they came from with this.

Out of curiosity: this test you gave your interviewees, was it done on a computer or pen & paper?

Computer. Web based, copy/paste enabled, no cheat tracking. So in short we let them use an IDE, cheat and copy/paste and they still flunk it.

Heh. I'll never forget the first time I started an interview process with an automated online test with a "cheat tracker". What it ended up really doing was flunking me for alt tabbing to documentation...

Sorry, I am laughing out loud at this response. I love how you wrote "cheat". Like! I know... not my best HN post. :)

I don't understand your comment, what was funny, I don't see them writing cheat unusually?

Hopefully, cheat isn't a normal state :-)

not complaining about fizzbuzz at all, but this is just a PSA to tell the interviewer to end the interview if the candidate gets a good solution in 5 minutes. Adding this "but can you tweak it moooooore" mess is just awkward and the candidate hasn't seen two dozen iterations from all the other candidates.

My pet analogy for this scenario is: Go to all of your employees. One by one ask them to solve a programming challenge for you. Let them know if they don't produce a suitable result (to your liking) in the next 40 minutes, you will fire them immediately. See how they do.

I don't think its wrong to do these types of interviews. But interview stress is real, and it affects people in strange ways. I bombed one of these interviews once and the interviewer was convinced I was "not a front-end programmer". I had just left a job where I had written literally hundreds of thousands of lines of (modern) front-end code, daily, for the last several years. More complex than what I'd be writing at that company (I was eventually hired in spite of this particular interview). And oddest of all, that interviewer was (very soon after) let go. Such is the world of programming interviews.

So yeah. I don't think its necessarily the worst. It might even be the best way to interview candidates. But we all need to take our asessemtns with a huge grain of salt and stop being so judgemental.

Your analogy doesn't really seem too similar to interviews. There will be more stress about getting fired and losing your income than potentially failing an interview which has no effect on your current income, since you still have your current job while interviewing, presumably.

That would be the case for all new grads (substantial portion of applicants), first time applicants (who may be working some other, terrible job), poeple who were laid off / fired, and people who have taken time off. Its not meant to be a perfect analogy, its mean to nudge people into the realization of how stressful interviewing can be. Especially if it is for a company you really want to work for.

You're missing out on loss aversion. Losing something you already have is far more stressful than not getting something that you don't.

Its not meant to be a perfect analogy but to ellucidate why it may be more stressful than you might imagine. But in the spirit of good discussion: Lets compare someone who has 7 years of experience at Facebook, 500k in the bank, and strong interviewing skills with an inbox full of recruiter spam to someone interviewing for Facebook, who has no other interviews, and is currently poor and in debt. Are you so sure the person at facebook would be more stressed than the person applying?

In our case, and this was 20 years ago, it was painfully obvious immediately whether the person knew anything about coding at all; we bent over backwards to do anything that would help (they could pick any language, we'd modify requirements, full internet access, etc) and a good percentage would just flounder.