It surprised me just as much. I cannot speak to the capabilities or thoughts of any given candidate. Ultimately the test was a test of using a single standard API that is based upon a standard model. If it were just a matter of instructions or syntax I suspect a candidate would gotten over that by either asking the right questions, trial and error, using a reference of whatever. I think what really destroyed people was a complete inability to perceive the page as a series of nodes all interconnected with various relationships.
> In my experience, if you give the candidate only one hour, access to documentation only helps with recalling past experience
That is only partially true in this case, because the candidates knew coming in they would have a code filter accessing a page using vanilla JavaScript. I cannot remember if I specifically mentioned refreshing on the DOM methods or not. I might have. This was 8 years ago. But they had at least a day prior notice to catch up on how to interact with a web page using JavaScript using only their code. The time pressure would have applied to the specific tasks asked of the candidates.
I think what really destroyed people was a complete inability to perceive the page as a series of nodes all interconnected with various relationships.
Fair enough. :) I've experienced a similar issue with interview questions where they often floundered on the question because they hadn't ever spent time digging under the surface. For example, I notice that a lot of (typically junior) developers seem to only think of JSON as a representation for hierarchical objects but never really spent time thinking about the typed-ness of its keys/values or the limits of what JSON can actually represent. An interview is a bad time to be confronted with that realization.
I cannot remember if I specifically mentioned refreshing on the DOM methods or not.
Well, it was a long time ago, so I'll give you the benefit of the doubt and just talk about what I've experienced. I go back and forth about how specific I should be about the type of questions I might ask. On the one hand, it seems kind of foundational that someone writing web frontends is aware of the DOM and how it works, even if they never directly interact with it. On the other hand, I think questions need to control as much as possible for the "preparation" factor because some people simply interview better or have more time to study.
It doesn't seem too crazy to me today that a competent FE dev these days might have spent their whole career in React, and therefore they don't really have a strong grasp on the DOM. If you had asked when I first started doing web FE development, that would have been crazy! But frameworks got a lot better and a lot more comprehensive since then.
I recently had an interesting experience asking a question involving dealing with UTF-8 encoding. I figured "everyone knows about UTF-8 right?" I retired the question pretty quickly once I realized that bit manipulation is not actually something you have to be even remotely familiar with in order to be a functional developer today in the vast majority of programming tasks, and therefore it wasn't even a good fizzbuzz-type test.
Yeah, I have an in-depth knowledge of the imperative DOM APIs, but I can't remember the last time I actually needed to use them in my day job. React provides a very good abstraction that one doesn't need to break out of very often for a lot of work.
It surprised me just as much. I cannot speak to the capabilities or thoughts of any given candidate. Ultimately the test was a test of using a single standard API that is based upon a standard model. If it were just a matter of instructions or syntax I suspect a candidate would gotten over that by either asking the right questions, trial and error, using a reference of whatever. I think what really destroyed people was a complete inability to perceive the page as a series of nodes all interconnected with various relationships.
> In my experience, if you give the candidate only one hour, access to documentation only helps with recalling past experience
That is only partially true in this case, because the candidates knew coming in they would have a code filter accessing a page using vanilla JavaScript. I cannot remember if I specifically mentioned refreshing on the DOM methods or not. I might have. This was 8 years ago. But they had at least a day prior notice to catch up on how to interact with a web page using JavaScript using only their code. The time pressure would have applied to the specific tasks asked of the candidates.