If you didn't clear the platform code challenge, leveling didn't come into it.
We do respond to responses we get after rejecting people, but it can take us a few days; we tend to do batches of them all at once. My take is: while people are still in consideration, we have an obligation (we don't always meet) to be responsive, because people are pending their career decisions on our own decisions. Once we've communicated that decision, we're less obligated to be zippy, because you have the high-order bit of what you need from us.
I don't know anything about your submission and deliberately didn't try to look it up, because our grading of your work sample is nobody's business here, and I don't want to create the impression of communicating anything about it.
Having said that, I can tell you what we tell everyone who asks why their code sample didn't pass:
We've made a decision to budget the amount of time we ask from candidates, end to end, from application to final decision, to the amount of time they would spend in a hiring loop at a typical company that just did interviews and resume screens.
That implies that we have to keep our coding challenges short and succinct. I can't ask you to spend days coding something; that's more than an interview would ask of you.
A consequence of a short coding sample is that there's less surface area for us to derive signal from. We have to be pretty greedy about selecting coding decisions to judge, because there isn't all that much code to work from (for anybody who doesn't know, our current platform code challenge is to write a TCP proxy).
A consequence of having to jealously collect signal from coding challenges is that we're limited in what we can tell you about our grading. We can't offer you the rubric we use (there is one; reviews are mostly blind, and done by a pool of engineers), because that deprives us signal. We'd learn how good you are at following directions, but not much about how you approach systems programming problems.
I'd want a scoring rubric (and fast responses to emails!) too. I see where you're coming from. All I can do is tell you where we're at.
> while people are still in consideration, we have an obligation (we don't always meet) to be responsive, because people are pending their career decisions on our own decisions.
I had been receiving very quick responses but haven't heard back in a bit over a week.
Last email said I was moving to the final round for the platform position. It said you'd follow with a couple more emails with further instructions but haven't received them. Sent a couple of pings over email.
Reaching out over HN in case this is a bug on your hiring tools or my email setup. I understand that sometimes things get busy behind the scenes. Just need an ACK to know if I'm still under consideration and make better informed career decisions.
"If you didn't clear the platform code challenge, leveling didn't come into it."
How does that make sense? I was told that you got a ton of applications and that I didn't meet the cutoff for the pool. Surely you can't just keep the top X (open positions + margin) and try to place them into appropriate levels if they make it through the remaining rounds? You very likely won't have an appropriate mapping of skills to your open roles and candidates will be left dissatisfied, right?
Isn't a hiring challenge supposed to be a more objective assessment of engineering skill? If there's less "there's less surface area for us to derive signal from" then maybe it's not better than the traditional model. To be told that I didn't do well enough to be considered for a junior position sucks because I'm a mid level engineer.
I'm certain that my proxy worked well. I was able to push ~7 GB/s through it spread across 1000 connections on my local machine. It can do that with reasonable resource utilization, good latency, no race conditions, and no errors. I also wouldn't say my notes were skimpy. I could have written more but I wanted to be respectful of the reviewers time.
In fact I would rather you openly shame my submission here because at least then I would have the oppourtinity to reflect and improve. I hope this level of criticism isn't too much. I just wish that software engineering interviews could be more rational.
That coding challenge just isn't where we make leveling decisions. The difference between an L1 and an L3 at Fly.io is mostly not about how you write proxy code, but rather how you manage scope, how you think about what customers need, and how you design systems.
We didn't tell you you weren't good enough for a junior position. That's not what our actual email response was to you, and it's not what I said in my previous comment. The single exercise you did with us doesn't currently have a leveling component, so there's nothing I can tell you about what level you'd have wound up at had you passed it.
I understand that you're giving me permission to relate more about how we scored your submission, but, again, I'm unwilling to do that, for all the reasons I gave upthread.
Our whole hiring process does level people, but the one challenge you did does not. It's pass/fail.
So: no, there is no such implication in being rejected at that point. I'm saying that explicitly right now, so there needn't be any uncertainty going forward.
I think it's also pretty important to understand that different companies have different leveling functions. Not being L3 at Fly.io in 2023 doesn't say much about what level you'd be anywhere else.
We do respond to responses we get after rejecting people, but it can take us a few days; we tend to do batches of them all at once. My take is: while people are still in consideration, we have an obligation (we don't always meet) to be responsive, because people are pending their career decisions on our own decisions. Once we've communicated that decision, we're less obligated to be zippy, because you have the high-order bit of what you need from us.
I don't know anything about your submission and deliberately didn't try to look it up, because our grading of your work sample is nobody's business here, and I don't want to create the impression of communicating anything about it.
Having said that, I can tell you what we tell everyone who asks why their code sample didn't pass:
We've made a decision to budget the amount of time we ask from candidates, end to end, from application to final decision, to the amount of time they would spend in a hiring loop at a typical company that just did interviews and resume screens.
That implies that we have to keep our coding challenges short and succinct. I can't ask you to spend days coding something; that's more than an interview would ask of you.
A consequence of a short coding sample is that there's less surface area for us to derive signal from. We have to be pretty greedy about selecting coding decisions to judge, because there isn't all that much code to work from (for anybody who doesn't know, our current platform code challenge is to write a TCP proxy).
A consequence of having to jealously collect signal from coding challenges is that we're limited in what we can tell you about our grading. We can't offer you the rubric we use (there is one; reviews are mostly blind, and done by a pool of engineers), because that deprives us signal. We'd learn how good you are at following directions, but not much about how you approach systems programming problems.
I'd want a scoring rubric (and fast responses to emails!) too. I see where you're coming from. All I can do is tell you where we're at.