I've read a few of this guys posts now and have consistently been rubbed the wrong way by them. I think I know why now. It's not that he's wrong. His analysis is reasonable and straightforward. I think it's that the basis for his analysis is ultimately a form of nihilism, coming from someone who (maybe?) used to be an idealist but was burnt by a bad experience and must now explain why believing in anything is misguided.
My instinct after reading this article is to pull back a bit and ask some larger questions. Why is it necessary for big tech companies to act this way? Why does bad code bother engineers so much? Are they actually misguided for feeling like bad code is a catastrophe, or is it really the fault of the broader economic sphere we all inhabit? Is it actually maturity to reconcile ourselves to drift powerlessly as faceless and titanic forces sculpt our reality? So many possible questions.
Because I’m being held accountable for the bad results of the bad code. Because I’m being held to task to fix the problems caused by the bad code on a schedule that I didn’t agree with. Because management is using someone else’s bad code to deny me a positive annual review.
“You’re a senior engineer - why did fixing this take so long?” Because of the garbage I had to wade through to even find the problem. Because I wasn’t yet working here when this was written and I have to take time to get my head around the insanity.
Yes, these are management problems. I’ve spent years managing managers and attempting to educate them on how bad code causes errors and delays. But for many reasons, it’s a Sysiphean task.
I don't think it's that. It bothers me a lot too, and not because anyone else is judging me or anything. I think it's just that it's depressing... it sucks to be doing bad work, on top of other bad work, and unable to do good work instead. It is incredibly frustrating to care about quality but be surrounded by and constrained to work on crap. Just feels like everything went horribly wrong somewhere and you're powerless to do anything about it and your only option is to suck it up.
I know how to fix this but I'm not "allowed to" can eat away at you easily. Then there are things that I know how I might fix but I wouldn't realistically be able to because it's all a lot to take on, there will almost never be enough time and hands to get it done, within the set constraints.
This is due to the way how incentives are aligned. Systems that are powering things for, say a decade at least but worked on Quarterly basis.
Why is this alive and well, then? Because it doesn't actually matter as long as money keeps rolling in. It is also possible that the losses caused by or efficiency not achieved do not show up in the accounts.
>I know how to fix this but I'm not "allowed to" can eat away at you easily.
This really is the worst, and that's why I left my first job. Funnily enough, I just took that job back after a few years but I am now the lead and sole developer on it, I'm having the time of my life doing what I've always wanted to do back then, and seeing the product now flourish.
The bad code didn't really matter, it was the fact that I was not allowed to improve it and forced to build new features on top of crappy code that made me quit in the first place.
> “You’re a senior engineer - why did fixing this take so long?”
exactly!
If a senior engineer is responsible for the problem, but he wasn't given room to develop a proper solution in the first place, then he cannot be blamed for the outcome. And yet, this is exactly what happens.
If a system had been made well, and functions well, same senior engineer does not get the credit for not causing problems!
This is what happens when management/stakeholders don't understand what is going on, but have an expectation that is not well communicated to the engineers (and also inversely too, where engineers are not communicating the situation properly to management - or they refuse to hear it/ignore it).
> “You’re a senior engineer - why did fixing this take so long?” Because of the garbage I had to wade through to even find the problem. Because I wasn’t yet working here when this was written and I have to take time to get my head around the insanity.
The arguably unproductive part of my brain craves to reach a day where I can say literally everything straight - if this would be a true description of the circumstances, then say exactly that.
Unfortunately, in addition to not only not being a “team player” this would also be really mean to people that had different time and resource pressures when they wrote the original flawed implementation. So both burning bridges and being kind of a dick.
At the same time, there’s sometimes people that develop stuff in such a way that I will never agree with them and a few people like that are enough for me not to want at a particular team/org even if it’s maybe like just 5 to 10% (and even then often it’s about them being inherently mean/nitpicky/whatever, rather than purely technical stuff).
Obviously the irony of the above isn’t lost on me so I just bite my tongue, focus on being professional and fixing the things I can and let the rest be without picking fights that will lead nowhere - otherwise I’d be a hypocrite and the situation couldn’t be described as anything other than “a bad culture fit” from an outside perspective.
But I’ve seen a lot of really shit code and it baffles me that alarm bells don’t go off more often and people just trudge along.
Then again, you do see a lot of drama in plenty of open source projects, so maybe that’s just human nature.
I agree it's often difficult, but I think it's possible to balance honesty with humility and being a so-called team player.
For example, if you see something that's sloppy [1], your honest assessment will likely help both the coder and the manager. But you do have to temper your language, and sandwich the criticism in something that won't make the person interpret it as "you're sloppy/stupid" or "you don't deserve to work here".
The additional problem is: you have to test it with baby steps. If a coder/manager can't take any feedback [2], not even the friendliest suggestion, you have to pick whether to (1) slowly train each other to communicate better, or (2) leave (it alone).
But ultimately (IMO), it's not worth training yourself to be less honest or dishonest. You risk losing the ability to do your best among other smart, honest and feedback-accepting people, or being an honest and feedback-accepting manager/employer yourself [3].
[1] Like many blocks of copy-pasted code, or massive scripts with no separation of concerns, which can really grind my gears.
[2] I've personally had a few discussions with peers and leads in teams who would get defensive, or interpret my feedback as politicking, even in a cooperative environment, and I didn't love that. But I also didn't love getting (what seemed as) "holier than thou" evals in (what I saw as) more competitive environments. So, maybe my cooperative is another's competitive, and vice versa.
[3] Because you can easily get used to a skewed or toxic environment, and accidentally spread that toxicity to others, or unintentionally be seen as toxic yourself.
Hm, those are all valid, but they're also from the perspective of only caring about external forces. It's as if the work itself is only relevant insofar as we get something out of it.
I got into this career because I like mucking about with this kind of work. Programming is sort of like building thought sculptures. Ok, maintaining them too. I spend a lot of my adult life working with these systems, so of course I care about their quality.
It is unpleasant to work on a hack-filled creaking monstrosity of a codebase. My skills, such as they are, don't help as much as it feels they should, because I rely heavily on understanding a system based on how it sort of has to work given what I know about the constraints, and hacky systems constantly confound my expectations. That means I have to spend way too much time and effort learning the idiosyncrasies and unpredictable corner cases, and so have less time to make the changes I want to make. It's like a carpenter working on a house where nothing is straight, nothing is level. Or a cook given flavorless or half-rotten ingredients to use. You spend more time picking out the bad (or picking out the good and discarding the rest) than actually cooking.
Any my scope is limited. Some changes are just not feasible to make, because there are too many hidden and/or unnecessary dependencies buried in the code. If you change one thing, everything falls apart. If you patch it together well enough to more or less work, you're faced with a long tail of subtle bugs that you'll be battling for a long time to come. There's no conceptual integrity.
Furthermore, working with a team that accepts a never-ending accumulation of creeping technical debt is also unpleasant. You can't trust that a positive review means anything. If you try to do something right and take pride in your work, your teammates will be resentful that you're not closing your bugs fast enough. Who cares that it means less maintenance overhead in the future if nobody's planning on sticking around, or people are depending on things being constantly broken so that they can spend their days knocking off relatively easy whack-a-mole style bugs?
Bad code is actually fine. It happens. But I want to know why the code is bad, whether the reasons are defensible or not. If they aren't, I know that the codebase is going to be littered with similar crap, and that my job is going to suck. That's what bothers me so much. This is true even if I'm the author of the bad code. Why didn't the reviewer point it out? Is it like the article said, and there's nobody competent enough to spot glaring issues? That is also a component of a job that sucks.
> Hm, those are all valid, but they're also from the perspective of only caring about external forces. It's as if the work itself is only relevant insofar as we get something out of it.
From the perspective of the organization that pays you to do it, it is? At best there may be another mission that it genuinely cares about, usually there’s only a profit motive (which is also fine). If you want to create software as an end in itself or enjoy the craft without compromise, it usually can’t be in the context of a job or a business
Our job is to produce the best possible outcomes given the constraints we’re faced with, and inform leadership so that they’re aware of the tradeoffs when they make their decisions. Sometimes those decisions are going to be bad, and obviously it’s justified to be frustrated then. Other times they are correct, even when it means compromises on the engineering side. That’s when we have to just suck it up (or go elsewhere)
I still think there’s room for enjoying the work of creating software even under imperfect conditions. Striving for perfection is for hobbies, or the very rare circumstance when it’s justified by the goals of the organization
I'm currently leading a team in a group of teams responsible for a terrible code base. It's riddled with tech debt, our DB is maintained by others and has absolutely atrocious performance, it often feels like I'm the only one who cares about all our tech debt, typescript errors, linting errors and browser warnings, and I worry that the reason I'm the only one is that I'm the most recent hire. I still care.
I'm now spending my weekend thinking about a problem where a really urgent feature I wrote doesn't work because the query behind it is too heavy for the database (the query isn't anything special, but it's against a view that clearly isn't properly indexed), and the DB people are complaining I'm overloading the DB. But it's going to be shown to the regulator for some important regulatory rule for which noncompliance is costing us tens of millions per year. Next Tuesday. Which they told me last Tuesday.
How am I supposed to write good code in that kind of environment?
> Hm, those are all valid, but they're also from the perspective of only caring about external forces. It's as if the work itself is only relevant insofar as we get something out of it.
Well, I don't know that we can separate external forces from why people might dislike bad code... being held to a standard that is unachievable because of someone else's mess and someone else's expectations is all too common in life, and the reality is that your livelihood might depend on meeting those expectations. If you like programming, you might also like maintaining or refactoring code; it goes with the territory, and it can be fun to see how something can be improved or challenge yourself to see how far you can improve it. But it tends to be less enjoyable when your ability to eat and house yourself rides on someone else's bad code.
> If you try to do something right and take pride in your work, your teammates will be resentful that you're not closing your bugs fast enough.
Right, but why would they resent that? Probably because management has expectations on the team, and they are concerned that you're sandbagging the metrics. It ends up being a management or cultural problem regardless.
> Bad code is actually fine. It happens. But I want to know why the code is bad, whether the reasons are defensible or not.
sandbagging or messing with metrics is an inevitable part of playing a role in $BIGTECHCO. Even if people don’t admit it or aren’t aware of it, it happens everywhere. If anyone resents it they just resent you’re doing it better than they could. much of what we do is all a political game. I’ve had situations where I cannot do something that clearly needs to be done, I in the past will find reasons to slow things down and point to (thing that needs to be done) as the reason why. Eventually the manager above you will prioritize it or discipline you. If you’re right and are valuable and your manager isn’t stupid, you’ll get your way, and they can take credit for the sudden velocity gain. I’ve also done the opposite in speeding things up to expose brittle unsustainable processes.
Not endorsing it, but, it just seems to me the way most performance management works at these companies (I came from a company that liked to think it was amazon with stack ranking) you have to do it or you will get shanked inevitably before you vest. This is just my personal experience. TLDR it’s how you make things “happen”
It’s very cynical. Thankfully these days I work on much smaller teams where everyone helps as much as they can because you’re constantly treading water, but there’s very little room for this kind of political maneuvering and bs.
Others have said some true things but the core is missed. It bothers engineers who care about mastery. It’s the same for architects who see design flaws in buildings that were done by those who don’t care about mastery. Or a filmmaker who watches a movie with lots of flaws. Caring about obtaining mastery means having the capability of seeing the flaws and resolving them.
It could be maturity to resign to powerless forces or it could be the inability to obtain that mastery or the willingness to let go of that mastery. I would think it’s more mature to fight the good fight, but at some point you just get old and tired I imagine. And it’s interesting to posit the author is nihilistic when this “uncontrollable forces” itself is a nihilistic take imho.
I don't think that's entirely true. Seeking mastery does not imply being a master.
If you have only ever seen one pattern to solve a problem, trivial example of inheritance, and therefore do that to the best of your ability then you have achieved mastery to your ability. Once you see a different pattern, composition, you can then master that and master identifying when which is suitable.
Lack of mastery is just using inheritance despite seeing alternative patterns.
Naturally mastery also includes seeking alternative solutions but just because a codebase uses inferior patterns does not mean those that came before did not strive towards mastery, it's possible that they didn't know better at the time and now cannot get the time to revise the work.
There's always a juggling act in the real world.
Assume incompetence and not malice, and incompetence is not a state of being. A person without experience can be seen as incompetent but quickly become competent with training or experience, but the code they write still stems from incompetence.
Strive to see your previous self as incompetent (learn something new every day)
7 out of 4 million active SWEs sounds about right :) how many codebases have you personally seen and went "holy shit, this is masterful/flawless/..."? even just libraries as codebases is most definitely 0...? our industry is basically few solid competent people followed by an ocean of mediocrity and incompetence... always been that way and always will be the way
I don't think it's possible to write a flawless codebase. That doesn't mean SWEs don't seek mastery of their craft. Moreover, achieving mastery doesn't mean you would actually want to write an 'ideal codebase', that seems like an art project disconnected from the purpose of the craft.
“mastery of the craft” is a myth… whatever codebase you see (and I’ve seen more than I can count) you will go “wtf is this?!” over and over again. furthermore, whatever codebase someone was lucky enough to initially start with, after X amount of time you asked those same masters (if they are still around) and they will inevitably tell you “oh boy, if I can start this over, I would have…” there just is no mastery, there are just people that deeply care about what they are building and try their best not to F it up along the way but all the “craft” and “mastery” talk is just empty words people talk around fireplaces and/or at various conferences where people pitching “craft” and “mastery” and “you need [insert dev process du jour XP/agile/…]” are there to make money selling the bullshit
My feeling when I found this blog was "so I'm NOT the only one!". It's the painful truth about staff+ engineering that I've also experienced, but haven't felt safe to talk about.
You're not wrong that there's something cynical or nihilistic about it. The core thesis is "do what the company wants, even if it's not what they should want". That idea may be unpalatable, but getting ground up in the corporate gears is worse.
My personal feeling is that there's a way to both agree with the underlying issues Sean writes about while being more optimistic and providing better alternatives. Something I feel I should start writing about more.
I think one of the issues is that engineers define bad code on a different set of dimensions than the business, and even amongst each other. Early in my career I was definitely guilty of this, that if the code didn’t fit my definition of perfect code, it must be bad. I’ve met great programmers (I don’t consider myself in this group) in my career who put even more strict criteria on something to be considered ‘not bad’. Then I worked at small companies, big companies, was an owner at one, and my definition of bad code narrowed. While my criteria are still subjective, if the code meets the business goals and a baseline of quality I consider it fine. Because outside of some absolute genius programmers like Carmack, few of us will look at someone else’s code and not think of any type of improvement.
I've come to quite a specific definition of "bad code", at least the main form of bad code that infests corporate environments. It's overly complicated code, but people disagree on what that means too, and I have a concrete definition of that now.
Bad, over-complicated code is code that requires you to form an unnecessarily detailed mental model of it in order to answer a particular question you may have about it.
People rarely read code in its entirety, like a novel. There is almost always a specific question they want to answer. It might be "how will it behave in this use case?", "how will this change affect its behaviour?" or "what change should I make it to achieve this new behaviour?". Alternatively, it might be something more high level, but still specific, like "how does this fit together?" (i.e. there's a desire to understand the overall organisational principles of the code, rather than a specific detail).
Bad code typically:
* Requires you to read and understand large volumes of what should be irrelevant code in order to answer your question, often across multiple codebases.
* Requires you to do difficult detective work in order to identify what code needs to be read and understood to answer the question with confidence.
* Only provides an answer to your question with caveats/assumptions about human behaviour, such as "well unless someone has done X somewhere, but I doubt anyone would do that and would have to read the entire codebase to be sure".
Of course, this doesn't yield some number as to how "complicated the code is" and it depends on the question you're asking. A codebase might be quite good on some questions and bad on others. It can be a very useful exercise to think about what questions people are likely to seek answers for from a given codebase.
When you think about things this way, you come to realise a lot of supposedly good practices actually pessimise code in this sense, often for the sake of "reusability" of things that will never be reused. Dependency injection containers are a bête noire of mine for this reason. There's nothing wrong with dependency injection itself (giving things their dependencies rather than having them create them), but DI containers tend to end up being dependency obfuscators, and the worst ones import a huge amount of quirky, often poorly-documented behaviour into your system. They are probably the single biggest cause of having to spend an entire afternoon trawling through code, often including that of the blasted container itself (and runtime config!), to answer what should be a very simple and quick question about a corporate codebase.
It's hard to write about the broader context with any expertise in a blog post written from personal experience.
My own thinking is that the big competitive advantage that big tech firms have over small ones is the power to mobilize very large numbers of developers onto a project.
Large projects that don't depend on a small core group of irreplaceable competent individuals are more repeatable for a business. So it makes sense to focus on making the repeatable processes more likely to succeed than simply hope that you happen to have the right team assembled for the job.
Assuming that any of your engineers are above or below average hurts the ability for the business to plan.
True but this isn’t something big tech innovated. This is exactly how most large organizations work. For example, and specifically in the military, servicemembers are fungible and they are rotated every 2 years. The military even encourages attrition at certain ranks and points in people’s career.
I think you are ascribing too much into this. If he does not believe anything then there will be nothing to write about. It looks like he does not believe in _your_ ideals - that does not make it nihilism though.
> Why is it necessary for big tech companies to act this way?
This question gets directly at the cause of the author's nihilism: the necessity is borne from the endless pursuit of positive quarterly growth, the "binding fiduciary duty to shareholders". Which is a lie, there is no such legally binding fiduciary duty. So the aforementioned necessity is also a lie. Companies could operate on a longer time horizon, let engineers write better code, make better products, maybe even consider societal good in their strategic planning, and still turn a healthy profit. But the cost of perhaps taking a few degrees off their YoY trend line is unacceptable to the insatiable greed of their controlling shareholders.
Interesting. Coming from big tech, it's actually pretty spot-on of an article. I think most folks at big tech have experienced this stuff too.
> Why is it necessary for big tech companies to act this way? Why does bad code bother engineers so much? Are they actually misguided for feeling like bad code is a catastrophe, or is it really the fault of the broader economic sphere we all inhabit? Is it actually maturity to reconcile ourselves to drift powerlessly as faceless and titanic forces sculpt our reality? So many possible questions.
These are all great questions. I have some thoughts, of course. But I'm not sure it's fair to describe OP as a burnt-out nihilist. The premise of the post is pretty reasonable actually.
I don't disagree that it's an accurate portrayal of reality. And of course, as I made clear in my post, I don't know if they're actually a burnt out nihilist.
My issue with the article is maybe more that the perspective and framing seem to want to lock out idealism from the conversation and encourage people to become ruthless, hard-nosed pragmatists in order to survive in this environment. Is that actually good, effective, or desirable? Again, lots of questions.
I worked on compilers and tools starting in 1981. Proprietary compiler technology has disappeared over the decades and the development tools/language/process/compute have changed considerably. But the skills and role of a compiler developer seem similar, although maybe it just means this sub-field of software has matured.
That's one of the many many reasons I miss my dad so much. He went straight from MIT to GenRad circa 1970. I didn't start doing software till 1998, but we had some years of overlap -- and it was profoundly gratifying to connect w/ him on engineering topics. The learning was even bidirectional tho I'm sure I got more out of our pseudo-professional interactions than he did. Sigh. I really miss him.
I think the engineer jumping between giant companies for three years or less every time rarely works on particularly key things. Big tech companies do a LOT of stuff, and most of it is crap that isn't moving the needle. This post describes teams that are constantly changing priorities (chasing trends?) and IME that's not true of the really core, central functions at companies. But very true of the "support/enabling" or "what else can we do?!" side functions.
For instance, Github Actions being a meh product is called out in the article - that's a classic "check the box" feature that's good enough for a lot of people (let's not forget that Jenkins was no picnic before it) but is never gonna massively increase GH's bottom line.
Those sorts of projects are easy places for politics to fester since they are easy to ignore for the most influential-and usually strongest-parts of leadership.
On the other hand, if you're on a core, mission-critical team and other people's code is turning into your bad performance review, you need to figure out if the problem is (a) bad/toxic manager or (b) a failure to keep your management chain informed at what the root issues are and how you can improve it.
> Why does bad code bother engineers so much? Are they actually misguided for feeling like bad code is a catastrophe, or is it really the fault of the broader economic sphere we all inhabit? Is it actually maturity to reconcile ourselves to drift powerlessly as faceless and titanic forces sculpt our reality? So many possible questions.
Nihilism is a defense mechanism when everything is moving against your world view.
STEM people of all walks of life join because of the challenge, the loveliness of an elegant solution, and the “art” you create that leaves your mark. Good engineers view their code as a makers mark. This code represents me as my art and I should do my best. Unfortunately (or fortunately) this is beaten out of you by senior engineer and the shell left is a nihilist and misanthrope.
Corporate programming strips you of your creativity, your autonomy, and your drive. It’s simultaneously strict, but too loose where it should matter. It’s spastic in its execution. There is often little rhyme or reason aside from “build fast make money”. No one appreciates your contributions. You show up to meetings where sales and PMs simultaneously wrestle to take the credit you rightfully deserve. You made it 10 years, here’s another low budget pizza party while the sales team gets an all inclusive in Ibiza. You wanted to make cool, elegant, things, and instead you’re just a factory worker that had to do 7 rounds of interview for the privilege of stacking premade widgets together until retirement.
The end result as I and many engineers at and over the one decade mark have realize. Nothing matters, no one respects you, and no one truly cares. You get paid and promoted whether or not your code is elegant, or safe. You get recognized by prostrating yourself in front of leadership. So why should I give a shit if my code works or not. It passes tests and I get to eat.
The industry is pathetic and we should stop calling ourselves “engineers”. Modern corporate programming is analogous to working on a widget assembly line in the limit. Most of us simply find our joy elsewhere and have learned “hearts and minds” as the rule of corporate life.
Every time I read his article I regret it. I literally mean every time, 100% of it. Judging by the title of the article, I didn't expect his reason to be "engineers working outside their area of expertise". I've seen good engineers figure out problems outside of their expertise plenty of times, so that's not a good reason either.
I feel like this article is the equivalent 16 paragraph stating you're likely to be correct only 10% of the time when you guess a random number from 1 to 10
My instinct after reading this article is to pull back a bit and ask some larger questions. Why is it necessary for big tech companies to act this way? Why does bad code bother engineers so much? Are they actually misguided for feeling like bad code is a catastrophe, or is it really the fault of the broader economic sphere we all inhabit? Is it actually maturity to reconcile ourselves to drift powerlessly as faceless and titanic forces sculpt our reality? So many possible questions.