"we found that differences in architectural complexity could account for 50% drops in productivity, three-fold increases in defect density, and order-of-magnitude increases in staff turnover."
I think I can speak for many of us technical professionals when I say, been there, done that.
This is exactly why I think the "myth of the 10x engineer" is so obviously false.
An average software engineer will create an overly complex system.
If a skilled engineer can come in and create something that doubles team productivy, decreases bugs by 60% and improves retention across the org by 10x, that's huge! That's not a 10x engineer, that's a 100x engineer.
The institutionalized brain damage is real, I see bright new hires coming in, within 3 months they are towing all the lines, spouting tautologies, seeking alignment with corporate velocity vectors to gain momentum, generate impulse. Talking in non-specific platitudes that could literally mean anything and thus mean nothing.
The biggest problem with a simple architecture and often why someone would complexify it up, is as a protection mechanism so you don't get some performance review motivated yokel doing a drive by to add in a feature and look like a rock star because it was so damn easy. NSS, that was the next step. You see this in open-source as well. Someone builds a beautiful foundation, and Steve rolls up with a submarine pull request to slam that cherry on top.
A complex architecture ensures that you can adequately gate keep.
The next aspect is you have to relentlessly say no to feature requests that literally destroy the system to add some new feature, the pressure to smash it to bits and make it a no longer simple system is such a perverse incentive that unless you have support from on high, every simple system will decay into a total mess, then the finger-painters will have moved on to some other system to predate on.
FYI, it's "toeing" the line. It's an expression from track and field, where at the start of each race the competitors place the tip ("toe") of their foot on the start line when they're ready to run.
Similar to the expression "Up to scratch." In a boxing ring, there is a line in the ring that boxers must come to to begin the fight. To be "up to scratch" is to be ready and worthy of the fight, and from that, to be of acceptable quality or capability.
"Toe the line" most likely has it's roots in military tradition [1]. To toe the line is to stand at the line for inspection - it's still used that way today (I had to do it myself in boot camp).
Relevant, as in this context, it means being obedient to the hierarchy.
> The next aspect is you have to relentlessly say no to feature requests that literally destroy the system to add some new feature, the pressure to smash it to bits and make it a no longer simple system is such a perverse incentive that unless you have support from on high, every simple system will decay into a total mess, then the finger-painters will have moved on to some other system to predate on.
Would you rather have a simple system that does nothing, or a complex / difficult-to-work with system that solves customer needs? It's easy to go too far in either direction.
> Would you rather have a simple system that does nothing, or a complex / difficult-to-work with system that solves customer needs?
With respect, this is clearly a false dichotomy.
Creating the "simplest possible system" that solves customer needs is the entire point of this discussion. Avoiding as much complexity as possible brings enormous benefits. (Less bugs, less employee turnover, higher productivity, etc.)
Never accept the notion that systems must be "difficult-to-work with" in order to solve customer needs.
It would be a false dichotomy, if I meant to say that you must choose one of these paths. I suppose "it's easy to go too far in either direction" was not clear enough. The argument I'm making is that the OP is creating a false dichotomy: that you must reject feature requests, or ruin the system.
My point is that there are many, many developers who "relentlessly say no to feature requests that literally destroy the system to add some new feature" when the reality is that they do not want to put in the effort required to add the (meaningful) feature to the system.
They are choosing system purity and laziness over delivering customer value, which requires more work in order to keep things simple.
In my experience, most of the software complexity is accidental, i.e. it doesn't directly emerge from customer needs.
And even if your framework is to use customer needs as justification for complexity, people will still use something like Kubernetes for an app with 10 monthly users, because it "helps us deliver features faster, have zero-downtime deployments and be scalable".
Most (non-product) engineers have a natural inclination to introduce more complexity, and will use mental gymnastics to justify it.
I never really had to make this choice. But I often had to choose between a system that is reasonable and actually solves the needs of customers, versus a system that is complex, made in a hurry and full of badly made features that only some project manager cares about. I know which one I prefer.
Just today, for example, I removed several thousand lines of code from a very complex form that no customer ever really used other than for testing. My boss told me "just nuke it". Life goes on.
I don't think it has to be so nefarious. What I've experienced is developers taking the path of least resistance and failing to perform proper analysis to arrive at a less complex architecture.
Why are folks taking the path of least resistance becomes an interesting question.
This is why small organizations produce stuff, then large organizations buy it and sell it as a product.
As an engineer you just have to ensure you have equity in the small company before it's purchased and/or hope the big company buys your company rather than just copying it and fighting you in court for a decade.
Is this because as groups grow in size, high level decision making power tends to move up to less informed management individuals?
You'll end up with a "I programmed a little bit back in the day" person who makes decisions based on how to currently keep their staff best utilized. Tools choices, boundary points, stacks, become chosen based on goals that are not rooted in design simplicity.
I've watched this story play out again and again and again at companies. Upper management will promote these types of individuals into lower/middle management because
a) they're deemed actually reassignable whereas a really talented Ux designer is obviously adding the most value doing that
b) people buried/invested in tech stack/design details are harder for owner/operators to relate to than individuals more like themselves.
Funny… I’d actually always heard it was Newton. I checked and looks like you’re def right here.
I actually expected to find that it was a fake quote. Lately, I keep finding that many of the good quotes are actually fake. It’s like they actually survive on the value of the quote and, if we’re honest, whether someone decades or centuries ago actually said it is only tangential.
Exactly this. I've done well for myself by focusing on building things that work, and shipping them on time. Anything else is extra and not to be done at the expense of the first two things.
Would you build something simpler and more reliable if it took you from shipping a week early to a week late?
I agree that anything else is extra, but most of my difficult professional engineering decisions are tradeoffs between those first two things: I can build on an older system less well adapted to current needs and deliver more quickly, or build something that fits current needs better but takes a bit longer. I sadly don't have enough days in life where there's a better and quicker option!
Put differently, it's a lot faster to put another layer of duct tape on top of the last duct tape patch of a pipe leak. But every layer of duct tape you put on makes it more difficult to replace the pipe segment underneath. Developing judgement on when to do which is the (difficult) trick.
We deliver the duct taped one one time so the software folks can start their work, and then we build the non-duct taped one wheel it in a week later and quietly swap it out for the duct taped one.
Yep, working with well-architected software systems is the difference between a low-stress well-paying job and rapid burnout. No surprise that the latter results in massive turnover.
It's remarkable how "add feature X" can be a 1 hour task or a 1 month task depending on whether the system was designed to evolve and scale in that direction. But the non-technical management or customer just sees it as a simple request, and the (Nth) software developer is left to pick up the pieces.
I suspect this goes both ways. I have worked at a place characterized by frequent internal shuffles, and more than once spent ages reverse-engineering code when the original team could have handled the problem in a fraction of the time.
I think I can speak for many of us technical professionals when I say, been there, done that.