One thing I am yet to see is a critical analysis of over-engineering. I constantly see articles like this that talk about how those young developers are looking for the newest and shiniest technology and I think that's just a lazy answer to the causes of over engineering. I for instance have tried my best to run away specifically from the new shiniest things like Kafka, webpack, mongo. But this is obviously a flawed strategy as those technogolies my help solve my problems. One might say careful cost/benefit analysis is the answer but it's most times easy for me to point out how much more beneficial the tech I am introducing is. And it could be but it would still be over engineering. Plus it's not really solving the real problem that is adding more incidental complexity to the solution than needed. It would interesting to see an article on causes of over-engineering from real examples so one could extract potential solutions.
I have witnessed other people do it at multiple companies and even been guilty of it myself once or twice.
I sat at the end of a long board room table full of senior engineers and directors and was the only one who didn't bring a laundry list of techs he was waiting to introduce.
I have seen older developers treat design patterns the same way that younger developers treat JS frameworks. Everybody wants to believe they have a toolbox full of solutions to problems. Maybe for the last generation it was Gang of Four, and for this one it's the HN front page and blogs.
A large part of it is because for most programmers, it is the only way to demonstrate growth in their own skills.
Relatively few developers get to lead projects, design truly interesting systems and so on. Most developers, especially in bespoke business software, end up writing large amounts of fairly un-challenging code day in day out.
But then if you just churn out the same sort of business logic for years on end, with little access to problems that are inherently challenging, how do you feel any advancement in your craft? For many developers it is through over-engineering or pointless re-engineering.
Common signs: tasks that are actually trivial suddenly start using home-grown frameworks. Obsession with vague rarely defined terms like "best practices", "clean code", "modularity" etc. An insistence that practices which were once considered an advance were actually a regression, like the recent trend to claiming that exceptions are bad because they make control flow "hard to understand", which justifies rewriting everything to use return codes, or XML is bad because everyone claims it is, but JSON with schemas is totally not the same and definitely how architectures are done Right™.
To get a perfectly simple design and implementation that is nonetheless powerful you really need a perfect match between the time/capabilities of the engineer and the inherent difficulty of the problem. That way the developer's mental energies will all be spent designing something that will work, and relatively little is left over for inventing new config file formats.
The funny thing is I actually faced this problem a lot last year. Since I read paul grahams essay on hackers I decided to focus that will for hard problems into my side projects(although this has taken my focus from popular frameworks to specialized libraries)
Nerds are raised on sci-fi. I was. They keep waiting for the future to happen. Scouring for "new technologies" is sometimes called "scouting." It's really important for a company to appoint "scounters." Then everyone else knows "scouting" is not in their job descriptions. Then they start to focus. Many organizations never graduate from future fetishism.
