Yes exactly. I recommend reading the book Design Patterns [1] for some ideas on how early "modular" software was conceptualized. This book was published in 1994 and still has a lot of relevance today.
One thing I notice is that software is becoming more brute force and less about "design" or "architecture" quality.
Good abstractions at the correct level and the correct data modelling can make the code around it very simple. Instead of nice designs that are easy to reason about, we have lots of unit tests to "prove" that it works. Instead of making a system reliable and trap errors properly, we use Kubernetes to launch another pods as soon as one goes down. Maybe I am just too old, but I feel we are loosing something going down this route.
(Ok, there is code that's functionality will mean that in inherently isn't simple, but I see far more CRUD apps that are built in an over complicatd manner, than code that is actually doing something algorithmically complex).
Writing perfect bug-free code is nearly impossible and expensive. Hardware is going to fail, network links can get flakey, packets can get lost. Accepting that failure is going to happen no matter what and building for resiliency to it is a far better approach.
Unit tests are along the same line- its been measured that fixing a bug in production is 10x more expensive than finding it in development. Unit tests help ensure that more bugs are found in development, there is less need for, slow, expensive, and error-prone manual QA processes that hold up release frequency.
Unit tests also help document the code to some extent. As teams turnover and new developers who may not fully understand the nuances of some code come in, Unit tests create guard rails that help catch edge cases that may not be obvious to new members of the team and when the test fails makes you go back and give more thought to those areas and why they behave the way they do.
Rereading this, you claim to be on the older side, which I am surprised about- this seems to me like an opinion that someone who is relatively new to the industry may have. I am continually amazed in the new and creative ways things can fail, building for resilience is a 100x better approach that also reduces stress tremendously- Instead of "All hands on deck no one is going home tonight- we just took the system down with this last release and will need to patch it ASAP" the conversation goes to "Ok- we rolled out the release to 1% of our users and we see error rates spiking. Lets roll it back and look at the logs to see what happened."
Similarly with unit tests- it used to be monthly or weekly release cycles, now we are down to multiple times a day because I don't need a programmer who washed out to get around to clicking on the right buttons to give it a green light.
These are tremendous improvements in the state of the art.
I am not arguing that unit tests are not valuable for the reasons that you have given, just that they seem to be used in place of decent abstractions these days. I have worked on well abstracted code without unit test and badly abstracted code with unit tests. The former is far easier to work with and bugs take a lot less effort to fix - that translates to costs in developer time.
> This is about cost optimization to some extent.
Is all the complexity and developer time that is required with Kubernetes really cost effective? We added it at my work and it took around 6 months of developer time from what I could tell (it wasn't me who set it up). Now we have to debug it and none of our team is an expert, so lots of guesswork and googling. Using number of small pods has given us a lot more problems than one big machine. If you are running out of memory because of a large upload, restarting a pod when it crashes doesn't help you.
Kubernetes has lowered our cloud costs and given us high availability, but it has increased development time in a number of ways. Maybe as we get more experience it will get better, but so far I would not say costs have been optimized as a result.
[1]: https://en.wikipedia.org/wiki/Design_Patterns