The reasons are somewhat complicated, though some are straightforward. There is a ton of money in software, and a ton of people who want to be devs. Which means that even if a company makes a lot of mistakes they can still be profitable. And even if they burn out or drive away a ton of talent there is still no end of bodies willing to try to do the work, they just may not have the same level of skills.
I think rigor is an important part of the "solution" but in a way that's too easy. It's simplistic to just say "do better", which is sort of what a call for rigor is. Ultimately I think it's not rigor but just seriousness which is called for. Right now the industry is downright delusional and avoiding even talking about some of the fundamentals of software engineering as a practice. So many people want to feel like they're on the bleeding edge, rather than struggling through the mire (which is the reality). We have to get to the stage where people are talking about these things first before we get to the stage where we can do anything about them.
Right now the industry is in a rough shape. And the industry doesn't realize this mostly because there's so much money still (and money is a salve for many wounds) and because not long ago the industry was in crisis, so we're still seeing improvement from that state. Merely producing anything of value from a software project is still a big deal, and not a sure thing. Moreover, shipping any major product or service without relying on "crunch mode" or a death march is a rarity in the industry these days.
But we can do better. Not just with rigor but with how people use their time and do work. The norm is still too close to everyone running around in firefighting mode all the time, while technical debt blooms and burnout looms. A better mode is to work at a slower pace, get the right work done at a higher quality level, and still maintain good development velocity because you aren't slowed down by your technical debt all the time. There are some, small, pockets of the industry where that's the norm, but it's still a hard sell. At the end of the day the default is still to fall back to the ridiculous notion that software dev is like factory work and you get more bang for your buck by just adding more bodies and by having more butts in seats (or fingers at keyboards) for more hours per week.
I think rigor is an important part of the "solution" but in a way that's too easy. It's simplistic to just say "do better", which is sort of what a call for rigor is. Ultimately I think it's not rigor but just seriousness which is called for. Right now the industry is downright delusional and avoiding even talking about some of the fundamentals of software engineering as a practice. So many people want to feel like they're on the bleeding edge, rather than struggling through the mire (which is the reality). We have to get to the stage where people are talking about these things first before we get to the stage where we can do anything about them.
Right now the industry is in a rough shape. And the industry doesn't realize this mostly because there's so much money still (and money is a salve for many wounds) and because not long ago the industry was in crisis, so we're still seeing improvement from that state. Merely producing anything of value from a software project is still a big deal, and not a sure thing. Moreover, shipping any major product or service without relying on "crunch mode" or a death march is a rarity in the industry these days.
But we can do better. Not just with rigor but with how people use their time and do work. The norm is still too close to everyone running around in firefighting mode all the time, while technical debt blooms and burnout looms. A better mode is to work at a slower pace, get the right work done at a higher quality level, and still maintain good development velocity because you aren't slowed down by your technical debt all the time. There are some, small, pockets of the industry where that's the norm, but it's still a hard sell. At the end of the day the default is still to fall back to the ridiculous notion that software dev is like factory work and you get more bang for your buck by just adding more bodies and by having more butts in seats (or fingers at keyboards) for more hours per week.