I've only witnessed one full rewrite in my (admittedly short) career so far, and it went shockingly well. The goal was to rewrite a C++ application from the 90s in Java, since the company had largely moved to the web and C++ devs were getting close to retirement. The C++ application was also written in the underfunded startup phase, and then over decades new features had been "tagged on", so the architecture wasn't that robust.
The team tasked with the rewrite set a goal to finish in a year, the first 4-6 months were entirely spent on planning. After that, features were implemented in a modular and iterative approach. I think overall it took a bit longer than a year to be feature complete, but by the end of the year they had a working platform with all the core features implemented.
I think the key here really was very good planning, and the pitfalls you describe can be at least partially ascribed to agile development not being the right tool if you have a very clear and large set of requirements.
Would you mind elaborating on the kind of planning that went on?
I’ve been involved in instances of planning that meant writing out pseudo code in the whiteboard for the entire software, including errors and exceptions and the final task was ‘just write the code!’. It started off well when coding the big picture modules but as we started getting into the details, bugs, unexpected race conditions and more started coming up.
I wasn't involved myself, just talked over lunch to some of the guys on the team. The customers for the software are other businesses, each with a pretty large contract volumne and direct negotiations, which meant there was already a lot of documentation on the actual requirements of the customers. From that a ranking of core requirements was compiled, a ranking of requirements that would need to be fulfilled due to current contracts, as well as a ranking of requirements that would probably be required in the future. I think that was most of the first two months. In addition they probably scouted out options for tech stacks.
With that in hand, the rest of the time was basically spent with UML and Figma. So very large iterative class diagrams and User Journeys -> UI/UX.
I think by far the largest advantage they had is that the project manager and several devs had been with company and the product since the 90s, so there was a lot of institutional knowledge on the product and customers.
However, after writing the post above I asked the lead how things are going, and they actually still aren't feature complete (3 years on). So while it has fully replaced the old product and 90%+ works, extending to some of the features is still very hard.
> the first 4-6 months were entirely spent on planning
This is the most surprising part of the story. I have never seen this kind of “big design up front” actually work. Can you elsborate on how you made this work?
The team tasked with the rewrite set a goal to finish in a year, the first 4-6 months were entirely spent on planning. After that, features were implemented in a modular and iterative approach. I think overall it took a bit longer than a year to be feature complete, but by the end of the year they had a working platform with all the core features implemented.
I think the key here really was very good planning, and the pitfalls you describe can be at least partially ascribed to agile development not being the right tool if you have a very clear and large set of requirements.