Have you considered or tried adding steps to create / review an engineering design doc? Jumping straight from PRD to a huge code change seems scary. Granted, given that it's fast and cheap to throw code away and start over, maybe engineering design is a thing of the past. But still, it seems like it would be useful to have it delineate the high-level decisions and tradeoffs before jumping straight into code; once the code is generated it's harder to think about alternative approaches.
Adding an additional layer slows things down. So the tradeoff must be worth it.
Personally, I would go without a design doc, unless you work on a mission-critical feature humans MUST specify or deeply understand. But this is my gut speaking, I need to give it a try!
Yeah I'd love to hear more about that. Like the way I imagine things working currently is "get requirement", "implement requirement", more or less following existing patterns and not doing too much thinking or changing of the existing structure.
But what I'd love to see is, if it has an engineering design step, could it step back and say "we're starting to see this system evolve to a place where a <CQRS, event-sourcing, server-driven-state-machine, etc> might be a better architectural match, and so here's a proposal to evolve things in that direction as a first step."
Something like Kent Beck's "for each desired change, make the change easy (warning: this may be hard), then make the easy change." If we can get to a point where AI tools can make those kinds of tradeoffs, that's where I think things get slightly dangerous.
OTOH if AI models are writing all the code, and AI models have contexts that far exceed what humans can keep in their head at once, then maybe for these agents everything is an easy change. In which case, well, I guess having human SWEs in the loop would do more harm than good at that point.
I have LLMs write and review design docs. Usually I prompt to describe the doc, the structure, what tradeoffs are especially important, etc. Then an LLM writes the doc. I spot check it. A separate LLM reviews it according to my criteria. Once everything has been covered in first draft form I review it manually, and then the cycle continues a few times. A lot of this can be done in a few minutes. The manual review is the slowest part.
