I don't know the PDF.js library. Writing both the client- and server-side for a PDF annotation editor would have taken 60 hours, maybe more. Instead, a combination Copilot, DeepSeek, Claude, and Gemini yielded a working prototype in under 6 hours:
> Writing both the client- and server-side for a PDF annotation editor would have taken 60 hours, maybe more.
How do you know? Seems to me you’re making the exact same estimation mistake of the people in the study.
> Instead, a combination Copilot, DeepSeek, Claude, and Gemini yielded a working prototype in under 6 hours
Six hours for a prototype using four LLMs? That is not impressive, it sounds insane and a tremendous mess that will take so long to dig out of the prototype stage it’ll effectively require a rewrite.
And why are you comparing an LLM prototype to a finished product “by hand” (I surely hope you’re not suggesting such a prototype would take sixty hours)? That is disingenuous and skewing the numbers.
> How do you know? Seems to me you’re making the exact same estimation mistake of the people in the study.
I have over 20 years of web development experience and 40 years of general experience writing software. I wrote the authors and they confirmed my thoughts:
"I totally believe it! Per the paper abstract, we find many factors driving results - and one of the factors is how experienced the developers are on the codebase, and how big/complex the codebases are.
"Given that this was a new and unfamiliar domain and new codebase, I would expect there to be much more speedup than the domain we studied!"
> Six hours for a prototype using four LLMs?
They have limits on the number of queries, so I used four different LLMs in tandem to circumvent query limits. I didn't write it four times using four different LLMs.
> it sounds insane and a tremendous mess
I posted the code. It's well organized, has few (if any) encapsulation violations, sticks to OOP quite well, works, and---if I knew the PDF.js API---would be fairly easy to maintain.
Yes, I stand by my claim that writing this annotation editor (PHP, HTML, CSS, and JS) would take me about 60 hours by hand and about 6 hours using the LLMs.
With most projects where innovation is a key requirement, the goal isn't to write textbook quality code, it's to prove your ideas work and quickly evolve the project.
Once you have an idea of how it's going to work, you can then choose to start over from scratch or continue on and clean up all the bits you skipped over.
Right now I'm in the innovation cycle, and having AI able to pick up whole API path strategies and pivot them, is incredibly amazing.
How many times have you used large API's and seen clear hands of different developers and URI strategies, with an AI, you just pivot.
Code quality and pen tests are critical, but they can come later.
> Code quality and pen tests are critical, but they can come later.
In my experience, no.
These kind of shortcuts taken at the beginning of the project is why velocity have a sharp descent after some times. Because you’re either spending time undoing all of it (unlikely to be allowed) or you’re fighting in the code jungle trying to get some feature out.
