> Why does the author feel confident that Claude won't do this?
I have a guess
| (I have almost zero knowledge of how the Windows CLI tool actually works. What follows below was analyzed and written with the help of AI. If you are an expert reading this, would love to know if this is accurate)
I'm not sure why this doesn't make people distrust these systems.
Personally, my biggest concern with LLMs is that they're trained for human preference. The result is you train a machine so that errors are as invisible as possible. God tools need to make errors loud, not quiet. The less trust you have for them the more important this is. But I guess they really are like junior devs. Junior devs will make mistakes and then try to hide it and let no one know
This is a spot-on observation. All LLMs have that "fake it till you make it" attitude together with "failure is not an option" - exactly like junior devs on their first job.
Or like those insufferable grindset IndieHackers hustling their way through their 34th project this month. Itβs like these things are trained on LinkedIn posts.
Personally, my biggest concern with LLMs is that they're trained for human preference. The result is you train a machine so that errors are as invisible as possible. God tools need to make errors loud, not quiet. The less trust you have for them the more important this is. But I guess they really are like junior devs. Junior devs will make mistakes and then try to hide it and let no one know