Not really. Got some code you don't understand? Feed it to a model and ask it to add comments.
Ultimately humans will never need to look at most AI-generated code, any more than we have to look at the machine language emitted by a C compiler. We're a long way from that state of affairs -- as anyone who struggled with code-generation bugs in the first few generations of compilers will agree -- but we'll get there.
>any more than we have to look at the machine language emitted by a C compiler.
Some developers do actually look at the output of C compilers, and some of them even spend a lot of time criticizing that output by a specific compiler (even writing long blog posts about it). The C language has an ISO specification, and if a compiler does not conform to that specification, it is considered a bug in that compiler.
You can even go to godbolt.org / compilerexplorer.org and see the output generated for different targets by different compilers for different languages. It is a popular tool, also for language development.
I do not know what prompt engineering will look like in the future, but without AGI, I remain skeptical about verification of different kinds of code not being required in at least a sizable proportion of cases. That does not exclude usefulness of course: for instance, if you have a case where verification is not needed; or verification in a specific case can be done efficiently and robustly by a relevant expert; or some smart method for verification in some cases, like a case where a few primitive tests are sufficient.
But I have no experience with LLMs or prompt engineering.
I do, however, sympathize with not wanting to deal with paying programmers. Most are likely nice, but for instance a few may be costly, or less than honest, or less than competent, etc. But while I think it is fine to explore LLMs and invest a lot into seeing what might come of them, I would not personally bet everything on them, neither in the short term nor the long term.
May I ask what your professional background and experience is?
Some developers do actually look at the output of C compilers, and some of them even spend a lot of time criticizing that output by a specific compiler (even writing long blog posts about it). The C language has an ISO specification, and if a compiler does not conform to that specification, it is considered a bug in that compiler.
Those programmers don't get much done compared to programmers who understand their tools and use them effectively. Spending a lot of time looking at assembly code is a career-limiting habit, as well as a boring one.
I do not know what prompt engineering will look like in the future, but without AGI, I remain skeptical about verification of different kinds of code not being required in at least a sizable proportion of cases. That does not exclude usefulness of course: for instance, if you have a case where verification is not needed; or verification in a specific case can be done efficiently and robustly by a relevant expert; or some smart method for verification in some cases, like a case where a few primitive tests are sufficient.
Determinism and verifiability is something we'll have to leave behind pretty soon. It's already impossible for most programmers to comprehend (or even access) all of the code they deal with, just due to the sheer size and scope of modern systems and applications, much less exercise and validate all possible interactions. A lot of navel-gazing about fault-tolerant computing is about to become more than just philosophical in nature, and about to become relevant to more than hardware architects.
In any event, regardless of your and my opinions of how things ought to be, most working programmers never encounter compiler output unless they accidentally open the assembly window in their debugger. Then their first reaction is "WTF, how do I get out of this?" We can laugh at those programmers now, but we'll all end up in that boat before long. The most popular high-level languages in 2040 will be English and Mandarin.
May I ask what your professional background and experience is?
Probably ~30 kloc of C/C++ per year since 1991 or thereabouts. Possibly some of it running on your machine now (almost certainly true in the early 2000s but not so much lately.)
Probably 10 kloc of x86 and 6502 assembly code per year in the ten years prior to that.
But I have no experience with LLMs or prompt engineering.
May I ask why not? You and the other users who voted my post down to goatse.cx territory seem to have strong opinions on the subject of how software development will (or at least should) work going forward.
>[Inspecting assembly and caring about its output]
I agree that it does not make sense for everyone to inspect generated assembly code, but for some jobs, like compiler developers, it is normal to do so, and for some other jobs it can make sense to do so occassionally. But, inspecting assembly was not my main point. My main point was that a lot of people, probably many more than those that inspect assembly code, care about the generated code. If a C compiler does not conform to the C ISO specification, a C programmer that does not inspect assembly can still decide to file a bug report, due to caring about conformance of the compiler.
The scenario you describe, as I understand it at least, of codebases where they are so complex and quality requirements are so low that inspecting code (not assembly, but the output from LLMs) is unnecessary, or mitigation strategies are sufficient, is not consistent with a lot of existing codebases, or parts of codebases. And even for very large and messy codebases, there are still often abstractions and layers. Yes, there can be abstraction leakage in systems, and fault tolerance against not just software bugs but unchecked code, can be a valuable approach. But I am not certain it would make sense to have even most code be unchecked (in the sense of having been reviewed by a programmer).
I also doubt a natural language would replace a programming language, at least if verification or AGI is not included. English and Mandarin are ambiguous. C and assembly code is (meant to be) unambiguous, and it is generally considered a significant error if a programming language is ambiguous. Without verification of some kind, or an expert (human or AGI), how could one in general cases use that code safely and usefully? There could be cases where one could do other kinds of mitigation, but there are at least a large proportion of cases where I am skeptical that sole mitigation strategies would be sufficient.
"And at no point does it ever occur to you to demand proof that measures such as this will have the desired effect... or, indeed, that the desired effect is indeed worth achieving at all."
(Shrug) There's a difference between prescription and prediction. I predict that after 50 years of doing the same old shit the same old way, the practice of programming is about to undergo a series of wrenching changes that amount to nothing less than revolution. Changes powered by radical new insights into the nature and function of language itself.
I'm not initiating these changes, voting for them, or attempting to persuade other people to do so, as the person I replied to in the other thread is doing. I do welcome having something new and interesting to learn and think about, though.
You do understand that LLM output is non-deterministic and tends to have a higher error ratio than compiler bugs, which do not exhibit this “feature”.
I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it.
I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it.
(Shrug) Tool use is important. People who are better than you at using tools will outcompete you. That's not an opinion or "combative," whatever that means, just the way it works.
It's no skin off my nose either way, but HN is not a place where I like to see ignorant, ill-informed opinions paraded with pride.
> If you don't review the code your C compiler generates now, why not?
That isn't a reason why you should NOT review AI-generated code. Even when comparing the two, a C compiler is far more deterministic in the code that it generates than LLMs, which are non-deterministic and unpredictable by design.
> Compiler bugs still happen, you know.
The whole point is 'verification' which is extremely important in compiler design and there exists a class of formally-verified compilers that are proven to not generate compiler bugs. There is no equivalent for LLMs.
In any case, you still NEED to check if the code's functionality matches the business requirements; AI-generated or not; especially in safety critical systems. Otherwise, it is considered as a logic bug in your implementation.
If you can look at what's happening today, and imagine that code will still be generated the same way in 10-15 years as it is today, then your imagination beats mine.
99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. It's not likely that any code that you and I run every day is.
> 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs.
Again, that isn't a reason to never check or write tests for your code because an "AI-generated it" or even assuming that an AI will detect all of them.
In fact, it means you NEED to do more reviewing, checking and testing than ever before.
> It's not likely that any code that you and I run every day is.
So millions of phones, cars, control systems, medical devices and planes in use today aren't running formally verified code every day?
Yes, I'm very sure. 99.9999% of the code you are running is not formally proven to be correct, and was not generated by a compiler whose output was formally proven to be correct.
Just curious, how much time have you spent in (a) industry, (b) a CS classroom, or (c) both?
You do understand that you are proving my entire point? It is still not a reason to *NOT* test or check your code implementation at all or to only rely on an LLM to check it for you.
What it really means is that software testing is extremely more important.
For running formally verified code every day, seL4 runs on the iPhone's security chip (secure enclave) in the hands of billions of users and it is a formally verified microkernel used for cryptographic operations from payments to disk encryption everyday.
This kernel is also used on medical devices, cars and in defense equipment, relied on by hundreds of millions of users.
> Just curious, how much time have you spent in (a) industry, (b) a CS classroom, or (c) both?
Lots of decades to know that no process developing safety critical system software would allow AI-generated code that isn't checked by a human or is only checked by other LLMs and using that as a substitute to writing tests.
If the majority of the code is generated by AI, you'll still need people with technical expertise to make sense of it.