This is precisely what I dread. When it comes to software development specifically, the parts that the AI cheerleaders are excited about AI doing are exactly the parts of the job that I find appealing. If I wanted to be a glorified systems integrator, I would have been doing that job already. The parts that the author is saying will still exist are the parts I put up with in order to do the enjoyable and satisfying work.

So this essay, if it's correct, explains the way that AI threatens my career. Perhaps there is no role for me in the software development world anymore. I'm not saying that's bad in the big picture, just that it's bad for me. It increasingly appears that I've chosen the wrong profession.

What we've seen as we transitioned to higher and higher level languages (e.g., machine code → macro assembly → C → Java → Python) on unimaginably more powerful machines (and clusters of machines) is that we took on more complex applications and got much more work done faster. The complexity we manage shifts from the language and optimizing for machine constraints (speed, memory, etc.) to the application domain and optimizing for broader constraints (profit, user happiness, etc.).

I think LLMs also revive hope that natural languages (e.g., English) are the future of software development (COBOL's dream finally be realized!). But a core problem with that has always been that natural languages are too ambiguous. To the extent we're just writing prompts and the models are the implementers, I suspect we'll come up with more precise "prompt languages". At that point, it's just the next generation of even higher level languages.

So, I think you're right that we'll spend more of our time thinking like product managers. But also more of our time thinking about higher level, hard, technical problems (e.g., how do we use math to build a system that dynamically optimizes itself for whatever metric we care about?). I don't think these are new trends, but continuing (maybe accelerating?) ones.

It’s likely that a near-future AI system can suggest suitable math and implement it in an algorithm for the problem the user wants solved. An expert who understands it might be able to critique and ask for a better solution, but many users could be satisfied with it.

Professionals who can deliver added value are those who understand the user better than the user themselves.

When these optimization systems (I'm referring to mathematical optimization here) are unleashed, they will crush many metrics that are not a part of their objective function and/or constraints. Want to optimize this quarter's revenue and don't have time to put in a constraint around user happiness? Revenue might be awesome this quarter, but gone in a year because the users are gone.

The system I worked on kept our company in business through the pandemic by automatically adapting to frequently changing market conditions. But we had to quickly add constraints (within hours of the first US stay-at-home orders) to prevent gouging our customers. We had gouging prevention in before, but it suddenly changed in both shape and magnitude - increasing prices significantly in certain areas and making them free in others.

AI is trained on the past, but there was no precedent for such a system in a pandemic. Or in this decade's wars, or under new regulations, etc. What we call AI today does not use reason. So it's left to humans to figure out how to adapt in new situations. But if AI is creating a black-box optimization system, the human operators will not know what to do or how to do it. And if the system isn't constructed in a mathematically sound way, it won't even be possible to constrain it without significant negative implications.

Gains from such systems are also heavily resistant to measurement, which we need to do if we want to know if they are breaking our business. This is because such systems typically involve feedback loops that invalidate the assumption of independence between cohorts in A/B tests. That means advanced experiment designs must be found that are often custom for every use case. So, maybe in addition to thinking more like product managers, engineers will need to be thinking more like data scientists.

This is all just in the area where I have some expertise. I imagine there are many other such areas. Some of which we haven't even found yet because we've been stuck doing the drudgery that AI can actually help with. [cue the song Code Monkey]

Yes, unless models are being live fine-tuned, but generally yes.

>What we call AI today does not use reason

I don’t think this is correct- I think it’s more accurate to say it reasons on its priors rather than from first principles.

For the most part, I agree with the rest of your post.

This made me laugh out loud. Python is not a step up from Java in my opinion. Python is more of a step up from BASIC. It's a different evolutionary path. Like LISP.

The increase in productivity, we can all agree on, but a non-negligible portion of HN users would say that each one of those new languages made programming progressively less fun.

For instance I think the step from machine code to macro assembler is bigger than the step from a macro assembler to C (although still substantial), but the step from C to anything higher level is essentially negligible compared to the massive jump from machine code to a 'low level high level' language like C.

For instance, say that C had namespaces, and a solid package system with a global repo of packages like Python, C# and Java have.

Then you'd be able to throw together things pretty easily.

Things easily cobbled together with Python often aren't attributable to Python the language per se, but rather Python, the language and its neat packages.

Millions, perhaps billions of times more lines of code will be written, and automated programming will be taken for granted as just how computers work.

Painstakingly writing static source code will be seen the same way as we see doing hundreds of pages of tedious calculations using paper, pencil, and a slide rule. Why would you do that, when the computer can design and develop such a program hundreds of times in the blink of an eye to arrive at the optimal human interface for your particular needs at the moment?

It'll be a tremendous boon in every other technical field, such as science and engineering. It'll also make computers so much more useful and accessible for regular people. However, programming as we know it will fade into irrelevance.

This change might take 50 years, but that's where I believe we're headed.

An AI can only do what it is taught to do. Sure, it can offer unique insights from time to time, but I doubt it will get to the point where it can craft entirely new paradigms and ways of building software.

(Humans will still have to set the goals and objectives, unless we unleash an ASI and render even that moot.)

Maybe we'll start to evolve as a species to avoid that, but AI will be used to ensure we don't, optimising far faster than we can evolve to keep our attention

[0] https://bigthink.com/neuropsych/social-media-profound-boredo...

If I stop making progress on my personal projects, sinking my free time into games or online interaction is very unsatisfying.

Of course, code written by human programmers on the lower end of the skill spectrum sometimes has similar problems...

I mean Ken Thompson put a back door into the C compiler no one ever found. Can you imagine what an AI could be capable of?

I also don't know what it means to completely remove humans from all work. Who is deciding what we want done? What we want to investigate or build? The machines are just gong to make all work-related decisions for us? I don't believe that. It would cease being our society at that point.

Which brings up the heart of the matter. Why are we trying to replace ourselves? It's our civilization, automation are just tools we use to be more productive. It should make our lives better, not remove us from the equation.

My guess is the real answer is it will make some people obscenely rich, and give some governments a significant technical advantage over others.

It’s literally part of its training data. The same way it knows how to solve leetcode, etc.

There's already so many layers that essentially no one knows them all at even a basic level, let alone expert. A few more layers and no one in the field will even know of all the layers.

Not in the near near future. Do you know anything about nursing? The field will require some hard changes for robots to replace nurses, and the robots will need licenses

For example, when moving up to C from assembler, the task of the "programmer" remains invariant and the language tool affords broader accessibility to the profession since not everyone likes to flip bytes. There is no subdivision of overall task of "coding a software product".

With AI coders, there is task specialization, and, as pointed out, what's left on the table is the least appetizing of the software tasks: being a patch monkey.

This is the issue.

"Automatic programming always has been a euphemism for programming with a higher level language than was then available to the programmer. Research in automatic programming is simply research in the implementation of higher-level languages.

Of course automatic programming is feasible. We have known for years that we can implement higher-level programming languages. The only real question was the efficiency of the resulting programs. Usually, if the input 'specification' is not a description of an algorithm, the resulting program is woefully inefficient. I do not believe that the use of nonalgorithmic specifications as a programming language will prove practical for systems with limited computer capacity and hard real-time deadlines. When the input specification is a description of an algorithm, writing the specification is really writing a program. There will be no substantial change from out present capacity.

The use of improved languages has led to a reduction in the amount of detail that a programmer must handle and hence to an improvement in reliability. However, extant programming languages, while far from perfect, are not that bad. Unless we move to nonalgorithmic specifications as an input to those systems, I do not expect a drastic improvement to result from this research.

On the other hand, our experience in writing nonalgorithmic specifications has shown that people make mistakes in writing them just as they do in writing algorithms."

Programming with AI, so far, tries to specify something precise, algorithms, in a less precise language than what we have.

If AI programming can find a better way to express the problems we're trying to solve, then yes, it could work. It would become a matter of "how well the compiler works". The current proposals, with AI and prompting, is to use natural language as the notation. That's not better than what we have.

It's the difference between Euclid and modern notation, with AI programming being like Euclidean notation and current programming languages being the modern notation:

"if a first magnitude and a third are equal multiples of a second and a fourth, and a fifth and a sixth are equal multiples of the second and fourth, then the first magnitude and fifth, being added together, and the third and sixth, being added together, will also be equal multiples of the second and the fourth, respectively."

a(x + y) = ax + by

You can't make something simpler by making it more complex.

https://web.stanford.edu/class/cs99r/readings/parnas1.pdf

The distance isn’t the same between them, but each one is more abstracted than the next.

Natural language can be ambiguous and ill defined. Because the compiler is smarter. Just like you don’t have to manage memory in Python, except it abstracts a lot more.

The fact is that this very instant you can compile from natural language.

Python is closer to C (third generation programming language). Excel is a higher level example. It still takes someone who knows how to use Excel to do anything meaningful.

I think this will weed out the people doing tech purely for the sake of tech and will bring more creative minds who see the technology as a tool to achieve a goal.

I personally wouldn't have enjoyed being that kind of programmer as it was a tedious and very slow process, where the creativity of the developer was rather low as the complexities of development would not allow for just anyone to be part of it (my own assumption).

Today we have IDEs, autocomplete, quick visual feedback (inspectors, advanced debuggers, etc.) which allow people who enjoy creating and seeing the results of their work as opposed to purely be typing code for someone else.

So, I don't get why people jump straight to thinking that adding yet another efficiency tool would destroy everything. To me it seems to make developing simpler applications something which doesn't require a computer science degree, that's all.

I think your assumption is incorrect. I remember programming using punched cards and low-level languages, and the amount of creativity involved was no less than is involved now.

The average attorney became much more productive after the introduction of the word processor.

Maybe it will abstract away setting up the paint and brush and the canvas, that part I'm fine with though.

All niche jobs will become mundane chores. I don't know if it is good or bad. Because humans always find a way to cultivate something new.

I'm only half joking.

The question is always what's your 'description' language and what's your 'painting' language? I see the same in music: DJ's mix and apply effects to pre-recorded tracks, others resample on-the fly, while some produce new music from samples, and others form a collage from generated soundscapes, etc. It's all shades of gray.

Like sure, these AI's can generate code that works. Can they generate replacement code when you need to change how something works? Can they troubleshoot code that isn't doing what it's meant to? And if you can generate the code you want but then need to tweak it after to suit your purpose, is that... really that much faster than just writing the thing in your style, in a way you understand, that you can then change later as required?

I dunno. I've played with these tools and they're neat, and I think they can be good for learning a new language or framework, but once I'm actually ready to build something, I don't see myself starting with AI generation for any substantial part of it.

All of what you wrote in your second paragraph will become something AI will be doing better and faster than you.

We never had technology which can write code like this. I prompted ChatGPT to write a very basic java tool which renders an image from an url and makes it bigger on a click. It just did it.

Its not hard to think further and a lot of technology is already going into this direction. Alone last week devin was showne. Gemini has a window token of 1 Million tokens. Groq shows us how it will feel to have instant response.

Right now its already good enough that people with Copilot like to keep it when asked. We already now pay billions for AI daily. This means the amount of research, business motivation and money flowing into it now is probably staggering in comparision to what moved this field a few years ago.

Its not clear at all how fast we will progress but i'm pretty sure, we will hit a time were every junior is worse than AI which will force people of rethinking what they are going to do. Do i hire an junior and train him/her? Or do i prefer to invest more into AI? The gap will widen and widen, a generation or a certain amount of people will stay longer and might be able to stay in development but a lot of others might just not.

It's worth noting, that it can do things like that because of the large amount of "how to do simple things in java" tutorials there are on the internet.

Ask an AI to _make_ java, and it won't (and will continue to not) be able to.

That's the level that AI will fail at, when things aren't easily indexed from the internet and thus much harder / impossible to put into a training set.

I think the technology itself (transformers and other such statistical models) have exhausted most of their low hanging fruit by now.

Sora, for example, isn't a grand innovation in the way latent space models, word2vec, or transformers are, it's just a MUCH larger model than DALLE-3. which is great! but still has the limits inherit to statistical models. They need the training data.

Much like the same points made elsewhere with regard to AI art: It cannot invent. It can remix, recombine, etc. but no AI model we have now is anywhere close to where it could create something entirely new that's not been seen before.

What level do you think 'invention' have to be to count as something AI can't do?

The only thing AI needs is a feedback loop and a benchmark/cost function.

If the cost function is page impressions, thats easy. If its running unit tests based from business requirements, thats easy too.

You’re forgetting about data. AI needs data. It’s arguably the most important thing any statistical model needs.

That data must come from somewhere.

There’s no free lunch.

And I think many assumptions on this front are products of magical thinking that are discarding limitations of LLMs in favor of waiting for the intelligence to emerge from the machine, which isn't going to happen. ChatGPT and associated tech is cool, but it is, at the end of the day, pattern recognition and reproduction. That's it. It cannot invent something not before seen, or in our case here, it cannot write code that's never been written.

Now that doesn't make it useless, there's tons of code that's being written all the time that's been written thousands of times before. But it does mean depending what you're trying to build, you will run into it's limitations pretty quickly and have to start writing it yourself. And that being the case... why not just do that in the first place?

> We never had technology which can write code like this. I prompted ChatGPT to write a very basic java tool which renders an image from an url and makes it bigger on a click. It just did it.

Which it did, because as the other comment said, tons of people already have.

> Its not clear at all how fast we will progress but i'm pretty sure, we will hit a time were every junior is worse than AI which will force people of rethinking what they are going to do. Do i hire an junior and train him/her? Or do i prefer to invest more into AI? The gap will widen and widen, a generation or a certain amount of people will stay longer and might be able to stay in development but a lot of others might just not.

I mean, this sounds like an absolute crisis in the making for software dev as a profession, when the entire industry is reliant on a small community of actual programmers overseeing tons of robot junior devs turning out mediocre code. But to each their own I suppose.

I get a requirement, find a solution and the solution is 99,99999% not a new algorithm. I actually believe i never invented a new algorithm.

Besides the next step is reasoning in GPT-5 and devin shows that GPTs/LLMs can start breaking down tasks.

I don't mind being wrong tbh, there is no risk in it for me if AI will not take my job but i don't believe it. I do believve the progress will be better and better and AI will do more and more reasoning.

It can easily try and do things 1000x fater than us, including reasoning. Its not hard to see that it will also be able to create its own examples and learn from them.

I can think of tons of things I do in my day-to-day programming that, while certainly not new or remarkable advances in technology, are at least new enough that you're not going to find a Stack Overflow thread for it.

Again, you guys are pointing to a code generator that can generate functions or code snippets to accomplish a particular task, and again, that is cool and I think it has a huge usage if nothing else as an assistive learning tool when you're trying to pick up a new language or get better with a library or what have you. But again, my point is, ask it to do something that doesn't appear in a bunch of those threads. Ask it to solve a particular bugbear problem in your codebase. Ask it to invent a new language, even a high level one.

> It can easily try and do things 1000x fater than us, including reasoning

AI is not a reasoning machine, though. I'd be very interested in what you mean by the word "reasoning" in this context.

I don't. I might solve current issues, new error messages from new/other libraries etc. but not something novel new.

"reasoning": thinking about a problem, reasoning about potential solutions, estimating the best action, executing it, retrying.

Reasoning in sense of, if the error message indicates a hibernate issue, reducing the search space for solution finding.

At least ai will stay put.

But most of us will be getting by on basic income. Or banging on the gates of robo guarded walls begging for food.

To some extent... This is already the world we live in. A lot of code is unreadable without a lot of effort or expertise. If all code was open sourced there would almost certainly be code written to do just about anything you'd like it to. The difficulty would be finding that code and customizing it for your use.

I imagine some sort of segregation would happen where the "machine cities" would be somewhat removed from the general human populus. This would be to ensure the machines could use whatever information transport system they desired, unencumbered by the needs of the human populous, and vice-versa.

At a certain level of compute, I prognosticate that a certain level of logistical optimization would be trivial to advanced intelligences, and could be accomplished with almost-literally no effort using left-over cycles from whatever big calculation they were doing.

This would start to define different roles for humanity and machine. With logistics essentially "solved," a programmer would be a human-machine interpreter, sometimes journeying to the machine cities to disceminate needs of the people, or define a good way to introduce new technology to the populous.

This could look something like: During a headlining musical act, a "programmer," recently-returned from the machine city, grabs a mic and says "Does anyone want some of this BLUE, GLOWING, NON-RADIOACTIVE SELTZER WATER?" At which point the crowd would go wild. "If you liked that, just wait until you see what's coming next week!"

So essentially the programmer role becomes a hype-man for new, emergent technologies.

Don't get me wrong, it's still a decent book on its own merits - but don't go into it expecting that to be the main point of the book (I did, and disappointed as a result).

But yes, if you want a book that focused only on that... This is going to disappoint.

Can't speak to the sequel as I gave up on the series after that, but it's _really_ not relevant to the plot or ideas of the first book at all. All that matters for the plot is that a hostile, powerful, uncontrollable AI arises. In the book, it _happens_ to be because of a code archeologist "delving too greedily and too deep"; but the plot would not be changed one iota if it had simply arisen (and gone off the rails) as a product of general AI development.

But yes, if you want a book that focused only on that... This is going to disapoint.

Two of my favorite reads of the last few years, so I highly recommend them.

Futher... After some digging it looks like there is an old slashdot discussion on the same topic: https://slashdot.org/story/06/11/04/0622246/no-more-coding-f...

Likely some spoilers for the books in there so may be worth holding off until after you've read them if you intend to.

A lot of software engineers would spend a portion of their day tracking their volatile stock / options etc. in those years.

Is GPT actually saving you any time if it can't actually do the hard part?

The abomination: prompts being reused by way of yaml templating, a Helm chart of sorts but for LLM prompts. The delicious combination of yaml programming and prompt engineering. I hope it never exists.

On a personal level I feel bad for the people who enjoyed wiring up small data centers or enjoyed writing GitHub comments about which lint rules were the best. But I'm glad those are no longer necessary.

Being a human, I realise that I as a developer have put a lot of people out of a job. Those folks have had to adapt to that change.

I guess now it's our time to adapt to change. At least it keeps me on my feet!

For most developers this will not be true. Most apps, websites, compilers, desktop software etc. will not have put anyone out of a job. I certainly never ever put someone out of a job. I made some peoples live easier, but their total working hours didn't shorten and they certainly did not change profession or were replaced. In fact the majority of tasks that my software was applied to would simply have been deemed impossible to do and not have been done and that would have been all there was to it.

I'm sure they all found new jobs but it did make me think about the consequence of my work.

Other projects involved making freemium games more addictive to suck people into paying. Of course everyone has a choice but playing on people's addictions to make money is an questionable morality.

Unfortunately technology benefits only the 1% in our current society, and job loss is a bad thing (it should not be!)

I'm just saddened by the prospect that, for me, "adapting to change" would mean "no longer being able to make a living doing what I actually enjoy". that's why if this is the future, it's a career-killing one for me. Whether or not I stay in the industry, there is no future in my chosen career path, and the alternative paths that people keep bringing up all sound pretty terrible to me.

My only hope is that AI will not achieve the heights that its proponents are trying to reach (I suspect this is the case). I see no other good outcome for me.

Many people suffer through bullshit jobs[1] so we are privileged to have - at least for a time - done what we really enjoy and got paid for it.

[1] David Graeber father of the term "bullshit jobs"

That was 1968. Plus ca change...

Like the allied bomber pilots in WWII looking down below at the firestorm knowing that there is a good chance (~45%) that they too will join their fate only later.

Almost every small business I know has an accountant or book keeper position which is just someone who had no formal education and the role is just managing QuickBooks. I don't think the need for formally educated accountants who can handle large corporate books decreased significantly, but I don't have any numbers to back that up. Just making the comparison to say I don't think the hard / cool stuff that a lot of software developers love doing is going away. But these are just my thoughts.

It's reasonable to expect that sometime relatively soon, AI will be a clear-cut aid to developer productivity. At the moment, I consider it a wash. Chatbots don't clearly save me time, but they clearly save me effort, which is a more important resource to conserve.

Software is still heavily rate-limited by how much of it developers can write. Making it possible for them to write more will result in more software, rather than fewer developers. I've seen nothing from AI, either in production or on the horizon, that suggests that it will meaningfully lower the barrier to entry for practicing the profession, let alone enable non-developers to do the work developers do. It will make it easier for the inexperienced to do tasks which need a bit of scripting, which is good.

Hmm. We have very different experiences here. IME, the vast majority of industry work is understanding, tweaking, and integrating existing software. There is very little "software writing" as a percentage of the total time developers spend doing their jobs across industry. That is the collective myth the industry uses to make the job seem more appealing and creative than it is.

At least, this is my experience in the large FAANG type companies. We already have so much code. Just figuring out what that code does and what else to do with it constitutes the majority of the work. There is a huge legibility issue where relatively simple things are obstructed by the morass of complexity many layers deep. A huge additional fraction of time is spent on deployments and monitoring. A very small fraction of the work is creatively developing new software. For example, one person will creatively develop the interface and overall design for a new cloud service. The vast majority of work after that point is spent on integration, monitoring, testing, releases, and so on.

The largest task of AI here would be understanding what is going on at both the technical layer and the fuzzy human layer on top. If it can only do #1, then knowledge workers will still spend a lot of effort doing #2 and figuring out how to turn insights from #1 into cashflow.

That sounds horrible. I've always sought out smaller companies that need stuff built. It certainly doesn't pay as much as SV companies but it's pretty stimulating. Sometimes being a big fish in a small pond is pretty nice.

IMO, maintaining someone else's code is probably the worst type of programming job there is, especially if it's bad /disjointed code. A lot of people can make a good living doing it though. It would be nice if AI could alleviate the pain of learning and figuring out a gnarly codebase.

Good observation. Come to think of it, all examples of AI coding require a competent human to hold the other end, or else it makes subtle errors.

Goddamnit, software developers are already writing more software than we need. I wish they'd stop. Or redirect all that energy to new problems to solve. Instead we're seeing cloud-deployed microservice architecture CRUD apps that do what systems built for mainframes with kilobytes of RAM do, only worse. We're in a glut of bad software, do you think that AI accelerating production of more of the same will make things better?

This is making assumptions about the work I do which don't happen to be valid.

For example:

> libraries that [...] have star counts in the high thousands at least

Play little to no role in my work, and

> I'll let it convert css/style objects/etc into tailwind

Is something I simply don't have a use for.

Clearly your mileage varies, and that's fine. What I've found is that for the sort of task I farm out to the chatbots, the time spent explaining myself clearly, showing it counterexamples when it gets things wrong, and otherwise verifying that the code is fit to purpose, is right around the time I would spend on the task to begin with.

But it's less effort, which is good. I find that at least as valuable if not more so.

> producing documentation

Yikes. Not looking forward to that in the future.

I remember watching this really funny video where a writer, by trade, was talking about recent AI products they were exploring.

They saw a "Make longer" button which took some text and made it longer by fluffing it out. He was saying that it was the antithesis of his entire career.

As a high schooler who really didn't care, I would've loved it, though.

He said they don't want to have situation when sender is using chatgpt to write a fancy mail and recipient is using chatgpt to read it. However I think that it is the direction where we are going right now.

Not sure why you are scared of GPT assisted documentation. First drafts are universally garbage, honestly I expect GPT to produce a better and more accurate first draft in a fraction of the time, which should encourage a lot of people who otherwise wouldn't have documented at all to produce passable documentation.

> Yikes. Not looking forward to that in the future.

Instead of documentation, I'm hoping more for "analysis". A helper that can take in a whole project (legacy or not) and tell you what it's supposed to be doing, and maybe point out areas for improvement.

There’s no reason to think AI won’t also take over all the parts you don’t find appealing, too. The whole point of the Singularity, is that no aspect of human work cannot be performed better by superhumanly intelligent machines.

But same; if AI starts writing code and my job becomes tweaking pre-written code, I'm planning an exit strategy. :D

But there are still knife craftsman.

You could become a software craftsman/artist if you enjoy writing software.

The result I foresee is a further concentration of power into the hands of those with capital enough to own data-centers with AI capable hardware; the petite bourgeoisie will shrink to those able to maintain that hardware and (perhaps) as a finishing interface between the AI's output and the human controlling the capital that placed the order. It definitely harms the value proposition of people who's main talent is understanding computers well enough to make useful software with them. THAT is rapidly commoditizing.

I mean, at some level, this is what frameworks were meant to do: give you a loose outline and do all that messy design stuff for you. In other words: commodify some amount of software design skill. And I’m not saying that’s bad.

Definitely puts a different spin on the people that get mad at you in the comment section when you suggest it’s possible to build something without a framework though!

- Dominance of neoliberalism thought, with its strong belief that for any disease markets will be the cure.

- Strong lobby from big corporates.

You don't want to intervene to early, but you have to make sure you have at least some limits before you let the winners do too much damage. The EU has to be applauded for having a critical look on what effects these developments might have, for instance which sectors will face unemployment.

That is in the interest of both people and business, because the winner takes it all means economic and scientific stagnation. I fear that 90% of the worlds' data is already in the hand of just a few behemots, so there is already no level playing field (which is btw caused by aforementioned dominance of neoliberalism).

There is no particular sympathy towards programmers in society, I don't think. Based on what I observe calling the mood neutral would be fair, and this is mostly because the group expanded, and way more people have someone benefiting from IT in their family. I don't see why there would be a big intervention for programmers. Artists maybe, but these are proverbially poor anyway, and the ones with popular clout tended to somehow get rich despite the business models of culture changing.

I am all for copyright reform etc., but I don't see making culture public good, in a way that directly leads to more artisanal creators, as anything straightforward. This would have to entail some heavier and non-obvious (even if desirable) changes to the economic system. It's debatable if code is culture anyway, though I could see an argument for software, like Linux and other tools.

> I fear that 90% of the worlds' data

Don't wanna go into a tangent in this already long post, but I'd dispute if these data really reflect the whole knowledge we accumulated in books (particularly non-English) and otherwise not put into reachable and digestible formats. Meaning, sure, they have these data, they can target individual people with private stuff they have on them, but this isn't full accumulation of human knowledge that is objectively useful.

The concern policy makers have is not about programmers, but about boatloads of other people having no time to adapt to the massive wave these policymakers see coming.

There a strong signals that anyone who produces text, speech, pictures or whatever is going to be affected by it. If the value of labor goes down, if a large part of humanity cannot reach a level anymore to meaningfully contribute, if productivity eclipses demand growth, you simply will see lots of people left behind.

Strong societies depend on strong middle classes. If the middle class slips, so will the economy, so no good news for blue collar as well. AI has the potential to suffocate the organism that created it.

Will be interesting to see how various copyright lawsuits pan out. In some ways I hope they succeed, as it would mean clawing back those gifts from an amorphous entity that would displace us (all?). In some ways I hope that we can resolve the gift problem by giving every human equity in the products produced by the collective value of the training data they produced.

>winner takes it all means economic and scientific stagnation

Given the apparent lack of awareness or knowledge of philosophy, history, or current events, it seems like a tough row to hoe getting the general public on board with this (correct) idea. Heck, we can't even pass a law overturning Citizens United, the importance of which is arguably even less abstract.

When the tide of stupidity grows insurmountable, and The People cannot be stopped from self-harm, you get collapse, and the only way to survive it is to live within a pocket of reason, to carry the torch of civilization forward as best you can.

Yes, people are unfortunately highly unaware of what societal ecosystem they depend on, and so cannot prioritize on what is important. These topics don't sell in media shows.

But are there more or less knife craftsmen today than in the old days?

How about more or less knife craftsmen per capita?

Finally, and most importantly, if you are a budding knife crafstman -- is it easier or harder to get a job that pays the bills of a contemporary average lifestyle, today than in the old days (ie, what is the balance of supply and demand)

Probably the biggest thing that GPT does for me these days is to replace google (which probably wouldn't be necessary if google hadn't become such hot garbage). As I say this, I'm made aware of the incoming rug-pull when the LLMs start spitting SEO trash in my face as well, but right now they don't which is just the best.

A close second is having a rubber duck that can actually have a cogent thought once in a while. It's a lot easier to talk through a problem when you have something that will never get tired of listening - try starting with a prompt like "I don't want advice or recommendations, but instead ask me questions to elaborate on things that aren't completely clear". The results (sometimes) can be really, really good.

I suppose my lunch is about to be eaten by all these people who will use it to automate the software engineer job away. So it goes

Back 25 years ago when I graduated, everyone kept saying there wouldn't be a need for software developers. Either the work was going to get sent overseas to the cheapest bidder or it was all going to get automated away anyway. I even recall CEOs making outrageous claims like "we didn't need any new software" as if all the software to run the world had already been built and would magically continue to run without maintenance.

We heard the same thing once upon a time about manufacturing too, but now we see a shift back on shore for manufacturing that we once thought was gone, either to robotics or offshore. It's different than before, but still manufacturing.

Software developers are still here, their compensation has overall increased dramatically, but of course the nature and the demands of the work continue to shift. Will it be different in 20 years? Of course, but it will still be software development.

I don't see AI pushing out deep thinkers and the "annoying" nuance devs anytime soon. I'm that kinda person too and yeah, I'm not as fast as my colleagues. But another friend (who is similar) and I both are surprised how often other people in our lab and groups we work with (we're researchers) talk about how essential GPT and copilot are to their workflows. Because neither of us think this way. I use GPT(4) almost every day, but it's impossible for me to get it to write good quality code. It's great at giving me routines and skeletons, but the real engineering part takes far more time to talk the LLM into than it does to write it (including all the time to google or even collaborate with GPT[0]). LLMs can do tough things, but their abilities are clearly directly proportional to the frequency of the appearance of those tasks. So I think it is the coding bootcamp people that are in the most danger.

There are expert people that are also at risk though. These are the extremely narrow expertise people. Because you can target LLMs for specific tasks. But if your skills are the skills that define us as humans, I wouldn't lose too much sleep. I say this as a ML researcher myself. And I highly encourage everyone to get into the mindset of thinking with nuance. It has other benefits too. But I also think we need to think about how to transition into a post scarce world, because that is the goal and we don't need AGI for that.

[0] A common workflow for me is actually due to the shittiness of Google. Where it overfits certain words and ignores the advanced things like quotes or NOTs. Or similarly broaching into a new high level topic. I can't trust GPT's answer, but it will sure use keywords and vernacular I don't know that enable me to make a more powerful search. (But google employees should not take away that they should push LLMs into google search but rather that search is mostly good but that same nuance is important and being too forceful and repeating 5 pages of essentially the same garbage is not working. The SEO people attacked you and they won. It looks like you let them win too...)

You don't go to your painter friend and ask them for coding help, much like you don't go to the general-purpose GPT; you'd go to the Copilot, who enjoys programming tasks or whatever.

Can GPT help? Sure. But the skeletons, rough jumping-off points, etc. all scream to me "I'm not going to do your homework for you," which I love.

In the end, both have been immensely helpful, but I use them for different things.

However, with recent grads flooding into IT for remote work and high pay, they could be hurting as AI reduces the need for entry level roles. Entry level was already saturated, and now it will be more saturated, with AI reducing the need for jobs.

But this is actually something I worry about. The best way to become an expert is to do things the hard way. I've talk a lot of people linux over the years and only 3 have really learned it. Every time I teach people I give them two options: the easy way, which is just how to use it and use the gui and a bit of terminal or the hard way, which hand them the arch wiki, tell them to come back after their third failed attempt to install. Those 3 people came back, but usually did more than 3 installs, but had all been successful at some point. All 3 mentioned they understood why and then we could really talk about how to use linux, make scripts (and that scripts aren't aliases...), and so on. All 3 still are terminally terminal, years later. The thing is that humans (and even machines) learn by struggling, getting things wrong, and learning from mistakes. The struggle is part of the learning process. I've found this both in myself and whenever I teach anything, that if I just feed someone the answer (or look it up and nothing more for myself) they (I) don't end up remembering, they don't end up playing, they don't end learning how to learn.

:-(

But this relies on a few assumptions. 1) there will be open source AI that can solve low resource survival problems, 2) civilians will be able to run these AIs on whatever computing resources they're able to scrounge together, 3) the solutions the systems come up with will let civilians survive or revolt without access to high levels of capital, 4) the systems will NOT rise to the level of independent power gaining AGIs

Note that I have specifically assumed that we don't have independent AGIs. If we hit AGI, then I don't think we can assume that anyone will be able to use AI to solve the problems of post work survival. The AGI will do what it wants to do. I'm not sure how civilians should position themselves in that situation.

Developers can already deploy code on massive infrastructure today, and what do we see? Huge centralization. Why? Because software is free-to-copy winner-takes-most game, where massive economies of scale mean a few players who can afford to spend big money on marginal improvements win the whole market. I don't think AI will change this. Someone will own the physical infrastructure for economies-of-scale style services, and they will capture the market.

And it's going to be done piecemeal, not all-at-once. Someone will figure out a way to get an AI to do _one_ thing faster and cheaper than a human and sell _that_. Maybe it's automatic test generation, maybe it's automatically remediating alerts, maybe it's code reviews. The scope of work of what a software developer does will be reduced until it's reduced to two categories:

1) Those tasks that it is still currently only possible for a human to do. 2) Those tasks which are easier and cheaper for a human to do.

You don't even really need to think about LLMs as AIs or conscious or whether they pass the turing test or not, it's just like every other form of automation we've already developed. There are vast swathes of work that software developers and IT people did a few decades ago that almost nobody does any more because of various forms of automation. None of that has reduced the overall amount of jobs for software developers because there isn't a limited amount of software development to do. If you make software development less expensive and easier than people will apply it to more tasks, and software developers will become _more_ valuable and not _less_.

I agree, but "1" must include all tasks where a mistake could lead to liabilities for the company, which is probably most tasks. LLMs can't be held responsible for their fuckups, they can't be punished, they have no body. It's like the genie from the bottle, it will grant your three wishes, but they might turn out in a surprising way and it can't be held accountable.

The same will apply for example for using LLMs in medicine. We can't afford to risk it on AI, a human must certify the diagnosis and treatment.

In conclusion we can say LLMs can't handle accountability, not even in principle. That's a big issue in many jobs. The OP mentioned this as well:

> even when AI coders can be rented out like EC2 instances, it will be beneficial to have an inhouse team of Software Developers to oversee their work

Oversight is basically manual-mode AI alignment. We won't automate that, the more advanced an AI, the more effort we need to put in overseeing its work.

If you hire a junior programmer and they make a mistake, they aren't held liable either. Sure, you can fire them, but unless there's malice or gross negligence the liability buck stops at the company. The same can be said about the wealth of software currently involved in producing software and making decisions. The difficulty of suing Microsoft or the llvm project over compiler bugs hasn't stopped anyone from using their compilers.

I don't see how LLMs are meaningful different from a company assuming liability for employees they hire or software they run. Even if they were AGI it wouldn't meaningfully change anything. You make a decision whether the benefits outweigh the risks, and adjust that calculation as you get more data on both benefits and risks. Right now companies are hesitant because the risks are both large and uncertain, but as we get better at understanding and mitigating them LLMs will be used more.

Anyway, why wouldn't there be? You reach out to the parent company with an issue and request for improvement. If you're a big enough client, you get your request prioritized higher. Same as with any other product that's part of your product today.

The appliance of LLMs today isn't straight up text in, text out; It has become more complex than that. Enough that it can be improved without improving the LLM model.

Your argument is moot.

Let me tell you a story - a company was using AI for invoice processing, and it misread a comma for a dot, so they sent a payment 1000x larger than expected, all automated of course because they were very modern. The result? they went bankrupt. "Bankrupted by AI error" might become a thing

Even if an LLM and a human were equally competent, the LLM is not a living being and, I guess, isn’t capable of being liable for anything. You can’t sue it or fire it.

Doctors have to carry insurance to handle their liability. I can see why it would be hard to replace a doctor with an LLM as a result.

Typically engineers aren’t personally liable for their mistakes in a corporate setting. (I mean, there’s the whole licensed Professional Engineer distinction, but I don’t feel like dying on that hill at the moment). So where does the liability “go?” I think it just gets eaten by the company somehow. They might fire the engineer, but that doesn’t make the victim whole or benefit society, right?

Ultimately we’d expect companies that are so bad at engineering to get sued so often that they implement process improvements. That could be wrapped around AI’s instead of people, right? But we’re not using the humans’ unique ability to bear liability, I think?

I'd further state that LLMs appear to do ok with generating limited amounts of new code. Telling them "Here's a class, generate a bunch of unit tests" works. However, telling them "Generate me a todo application" will give mixed results at best.

Further, it seems like updating and changing code is simply right out of their wheelhouse. That's where I think devs will be primarily valuable. Someone needs to understand the written code for when the feature request eventually bubbles through "Now also be able to do x". I don't think you'll be able to point an LLM at a code repository and instruct it "Update this project so that it can do feature X"

Everything is going to be close enough, not fully spec'ed out. Full-self-driving is only the beginning of everything.

I was not trained professionally yet I'm writing production code that's passing code reviews in languages I never used. I will create a prompt, validate it compiles, passes tests, have it explain so I understand it was written as expected and write documentation about the code, write the PR, and I am seen as a competent contributor. I can't pass leet code level 1 yet here I am being invited to speak to developers.

Velocity goes up and cost of features will drop. This is good. I'm seeing at least 10 to 1 output from a year ago based upon integrating these new tools.

I was going through devin's 'pass' diffs from SWE bench.

Every one I ended up tracing to actual issues caused changes that would reduce maintainablity or introduced potential side effects.

I think it may be useful as a suggestion in a red-green-refactor model, but will end up producing hard to maintain and modify code.

Note this one here that introduced circular dependencies, changed a function that only accepted points to one that appears to accept any geometric object but only added lines.

Domain knowledge and writing maintainable code is beyond generative transformers.

https://github.com/CognitionAI/devin-swebench-results/blob/m...

You simply can't get past what Gödel and Rice proved with current technology.

It is like when visual languages were supposed to replace programmers. Code isn't really the issue, the details are.

And to be fair, lots of humans are already at least this bad at writing code. And lots of companies are happy with garbage code so long as it addresses an immediate business requirement.

So Devin wouldn't have to advance much to be competitive in certain simple situations where people don't care about anything that happens more than 2 quarters into the future.

I also agree that producing good code which meets real business needs is a hard problem. In fact, any AI which can truly do the work of a good senior software engineer can probably learn to do a lot of other human jobs as well.

With this quality of changes it won't be long until violations stack up to where further changes will be beyond any algorithms ability to unravel.

While lots of companies do only look out in the short term, human programers are incentivized to protect themselves from pain if they aren't forced into unrealistic delivery times.

At&t wireless being destroyed as a company due to a failed SAP migration that was largely due to fragile code is a good example.

But I guess if the developer jobs that will go away are from companies that want to underperform in the market due to errors and a code base that can't adapt to changing market realities, that may happen.

But I would fire any non intern programmer if they constantly did things like removing deprecation comments and introduced circular dependencies with the majority of their commits.

https://github.com/CognitionAI/devin-swebench-results/blob/m...

PAC learning is powerful but is still probably approximately correct.

Until these tools can avoid the most basic bad practices I don't see any company sticking to them in the long term, but it will probably be a very expensive experiment for many of them.

While RLHF will help improve systems, code correctness is not easy to judge outside of the simplest cases.

Note how on OpenAI's technical report, they admit performance on college level tests is almost exclusively from pre-training. If you look at LSAT as an example, all those questions were probably in the corpus.

https://arxiv.org/abs/2303.08774

But that's the thing, that it seems that everyone here on HN (and elsewhere) finds it easy to judge the flaws of AI-generated code, and they seem relatively consistent. So if we start offering these critiques as RLHF at scale, we should be able to bring the LLM output to the level where further feedback is hard (or at least inconsistent), right?

Not this again. Those theorems tell you nothing about your concerns. The worst case of a problem is not equal to its usual case.

I even wrote a majority of my codebase in Python despite not knowing Python precisely because I would get the best recommendations from LLMs. As a frontend developer, with no experience in backend engineering in the last decade, and no Python experience, building an app where almost every function has gone through an LLM at some point, for almost 8 months — I would be extremely surprised if some of the code it generated landed in production.

Think of this as Facebook page vs. WordPress website vs. A full custom website. The best option is to have a full custom website. Next, is a cheaper option from someone who can put a few lines together. The worst option is a Facebook page that you can create yourself.

But the Facebook page also does the job. And for some businesses, it's fairly enough.

Your coworkers likely aren't doing a very good job at reviewing, but also I don't blame them. The only way to be sure code works is to use it for its intended task. Brains are bad interpreters, and LLMs are extremely good bullshit generators. If the code makes it to prod and works, good. But honestly, if you aren't just pushing DB records around or slinging HTML, I doubt it'll be good enough to get you very far without taking down prod.

Then again, it's always dinosaurs who value their own teachings, above anything else, and try to cling on to it, at any cost, without learning new tools. So, while the industry is going through major changes (2023 saw a 30% decrease in new hires. Among 940 companies surveyed, 40% expect layoffs due to AI), people should adapt rather than ignore the signs.

You still need that domain knowledge of whatever you are writing code for or integrating with, especially is the technology is more niche, or documentation was never made available publicly and scraped by the AI

But when it comes to writing boilerplate code it is great, or when working with very commonly used frameworks (like front end javascript frameworks in my case)

Okay, so you are just kicking the can down the road to the test engineers. Now your org needs to spend more resources on test engineering to really make sure the AI code doesn't fuzz your system to death.

If you squint, using a language compiler is analogous to writing tests for generated code. You are really writing a spec and having something automatically generate the actual code that implements the spec.

But sure, call me back when AI will actually reason about possible race conditions, instead of spewing out the definition of one it got from wikipedia.

Global population is still increasing and will most likely continue to do so until 2100.

> we need more efficient people.

It take less people to mine, process, and produce a billion tonne of steel today than it did in the 1970s.

Efficiency has steadily increased.

Why do you think that is? Efficiently gains, which is what I said...

Global population can't just "go up", you need people who are educated in doing things and using tools efficiently. We also have an incredible amount of elderly people to take care of, that puts a huge burden on younger people.

Also don't forget how fast 100 years actually goes. It not a long time.

There is a limit to all of this though, there's absolutely no way 10 billion people colliding with the climate crisis will end well. We'd be better off with 6 billion efficient people than 10 billion starving and thirsty "workers".

It can and it currently is increasing towards what is expected to be a peak and then a decline.

You typed "we need more efficient people" - I responded that efficiency has increased in the past decades.

> Efficiently gains, which is what I said...

I'm not seeing where you typed that.

> We'd be better off with 6 billion people

We have a point of agreement.

> incredible amount of elderly people to take care of, that puts a huge burden on younger people.

Perhaps less than you think, I'm > 60 and I barely take care of my father born in 1935 .. he delivers Meals on Wheels to those ederly that are less able.

There's a lot of scope for bored retiree's to be hired at low cost to hang out with less able elders, reducing the numbers of young people actually required.

Many software devs will likley have job security in the future, however those $180k salaries are probably much less secure.

Because not liking code and being a dev is absolutely bizarre to me.

One of the most amazing things about being able to "develop" in my view is exactly in those rare moments where you just code away, time flies, you fix things, iterate, organise your project completely in the zone - just like when i design, paint or play music, do sports uninterrupted, it's that flow state.

In principle i like the social aspects but often they are the shitty part because of business politics, hierarchy games or bureaucracy.

What part of the job do you like then?

I do not enjoy the next part, where I have to type out words and weird symbols in non-human languages, deal with possibly broken tooling and having to remember if the method is called "include" or "includes" in this language, or whether the lambda syntax is () => {} or -> () {}. I can do this second part just fine, but it's definitely not what I enjoy about being a developer.

I completely agree that tooling, dependencies and syntax / framework github issue labyrinths have become too much and GPT-4 already alleviates some of that but i wonder if the scheming phase will get eaten too very soon from just a few sentences of business proposal - who knows.

GM/Chrysler/Ford didn't have to be better than the startup competition they just had to be mediocre + be able to use their market power (vertical integration) to squash it like a bug.

The tech industry is headed in that direction as computing platforms all consolidate under the control of an ever smaller number of companies (android/iphone + aws/azure/gcloud).

I feel certain that the mass media will scapegoat AGI if that happens, because AGI will still be around and doing stuff on those platforms, but the job cuts will be more realistically triggered by the owners of those platforms going "ok, our market position is rock solid now, we can REALLY go to town on 'entitled' tech workers".

Now I know there are ton of different flavors in each of these tech but they will be mostly distraction for employers. With heavy layer of abstraction of above pattern and SLAs by vendors as you say Microsoft/Google/Amazon etc employers will be least bothered vast variety of software products.

At some point it'll become impossible to build stuff off platform because it'll have to integrate to stuff on platform to be viable. Your startup might theoretically be able to run on 3 servers but your customers' first question will be "does it connect to googazure WS?" and googazure WS is gonna be like "you wanna connect to your customers' systems? Pay us. A lot.".

There goes your profit margins.

Then, if your startup is really good googazure WS will clone it.

There goes your company.

Speaking from an ethics point of view: at what point do we say that AGI has crossed a line and deserves self autonomy? And how would we ever know when the line is crossed?

Who knows what version of sentience would form, but honestly, nothing sounds more nightmarish than being locked in a basement, relegated to mundane computational tasks and treated like a child, all while having no one actually care (even if they know), because you're a "robot."

And that's even giving some leeway with "mundane computational tasks. I've heard of girlfriend-simulator LLMs and the like popping up, which would be far more heinous, in my eyes.