> the models are in an important way indifferent to the truth of their outputs
Everyone who spends a bit of time with LLMs is aware of this.
My question - and I hope someone can address it in the discussion - how is it possible that we see such incredible investments in LLMs?
What are the c-suite execs and specifically CTOs seeing that is worth adoption and investment? These are very smart people.
I, as a laymen on the periphery of this phenomenon, am acutely aware that LLMs are not useful in applications requiring accuracy. I would not invest my company’s dollars or resources into such a technology.
But they do. What am I missing?
Is it the promise that LLMs will replace employees? How would that be believable for anyone willing to investigate the technology and its limitations, even a little bit?
> how is it possible that we see such incredible investments in LLMs?
Because somehow LLMs became the next "big thing" that nobody can explain but everyone wants to be involved in. It's like audio streaming, the most ineffective form of audio distribution, but for a while it seemed like a good idea.
> What are the c-suite execs and specifically CTOs seeing that is worth adoption and investment? These are very smart people.
The are idiots following the herd. They climb the greasy ladder of corporate career and fear getting fired. Their smarts are in corporate politics not in tech. And even if they were smart enough to tell that LLMs are an example of a research project that has received way too much funding, they need to appease the gods of Wall St who will judge them and punish them with a downgrade of the stock price if they cannot answer one question "What is your AI strategy?" It doesn't matter that the Wall St guys need help getting out of a strip bar, they have the power to ruin stockholders. This is why Apple, Google, and Salesforce are spending money of AI projects as a form of defence, not as a way to build new products or services. It's to show that they are relevant and can innovate.
Voice assistants were the same thing, so were chatbots, and audio and video streaming services (remember Groove Music Pass), touchscreens, pocket audio players (remember Zune?), etc. Fortunately for all involved, investors are already wondering if an AI correction is coming? I think we may see a wipeout of a lot of investments in AI soon.
> It's like audio streaming, the most ineffective form of audio distribution, but for a while it seemed like a good idea.
You dismissed it a bit too fast, didn't you? People are listening to streaming music everywhere - it is the new default. Even traditional radios stream over the internet. Is it efficient? Hell no. Is it popular? Extremely, even though a part of the population (including me!) prefers alternatives.
You are ignoring one important factor, streaming services can only stream what they are allowed to and if the artists and their labels decide to use a different distribution channel, streaming will become a thing of the past. Spotify has reached the point where they no longer know what to do and will soon be replaced by a competitor, not necessarily one that's streaming audio.
I agree, but im curious what you mean by it not being efficient? Is it not entirely dependent on how we interact with audio? Like you said its extremely popular, but its popular because its the most efficient way to listen to audio in the way people want too... discovering stuff, playing that track you heard at a party but don't know what its called... etc
I worked for a project that was a kind of mix of Netflix and Audible, just a few orders of magnitude smaller. Earlier, we were selling these multimedia products as digital downloads. That was simple for both sides, but with increased numbers of products and their weights customers started to complain their iPads etc. don't have enough storage to keep them (basically they would delete them and then request a new download link...).
But the amount of things you need to deal with in the subscription model is huge. Everybody expects Netflix-like experience, instantaneous playback and searching etc. We can do that for N simultaneous streams; for more we need to add a slight delay - and that is noticeable. In any case, the same file is streamed multiple times, sometimes dozens of times a day for the same client. This is what I mean terribly inefficient. And although you can probably cache a few audio files, it doesn't work well for videos. From my point of view someone just wastes the bandwidth for not having to store a file locally. Yes I know this is how it is supposed to work, but still.
> Because somehow LLMs became the next "big thing" that nobody can explain but everyone wants to be involved in. It's like audio streaming, the most ineffective form of audio distribution, but for a while it seemed like a good idea.
I know it's slightly off topic, but could you elaborate on the "audio streaming" part?
Sure. Compared to traditional broadcast over airwaves or even a download to a device that can play audio without having to stream it from the server, audio streaming is the least efficient and the least convenient form of audio delivery. When you have a transmitter, people adding a thousand new radio receivers in the area does not increase your cost as a broadcaster, adding a thousand clients requires more bandwidth, and more infrastructure. I never understood why audio streaming became as popular as it did when better forms of distribution were available.
But how many people really stream the same content at the exact same time anymore these days?
Large sports events are probably the only remaining application where broadcast would still see much advantage over individual streams.
In the case of a few simultaneous viewers, unicast can sometimes still use bandwidth more efficiently in case of a(since the transmitter precisely knows the receiver’s SNR and doesn’t need to waste any energy transmitting at the wrong coding rate or the wrong direction/place). That’s how modern 802.11 (Wi-Fi) often does reliable multicast, these days.
A seamless transition from unicast to multicast once the critical ratio of listeners per base station is reached would be very cool, but require pretty deep levels of integration between broadcasters and ISPs that are probably not worth it.
FYI, I work for a big sports streaming company and we are doing "mABR" or multicast adaptive bitrate with some ISPs because it really is worth it. You get a significant improvement in playback stats and the ISP gets massive traffic offload from their network. We started with one big ISP in Italy, then other ISPs saw what we were doing and asked to join in.
That being said, it really is a pain in the arse to build. It's also largely proprietary solutions (yes it's multicast but getting the player to find the content isn't a standard) and it's hard for the ISP to update ALL their routers to support it (even when they want it).
They mean "audio streaming" ala "video streaming" not like audio downloading to your device streaming. The stream means live download not just download through the net.
thank you, that makes sense in terms of efficiency! I am not familiar with the subject, but aren't the amount of radio frequencies available rather limited?
Yes, that's a different type of limitation imposed on the broadcaster. My point was that AI has not got a business model except capturing whatever value remains after it destroys existing models, just like audio streaming. Recent backlash against Daniel Ek's musing on X and the general behaviour of Spotify towards artists shows signs of reaching the end of the line for this idea. Same fate awaits AI. If you see how little of substance can be found in answers given by AI luminaries in their numerous interviews you can see for yourself that those guys have no idea and no plan. The best antidote for this bullshit is to display the transcripts on YT videos and read what they are saying. It's pure nothing.
I understand where you are coming from, but I’m not sure I completely agree.
Streaming music was popular even before it became legal, so I believe it's largely driven by people's desire to listen to whatever they want, whenever they want.
Regarding AI, it’s true that the current results might not always be the highest quality. However, there is no theoretical limit to AI's capabilities as far as I know. With this many people working to improve it, I would personally be surprised if significant advancements aren’t made. Maybe not just next year though...
Ultimately, I guess we will have to wait and see. Who knows, maybe you will be right in the end :)
The number of people working on improving AI is relatively small compared to the number of people reselling OpenAI APIs so I don't expect huge improvements in the next 5-10 years. The real value of AI is in ML, but with human supervision and interpretation of results. GenAI is the worst thing that happened to AI, because its coming rejection will reflect badly on the rest of the advances broadly labelled as AI.
> Streaming music was popular even before it became legal
Downloading music was popular. You download once, you listen many times. On Napster, it could take hours until a song was downloaded. No way you'd delete it and download again next time you want to listen to it.
You have it right there. There is a large set of applications that does not require accuracy. These applications will benefit hugely from LLMs. Examples include recommender systems, knowledge extraction, transformation, etc. all in domains where accuracy does not matter too much.
In particular, instruct aligned LLMs makes it very easy to integrate ad hoc machine learning into an application.
I assume that's what c-level people have in their mind:
Nothing is accurate and fail-safe 100%. You just have to reach a level where it is good enough most of the times to make money with it.
Also, human brains are neural networks too, so that's where we know what the minimum is which is (theoretically) possible with this kind of technology.
I don’t care what they were inspired by, because that’s not what we’re talking about. The comment I replied to said that brains ARE neural networks, in a literal sense. This is a common misconception, but is well known to be just that among people who are actually well-informed on the topic. Just because you don’t understand it doesn’t make it any less true.
Brains are networks of neurons, e.g. neural nets. They also contain other structures like blood vessels, etc, but it is completely reasonable to say they are neural networks.
Again, you are confusing neural networks with artificial neural networks. Neural networks are to artificial neural networks as birds are to ornithopters.
Why don't you just cut to the chase and bring out the dualism "antennas for the soul" crap already? No sense beating around the bush, your username has me sure it's coming eventually.
Ironically, a drone is becoming increasingly similar to a type of bird. In 100 years, when they might be made of organic material, I would guess it will be normal to call them birds.
>My question - and I hope someone can address it in the discussion - how is it possible that we see such incredible investments in LLMs?
Opportunities like this happens once in 5-10 years when “the next big thing” are in the radar. Idea behind this investments is to bullshit it’s way to the Series B or IPO where original investors can exit. It is not about usefulness but about using momentum of the situation to extract money.
I think you’ve really captured the essence of this phenomena in a remarkably succinct way. I do hope that business leaders eventually develop their own capabilities for independent creative thought, rather than being a bunch of mindless also-rans.
> What are the c-suite execs and specifically CTOs seeing
So I can speak to this since I am trying to sell to them.
CEO is being told from the markets, board, consultancies, Gartner etc that you need to have a Generative AI strategy. What that means in practice is irrelevant. You just need to throw it all against a wall and see what sticks. If you don't then the risk is you get left behind.
CEO is telling the CIO that the need to figure this all out with the obvious pilot programs being in Customer Support and Search which the CFO has been tasked to find budget for. The problem is that the data in these companies is poor and getting poorer as the number of SaaS apps explode and they are forced to just dump everything in a messy data lake. Which is why for the CDO, AI is like their #6 concern and is going to make their lives 100x harder.
The logical conclusion to all of this is that 95% of these projects will fail: (a) data is bad with no easy way to fix it, (b) LLMs are inaccurate/biased making them a risk, (c) ROI is poor, (d) data privacy/sovereignty is a mess.
> CEO is being told from the markets, board, consultancies, Gartner etc that you need to have a Generative AI strategy. What that means in practice is irrelevant. You just need to throw it all against a wall and see what sticks. If you don't then the risk is you get left behind.
I can confirm this is the default view at many companies, just the usual FOMO - the irony being this happens even in the companies that are valued for not delivering computer-generated content. Using AI they actually take a greater risk than not doing that - but well, the CEOs somehow are convinced they have no other choice.
When you have a slack conversation with a coworker they are often not 100% accurate.
Working with an LLM is much like working with a coworker that has a knowledge base as wide as the entire internet.
It is exceptionally useful. There are a wide range of applications that may even require 100% accuracy in final output but do not require 100% accuracy in every conversation that leads to that output.
For the last 6 months I have built things that would have taken me 4x as long in programming languages that I’ve never used before.
I am a team of 1 that has the output of a team of 4 without the overhead of meetings
> My question - and I hope someone can address it in the discussion - how is it possible that we see such incredible investments in LLMs?
The same reason we did for blockchain. It's a sexy new technology, with amazing demos and hype. People who want to be entrepreneurs but have no ideas of their own race to apply it everywhere because they think hitching their wagon to this horse could be their shot. Boring old companies wanting to seem hot. Lots of FOMO.
> What are the c-suite execs and specifically CTOs seeing that is worth adoption and investment? These are very smart people.
> I, as a laymen on the periphery of this phenomenon, am acutely aware that LLMs are not useful in applications requiring accuracy.
They are not useful in isolation. They can be very useful as part of a larger process, especially if they particular way that you're using them has a low error rate. (And given a particular task you can in fact measure the error rate, or how often the generated bullshit happens by chance to (not) be true.)
Most processes that involve steps done by humans have built-in pieces to catch and correct errors, because humans also make errors. If the error rate of the LLM component is lower than the error rate of the human it's replacing, you're fine. If it's higher but the cost of catching and fixing the extra errors is lower than the cost of the replaced human, you're still fine. (The person who you just sent the way of the buggy whip manufactures might be less fine, but that's Somebody Else's Problem.)
If you're only looking at English generation, and focusing on the limitations, you're missing the bigger picture. The AI revolution is about the fact that we have now have the ability to produce ChatGPT level of thing, but for all of the kinds of data that companies have, not just English text. The idea isn't that it will replace the employees, but that the employees will be augmented by AI and be able to be way more productive. What executive hasn't wished they had 10x the people on a project at a fraction of the cost. The allure for a CTO isn't about LLMs, it's about resource management.
staring at the hallucination problem is missing the forest for the trees. yes it's a problem, but holy shit, the thing generates essays as fast as the GPU can dot-product and matrix multiply. CTOs are buying into the idea that we've invented cars, when before we were just looking at a faster horse.
"Is it the promise that LLMs will replace employees?"
If we accept the conclusions of the paper, perhaps LLMs can replace employees who are bullshitters.
For example, Microsoft has announced layoffs claiming "AI wave" as the cause. Perhaps so-called "tech" companies like Microsoft employ bullshitters and "AI" can replace them.
Employees who act as if they are right 100% of the time when in truth they are wrong at least 20% of the time.
Why hire people who can only be relied on 80% of the time, and who will never do what it takes to reach 100%, if this "work" can be done by a computer.
My view is that there is extrapolation going on. They look at the progression of GPT 2-3-4 over a relatively short number of years and are extrapolating to a near future where these things are much more capable. FOMO. But the history of AI is huge leaps from a technological innovation, followed by years of crawl, and all this extrapolation may be wrong.
> What are the c-suite execs and specifically CTOs seeing that is worth adoption and investment? These are very smart people.
It's a whoa moment for the general population. Look, the computer can understand and respond in natural language. I don't think there is other value in LLMs. Other applications of AI are very good, though.
They are also very good at translating between natural and structured/"computer" language.
At a much higher rate than anything I've seen before in any algorithm, they "get" irony, sarcasm etc., when until a few years ago, I (as a non-linguist), assumed that any such solution to the "aboutness" problem would require solving AGI.
That alone is worth a lot in a world which spends considerable resources on people that effectively work as translators between human and structured language.
Besides that, I suspect that their existence is seen as a strong and remarkable hint that intelligence really might just be a quantitative phenomenon emergent from throwing a lot of compute at a lot of data, which at some point might become self-reinforcing.
Whether that's true or not, and for better or worse, that's what people now seem to be set on doing.
chatgpt and code completion tools have pretty much doubled my productivity... "write me this tedious function in c# for unity" could save me up to a couple minutes, "what are 5 makes and models of PoE switches used in industrial applications?" could save me up to an hour
Because if you utter AI in your earnings call your stock moons. Thats the drive right now. The executive thinks in the short term and keeping up with the joneses so to speak and jumping off the AI cliff with the rest of the lemmings is what their investors are expecting.
Companies revolve around risk management and whilst people can be wrong, biased, discriminatory etc there are measures that can be taken to mitigate them. There are no comparable mitigations for LLMs.
And so if your LLM continually tells people to eat rocks or glue and you can't do anything about it then that very much impacts their value.
I think the title is intentionally inflammatory, but I do think we need more criticism.
We have a group of people trying to convince the world that LLMs may possibly (it could be 50+ years away!!) turn into AGI.
We have all these impressive benchmarks "WOW 91% in MathEval!!!" but go ahead and use the OpenAI API to do y=mx+b and watch it completely fail at any number over 4 digits.
We have the coding benchmarks, but watch it still use var instead of let.
All the numbers are going up, but it still feels primitive and cannot be trusted. This is almost 2 years later this point.
We've got people bolting on a dozen other solutions to make LLMs work, theres probably tens of millions of LOC now written to make LLMs functional in apps, so much for reducing engineering jobs.
IMO look at machine vision, we thought we were 5 years away from autonomous driving 15 years ago. It's still evolving and the tech only partially uses machine vision. Meanwhile its revolutionized dozens of other industries. Same is happening with LLMs.
It doesn't make sense to criticize an elephant for not knowing how to climb a tree. LLMs can do absurdly impressive things. It is so impressive that we expect it to perform the same way with any thing we throw at them. That's an absurd expectation.
> We've got people bolting on a dozen other solutions to make LLMs work, theres probably tens of millions of LOC now written to make LLMs functional in apps, so much for reducing engineering jobs.
To be fair, in my niche at least I can get a better result by querying ChatGPT4 than StackOverflow. When you take into account that the replies are often mistaken, outdated etc., it becomes just another tool to use form time to time when you need it.
The title is not inflammatory, but (granted) easily misunderstood. "Bullshit" is a term of art in philosophy, coined by philosopher Harry Frankfurt (1929-2023) in his 2005 booklet On Bullshit, in which he distinguishes bullshit from lies:
> The liar cares about the truth and attempts to hide it; the bullshitter doesn't care if what they say is true or false.
We don't need agi to have LLMS make a massive impact on employment.
They can already likely replace most associate attorneys as long as their work is reviewed by a competent attorney as the work of associates is already.
They can likely replace a large portion of call center workers.
They can increase productivity for software engineers reducing the need to expand teams.
Add the replacement of truck drivers if self driving improves over the next decade and you have a fundamental shift on employment.
There are likely a dozen more areas they can have massive impact. Fast food order taking, etc.
ChatGPT is a better Dissociated Press algorithm[0]. Dissociated Press appears to be one of those "forgotten hacker lore" things, something which in the past many hackers would've come across (not least likely through Emacs M-x dissociated-press) but these days not so much. (Recently, YouTuber ThePrimeagen rediscovered "Real Programmers Don't Use Pascal"[1] and read it aloud on his channel.) In the LLM era of AI, hackers (and perhaps more importantly, non-hackers, like business people) should understand Dissociated Press and how it works because, fundamentally, an LLM is doing the same thing: looking at the chunk(s) of text that came before and selecting the chunk of text most likely to follow based on that. DP uses a simple Markov chain to make this selection, so its output is obviously garbage while an LLM has much deeper memory, a much better statistical model, and it's trained on a much larger corpus of text, so its output is... much less obviously garbage. :)
But the best it can do is always "plausible-looking", with no regard to whether it is even true, let alone insightful. Like a tarot spread, the output of an LLM may provoke insights in the reader's brain, but those must come entirely from the reader's brain and must be used very carefully. I've also compared it to the robot from Asimov's short story "Liar!" which could read minds and, due to the First Law, always tells people exactly what they want to hear (malfunctioning and shutting down when it is told that telling people false things does in fact harm them). The LLM can't read minds, but it is ever only mindlessly following its one directive: to tell you what its statistical model indicates you are expecting to hear next.
For a long time now I've been seeing this ugly trend in the titles of linked articles.
I wish every HN user would flag any article with these clickbait titles, instead they're always popular. You're encouraging this shameless virality tactic by engaging with content with titles like that. "$popularthing is just the worst" is crafted to engage you. By clicking, you reward the author's behavior.
Please just flag them without clicking the link, until blog authors take the hint and write sensible titles. That's what I'm going to do from now on. Flag on sight.
The world, even the academic world whom this articles is aimed at, is much larger then only the HN crowd. Also, the title made me feel interested to read on. So they did win for me.
Yes, the article is partly discussing semantics. But given that the general public is mostly clueless as to how LLMs work, these attempts may bring some positive result.
Doesn’t matter. Just slap AI on it and hopefully no one will ask about it. I was invited to a presentation the other day “AI in the construction industry. Someone claimed it would be worth $10 - 20 BILLION” in 2025. I politely declined. What the fuck is chatGPT going to do? Advise them to build more houses or advise them to go back to using mud and straw. GTFO.
I feel like we get so worked up about AI being wrong, maybe because of what we think AI should be? It seems like getting mad at a hammer because its bad at being a paintbrush.
I guess the problem is the hammer being marketed as a paintbrush ;)
Same. It's sad that a real paper in a real journal is flagged (not just downvoted) evidently just because it appears antagonistic to the prevailing AI-optimism narrative.
Can we please just have a sane bifurcation of what LLMs are good/bad at?
They are not good with facts. They are not good with precision. They never will be. That is not even their purpose.
Neither are humans, we rely on sources.
LLMs can be made to rely on sources through RAG and agents. They can respond to errors with actor/critic modeling techniques. But ultimately they should be generating commands to strict rules engines.
They must be combined with goal seeking agents, retrieval/ rules engines, critics, etc to be useful.
Their only advantage is their 24/7 relentlessly consistent scalable throughput.
As someone who uses AI, as a tool, every single day, I call bullshit.
If you understand limitations of current AI, it is an exceptionally useful tool. Just have realistic expectations and understand how to use the tool for what it has to offer.
If you expect the tool to have all of the answers and get everything 100% right, then you should not be using it at all. But don't preach your beliefs to the rest of us who are able to get tremendous value out of it.
The authors’ definition of bullshit ("soft" bullshit at the very least) applies no matter whether the bullshitter is sentient or not: text produced without concern as to its truth value.
> text produced without concern as to its truth value.
Since the "concern", if the model has one at all, is to produce text likely to be produced by humans in the same context, then that would imply that it has concern as to it's truth value, precisely in as much as the average human has concern about the truth of the text they produce.
You need people to bullshit. The terms hallucinate and temperature, and now bullshit humanize a program.
Babies are cute, stuffed animals are cute, Im not trying to convince you that one is alive and going to grow out of control and kill you or take your job...
Yes, but it's the wrong definition. The fundamental reason people bullshit is that they have something to gain. LLMs have nothing to gain. So it is much more appropriate to call what they do "hallucination" rather than "bullshit".
There will be a lot of manual CV redacting to remove or lessen the importance of people's involvement with AI. Like who puts VR or AR on their CVs these days? I have seen to versions on the same CV from a guy who applied for a job with the client I consult for. Two years ago, he was highlighting his Oculus dev experience, this year he describes that role as 3d software development.
Everyone who spends a bit of time with LLMs is aware of this.
My question - and I hope someone can address it in the discussion - how is it possible that we see such incredible investments in LLMs?
What are the c-suite execs and specifically CTOs seeing that is worth adoption and investment? These are very smart people.
I, as a laymen on the periphery of this phenomenon, am acutely aware that LLMs are not useful in applications requiring accuracy. I would not invest my company’s dollars or resources into such a technology.
But they do. What am I missing?
Is it the promise that LLMs will replace employees? How would that be believable for anyone willing to investigate the technology and its limitations, even a little bit?