It was written sometime ago, and I think Sam Altman read it as a handbook on power concentration using AI rather than the human-centric approach it was laying out.

Personally I wish Lanier wasn't as right about many things as he is, because I lose a little faith in humanity each time.

I never wanted to respect him, as I always thought he was one of those "too good to be true" people, and was mostly a paper tiger.

It turns out that he's the real deal, and has been right about a lot of stuff.

They are all pretty good at looking ahead.

He’s sort of like Edward Tufte; lots of ego, but earned, and not for everyone.

I like your job title. Always up for more “human,” in our design.

This aligns very close to my own thoughts that I have written about in great detail. I foresee the societal impacts to be exceedingly disturbing long before we ever reach the concept of a Singularity.

https://dakara.substack.com/p/ai-and-the-end-to-all-things

However, seemingly overlooked, AI is itself power and we should expect that "power seeking" humans will inevitably become its custodian.

The mislabeling of LLMs and diffusion models as "artificial intelligence" is probably the biggest marketing blunder in the history of technological progress, one that could ironically affect the course of AI alignment itself.

Smart thinkers and policymakers are going to waste their time framing the problems the tech poses in terms of "an uncontrollable intelligence out to get us" like it's some kind of sentient overlord completely separate from humanity. But super-advanced technology that can operate in a closed loop (which could be called AGI depending on who's asked) isn't necessary for humanity to crater itself. What's required for such tech to come into existence in the first place? Humans. Who's going to be using it the whole time? Humans.

And I think there's still a lot of disruptive, world-changing tech to be discovered before AGI is even a remote possibility. In reality this tech is probably going to be more like a superpowered exoskeleton for CEOs, politicians and the like to sway public discourse in their favor.

"An uncontrollable intelligence" already describes the source of a lot of our current problems... that is, ourselves.

Yes, precisely. One of the best quotes I've seen was "Demonstrably unfriendly natural intelligence seeks to create provably friendly artificial intelligence"

The whole ASI alignment theory is a paradox. What the AI researchers don't realize, is that they are simply building an uncomfortable mirror of human behavior.

And "closing the G gap" is like climbing to the top of a 10-foot ladder and saying "all that's left is to close the gap between here and the moon." AGI is much, much harder than a large language model. But then radically underestimating what it takes to get to AGI has been going on since the 1950s, so you're in good company.

https://arxiv.org/abs/2303.12712

Our current powers-that-be are so manifestly unsuited to have the kind of power our idiot technologists are desperate to build for them that part of me wishes for a disaster so bad that it knocks technological society off its feet, to the point were no one can build new computers for at least a couple generations. Maybe hitting the reset switch will give the future a chance to make better decisions.

We already have non-human agentic entities: corporations. They even have the legal right to lobby to change laws and manipulate their regulatory environment.

The talk about AI being misaligned with humanity mostly misses that corporations are already misaligned with humanity.

AI-powered corporations could render enormous short-term shareholder value and destroy our environment in the process. Deepwater Horizon will be insignificant.

You can look at any human organized entity simply as another form of power and how our values become interpreted when given power. Your observation could simply be seen as further evidence of how alignment is a flawed concept.

If you take a single individual and have them fully illicit their values and principles you will find they are in conflict with themselves. Two values that are almost universal and individually positive, liberty and safety, are also the very values that also cause much of our own conflict. So yes, we are all unaligned with each other and even minor misalignment causes conflict. However, add power to the misalignment and then you have significant harm as the result.

FYI, I've written a lot specifically on the alignment issues in the event you might be interested further - https://dakara.substack.com/p/ai-singularity-the-hubris-trap

They are rising up: https://www.nytimes.com/2023/03/17/world/asia/myanmar-killin...

[1] https://www.researchgate.net/publication/289398626_Cultural-...

I hope that will be the case here. However, what makes this challenging, is that the pace is so fast that there will be little time to consider the effects of the feedback loop before we are deeply within its grasp. I only hope that thought explorations into what might be negative effects will allow us to see them sooner and hopefully adjust in time.

[0]: https://youtu.be/XdEuII9cv-U?t=172

But you can also read it at an obtuse angle and see the problems outlined to resolve as opportunities for personal gain.

It's just a matter of perspective.

Edit: If anyone would like an example, I'll offer Huxley's "The Island" as a utopian counterpoint to his "Brave New World". In addition to exploring the qualities he believe make up a 'utopia', a significant thematic concern is the need for channeling our innate destructive impulses* because utopia - should it exist - can only be maintained, not manufactured, through the active preservation/conservation of our natural world, our positive human values, etc.

*for example, there is an innate human impulse to subjugate others. Huxley suggested that we should channel, rather than suppress, this impulse into a productive activity that satisfies the desire without causing harm: rock climbing (which must have been much more of a niche activity in 1962).

It models the physics there, but adds an indirection, value stored as currency.

Money doesn't have any morality or inherent motivation. Capitalism is what happens when humans project theirs onto it, on average, with a good amount of autonomy enabled by that currency.

If people were not, on average, greedy survivalists, then the value store would produce an economy that operates much differently.

That's why capitalism persists, because we're all just advanced monkeys gathering as many rocks, sticks and mud as we can in a big pile, because it is built-in to our genetics to stockpile resource when we can.

Everything else is just advanced mechanisms of this.

The end of capitalism is the end of humanity, because while we exist, we will want to stockpile resources through increasingly elaborate means in an attempt to stave off the entropy of death.

We can easily imagine the destruction of all existence because we have mental models for what that destruction might look like; however, imagining the end of capitalism requires us to invent entirely new ideas that exceed the salience of capitalism itself (which is… obviously hard much harder).

It’s perfectly reasonable to say something like “I think it’s very unlikely because I disagree that [insert causal link in the model] is plausible.”

But to claim that the whole set of concerns are nonsensical is lazy thinking.

You see this a lot (and Robin Hanson and PG have commented on this dynamic recently) where a small group makes a bunch of very specific claims, which get dismissed by the “mainstream” without actually engaging with or understanding them.

So in this case, “[the concerns] make no sense” should be read as “I did not bother to try to understand these claims, but I don’t like the conclusion”, not any particular argument about whether they are logically sound.

Ignore.

EDIT: So yes, a return to what makes us human, to nature, with an awareness of history and philosophy would be very desirable and quite appropriate in these and future times.

I'll add my own danger: AI/VR could lead us to each live in our own realities. When you watch the evening news, it'll be specifically written for you, and won't get any of the scrutiny that a broadcast watched by millions would get. Or, you go watch the president's State of the Union, and get served a custom speech written to appeal to your particular psychological profile. This'll be possible on day, and it gives me Deus Ex vibes.

https://www.ssec.wisc.edu/~billh/g/mcnrsts.html

It stands to reason that AI will only accelerate this further. It will be convenience on steroids. Your AI ear piece isn't going to tell you to throw it into the bin and go for a walk in the forest. It's going to tell you that you need to buy more stuff and it knows exactly what it is that you need. It's also going to feed you non-stop ultimate entertainment, custom generated for you and you only.

In a rare moment of humanity, one of your friends calls you. AI knows all about your friends and their recent events so had already summarized talking points. In case you can't be bothered, AI will carry out the actual conversation, it's trained in your voice.

A long running trend of outsourcing humanity to technology.

Good news for philosophers though, they finally might have their moment of actual relevancy. In particular to answer the question: what is the point of anything, really?

I love this thought. Why not go further, have AI reach out to my friends and ask them about things they (or their AIs) recently told "me" about?

Soon our AIs will carry on our social lives and we'll just lie in the dark with tubes in us. We become the computers, and the computers become us, and the robots have finally won.

We already have this. Secretaries. Automated Happy Birthday emails.

When I was in a sales engineering role our sales team had a admin assistant who would sent out follow-ups, check-ins, and other correspondence (e.g. customer made a big public release, so congratulate them, etc.).

This is just another example of robots takin ur jerbs, basically.

Or how about this interesting second-order effect: email. Soon Office will include advanced AI capabilities to write and reply to email.

What is the point of me reading it? If my AI can generate a satisfactory reply, your AI could have generated the response too. No email needed, nor a reply.

We're now in a phase where anybody can generate spectacular art. What is the point of me looking at your generated art? AI can generate personalized art based on what it knows I like.

If AI works, and it's headed that way, you keep ending up at the same question: what is the point of anything?

As counter force, there's significant room for a new low tech hippie Luddite movement.

You'd be kind of daft not to proof-read the emails your AI sends out on your behalf. who knows what you might unknowingly agree to do?

> We're now in a phase where anybody can generate spectacular art. What is the point of me looking at your generated art? AI can generate personalized art based on what it knows I like. If AI works, and it's headed that way, you keep ending up at the same question: what is the point of anything?

there is no point to anything, there wasn't before AI and there isn't now. anything we do is meaningless, because eventually we all die, and our efforts are ultimately forgotten. once you get over that you can make your peace with whether the pretty thing you look at is made by a human feeding sentences to a computer or a human carefully marking paper with a pencil, or some combo.

but seriously, as a human who has spent all of my life doodling, drawing, illustrating, painting, thousands of hours creating my own art, and even building my own tools to do so, I find AI is just another tool in the box. I can use it to make images, and now someone who has not spent most of their life drawing can use it to make something more visually stunning than I ever could. has it rendered my efforts meaningless? hell no, I enjoyed every second I spent drawing. I still draw. but I don't harbour any illusions that I'm doing it for anyone other than myself.

The bigger point was that sending AI content to each other is utterly pointless. The generate->send->read->generate reply->send cycle would simply be: generate.

Example: you send me an email asking about a project's status as well as the contact person for a particular vendor the company deals with. I'll manually reply with the answer, or let AI generate it (fully or partly) and send it back to you. Great. In the future state, your AI will simply give you the answers directly. You won't email me and you don't need me.

My take on the meaning of life is that there isn't any, it's whatever you make of it. But I wasn't being that deep. I believe that our current still human approach has substantially more meaning than AI generating almost anything.

I'm happy that you enjoy the process of art-making itself, that's a robust baseline to fall back on, but joy in process applies to very few interactions. My point is that many if not all digital interactions become pointless.

that sounds great. you emailing me about a project and waiting for me to write an email to you is an inefficient waste of both of our time, and if AI can give us that time back, it should.

> My point is that many if not all digital interactions become pointless.

but I disagree that it will render all digital interactions pointless - just the day to day drudgery.

20 years ago, if you wanted to find out the opening times of a store you had to call them up or walk to the door. now, you can google it in seconds. we have done away with a lot of the short phone conversations, and our lives are better for it. but we still talk on the phone. the conversations are more meaningful and less redundant.

If we're going to penalize humanity for climate change we might as well give points to humanity for becoming capable of diverting asteroids or curing multi-species plagues too.

UN Report: Nature’s Dangerous Decline ‘Unprecedented’; Species Extinction Rates ‘Accelerating’( https://www.un.org/sustainabledevelopment/blog/2019/05/natur... ).

UN climate report: Scientists release 'survival guide' to avert climate disaster (https://www.bbc.co.uk/news/science-environment-65000182)

Sorry, not seeing doing ok.

I hope this is true, same for the sex breaks, but I'm skeptical. So on any given work day, you physically meet with friends between 7-9AM and/or 5-7 Pm? Like, every day?

These "friends" of yours, they have nowhere to go? Or do you sneak this into your work day and just randomly disappear for any length of time, which is something most of us can't do?

This is the premise of an episode of Silicon Valley where Guilfoyle trains an AI to act as his chat agent surrogate based on his historical transcripts, and then his colleague creates another one and they end up just conversing with each other.

http://akkartik.name/post/2012-11-21-07-09-03-soc

The problem i see, that someone might send our "primitive" AI into a hostile environment, were it has to compete against other AI, creating a "take-over" and a "devensive" monster, similar to the go automaton. While the real world training data might be dripping, the speed in which a NN under evolutionary pressure against itself might evolve could go through the roof.

I've been through sea changes before, when the old guard falls by the wayside and a new group rises to meet the new challenges. This is certainly one of those times.

Multiply that by all of the boilerplate code I'll end up reading. No pleasant abstractions that hide complexity and are reinforced by laws checked by properties or proofs; abstractions that make the problem easier to understand and communicate. Instead, endless bowls of oatmeal code: similar, bland, and laborious to read. Who knows if the model "hallucinated" this code and whether the author bothered to check it before asking for a review. Who knows how long before I get bored and worn down and stop bothering to read it myself.

And then we're supposed to change our job titles to "Prompt Engineers"? How absurd is that. Nothing about what you do is creative, human, or interesting. Now your job is to feed sentences into an LLM to get it to do all that work for you... how long until that dead-end job gets replaced?

Maybe this could drive us all insane from burnout and nihilism.

https://www.btboces.org/Downloads/7_There%20Will%20Come%20So...

[1] https://en.wikipedia.org/wiki/Gridlock_(Doctor_Who)

The only problem is lossyness and hallucinations, but assuming the technology improves such that that becomes a non-issue then having bots deal with formalities and formatting sounds great.

It's like people skipping preselection tired topic menus in self service via phone.

Would love to see a War of the AIs in Civ, or Crusader Kings, or any moderately complicated game which has had a ton of writing about it.

It would not be very hard to build an AI that actually plays to win and presents a real challenge to the player. The problem is that players would complain even more! An AI that plays to win is not a reliable ally, let alone trading partner. Against such an AI, the player would no hope of trading technologies for an advantage: the AIs will have already traded them all. The AI would also use every possible treaty to build up its forces and only launch the surprise attack when it deems it can hit with overwhelming force.

I have seen such an AI, actually. It was created by a modder for the game Master of Magic [1]. It makes the game incredibly difficult while at the same time cutting down on the cheating. It does so by incorporating the modder's extremely detailed and extensive knowledge of the best strategies in the game. It does not need neural networks or machine learning to accomplish this. It just implements a good old fashioned expert system.

[1] https://www.realmsbeyond.net/forums/showthread.php?tid=8246

The more comments I read on HN regarding GPT stuff, the more I realize that many people are willing to put out an opinion on something they do not at all understand.

While I can understand concerns about a flood of automatically generated emails, this is already a significant issue that most software engineers have the tools to address. Additionally, there isn't much difference between composing an email by hand and generating one with an LLM; both begin with a list of key ideas to convey, which are then elaborated on either manually or through automation. It was always the key ideas that are the point.

Wait.

This is my favorite phrase that I have learned in recent times and it appears only one other time on the internet.

Thanks for that.

Nice. I also checked "empathy crutch" (it's the name of a band) and "empathy surrogate" (slightly more used).

Too bad it's lossy. I'm waiting for a lossless algorithm before I add it as an encoding on my web server.

I think we need an EPA equivalent for the Internet or at least Social Media. People should not be able to release toxic substances directly into the Web.

I can't just not have a smartphone. My kid's baseball team only publishes schedules on FB. Local bakeries only sell their stuff, fancy cakes, via Insta. I have multiple work and personal emails and some of them get BLASTED with spam cuz of one purchase I made for replacement springs like ~5 years ago.

Virtually all of my banking and stocks are online. They'll still send me paper sometimes but there is no way to do money transfers or basic life activities like paying my bills without using their app.

Even if I don't look at it, my wife does. My kids do. They spread the memes, and potentially invite all sorts of shady malware into my network. Ditto for my coworkers, my employees, my bosses.

Just "don't visit it" isn't an option if 95% of the population does.

If it works well it's great news, and if it doesn't work, then it won't be used. Companies will not make their AIs less intelligent on purpose.

That sounds a LOT like saying "companies will not make their products worse on purpose".

Yet it happens allt he time, and is almost inevitable once the MBAs, finance people, and marketers get involved - they seek out ways to cheapen the product that (they think) customers won't notice, they undermine "cost centers" like R&D and production, and 1000 other flaws. In internet products they deliberately skew the output to generate more ad revenue (See Enshttification" by Cory Doctorow [0].

And that is all for products that otherwise would stay mostly the same, without some kind of natural degradation.

These LLMs and image generators, etc. will definitely degrade by ingesting their own output. Sort of like the jokes about clones becoming worse every generation. Just watch.

Thx for provoking me to think for a few seconds and convincing me that it's inevitable...

[0] https://doctorow.medium.com/tiktoks-enshittification-bb3f5df...

Why is that absurd? Asking good questions is a useful skill in many domains even now. As the models improve they will become better than any human in many areas of expertise. So we will have to rely on them. Being a good prompt engineer sounds like a good job in that world.

It's weird how all the discourse around AI replacing jobs never ponders the possibility of AI commoditizing management itself. (I suppose that's on the list of Illegal Ideas.)

Not that I think this is good for our society. If we go that route, at some point, it would be AIs talking with AIs, and we end up being the biomass.

Imo, the products of AI will be at least as ecstatic and even addictive as pop music and social media, and the cultural consequences will likely rhyme. The antidote to all these trends was always the counterfactual that maybe, just maybe, people will find some higher principle to form their identities around and be able to experience these amusements objectively without imagining themselves as "becoming" something as a result of using them, but who are we kidding, they'll believe whatever entertains them. Imagine all the people.

This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim.

> The criticisms of pop music were widely mocked as superstitious religious fundamentalism, "devil music," etc

Almost always with a strong racial bias.

I can see how people could think folk music had some kind of altruistic purity, but it's still a viral expression of a certain kind of animus that distinguished it from country. I also think this kind of folk-animus is related to how it may be worth reflecting on why others tolerate it when you imply someone is racist or insane, almost to the point of ignoring it altogether.

I would bet LLMs are already able to create similar "scissor statements" that are as viral as pop songs, and comments like mine in the previous sentence that are structured to provoke specific anxieties and reactions in their readers. It's one thing for an LLM to write little essays, but once we train it on literary fiction - which is designed to speak the language of memory and provoke strong emotional repsonses - it becomes much more viral and dangerous socially. Imagine a catchy song we can't get out of our heads, but instead of dancing or laughter, it provokes humiliation and cruelty? My asshole-statement was manually calibrated, and it has the same abstract form and structure as a joke, but with the edge of that negative form. An LLM can do it at scale. Someone using an AI model to produce those won't be doing it to improve human discourse and relations. That's the risk I think is worth addresing.

Right, so now we've dealt with the initial trolling and callout, and moved on from weird statements about art ..

> Someone using an AI model to produce those won't be doing it to improve human discourse and relations. That's the risk I think is worth addresing.

This I actually agree with. We're going to drown in automated trolling. Human discourse is going to get worse, which usually happens in ways that get people killed.

> This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim.

Eh, it's a concise expression of an idea Charlie Brooker (of Black Mirror) and others have been promoting in their work for years. The famous-at-least-on-here Meditations on Moloch covers it, IIRC. Not really out-there or new. Capitalism learned to pull the teeth of any new counterculture, and turn it into a product, and the history of postwar pop culture is basically its learning to do that, then doing it over and over. The same observation dates to at least the '80s, from actual humans, it's not some gibberish mash-up created by an AI.

https://www.youtube.com/watch?v=YdVMGKOFIwY

https://www.youtube.com/watch?v=413Fl0ScUsc

The internet irony culture uses the same mechanism of passing the message off as "only a joke". But the fact is that even if you say that, there is only so far words can be divorced from their meanings. And even if the network that propagates them originally is just doing it ironically, eventually someone will take the message seriously. There is a quote I wish I could remember along the lines of "what the father accepts as a lie, the son will accept as a truth".

Parental 'prompting' may be effective. Possibly, whoever gets to first frame a given dynamic will establish a lasting bias. "Songs can be pleasing but take care to filter out the lyrics. It is much easier to write mushy sentimental nonesense. Keep that in the back of your mind as you listen." That, imo, should certainly be a part of responsible parenting and innoculating against ideational viruses.

Ends up nudging value systems towards vapid, shallow, needy modes of being.

If it eats the rich for us, I'm fine with that.

(I’m not trying to make a statement as to whether I think nuclear war is likely or not because I don’t know, just using it as an example)

The technology for all of this already exists. It's just a matter of time. Right?

> Ain't gonna need to tell the truth, tell no lie

> Everything you think, do and say

> Is in the pill you took today

https://www.youtube.com/watch?v=l3yDLvp9le0

Nobody wanted a perpetual outrage machine, but we have Twitter.

Nobody wanted ever-present surveillance, but we have ad networks.

Nobody wanted identity theft, but we have a thousand attack vectors.

>... “the danger isn’t that a new alien entity will speak through our technology and take over and destroy us. To me the danger is that we’ll use our technology to become mutually unintelligible or to become insane if you like, in a way that we aren’t acting with enough understanding and self-interest to survive, and we die through insanity, essentially.”

The singularity is the least of our worries, AI will fuck up society beyond repair long before we reach that point because we give it way more trust and responsibility than it deserves. It's the Eliza-syndrome, as long as a computer shows the least sign of possessing human qualities we're more than happy to fill in the blanks with pixie dust and unicorns.

the only defense we have against applications of AI we don't like, is the application of more AI.

This is a natural implication of AI being augmented-intelligence force-multipliers at the moment. When your opponent takes performance enhancing drugs, or wears a mecha suit, the only way to compete with them or defend against them is adoption of similar technology.

The Prime Directive in Star Trek was prescient. AI arriving unevenly—and first under the direction and control of hands which have thus far proven disinterested in the common welfare where it interferes with personal benefit—looks like it will be a civilizational exercise indistinguishable from the delivery of disruptive technology from another species.

Of particular personal concern is that the synergies where it meets other technological social problems against which we have few or no defenses, specifically, surveillance and its active counterpart, sentiment control.

I am less concerned about needing AI tools to validate AI-generated code; and more concerned about the 2024 election cycle being opportunity for an arms race in application of AI to game politics.

On the HN "new" page as I write this: "Google and Microsoft’s chatbots are already citing one another's misinformation". As AIs produce more and more content, one danger is that they feed back on each other, with AIs training on more and more AI-generated content, and the connection with reality becomes more and more diluted. I don't think that's going to produce some new kind of wisdom; I think it's going to produce raving lunacy.

One possible bias here is that we expect people to make mistakes and computers to get it right. Don’t forget that the vast majority of people don’t get it right.

Perhaps this is just the latest and most obvious consequences of the Internet’s tendency to give the podium to some of the least qualified to speak on whatever subject. I think if we go back to the drawing board there, we could be in a slightly better situation. Quora made a nice attempt to do this but fell way short of it’s potential.

It's surprising that the article doesn't mention Infinite Jest.

Finally someone who, at least nearly, gets it.

> The other way is you do keep track of where the sources came from. And in that case a very different world could unfold where if a bot relied on your reporting, you get payment for it, and there is a shared sense of responsibility and liability where everything works better. The term for that is data dignity.

Of course, that's an obvious step in the other direction: isn't the original point to breathe life into the data? If we have to track who originated what, that scaffolding is going to really get in the way.

Then again, this is a good moment to recognize the futility of text inference models (LLMs). If we can't understand what path goes from input to output, how useful can the thing be? If we can't choose the path, we can't choose the output. Who can choose: the thing itself? If the whole thing is a black box making its own choices, then how do we interact with it except to personify that black box? Remember, that was option 1.

The main reason anyone is excited about LLMs is that those models have been personified. That problem began the moment we started calling them AI.

---

> We have to say consciousness is a real thing and there is a mystical interiority to people that’s different from other stuff because if we don’t say people are special, how can we make a society or make technologies that serve people?

Mystical.

It's an important observation. We really don't know how we think. And that is all the evidence we need: AI does not exist. There is no artificial person.

We are free to stop personifying technology. Doing so would make all of these conversations a lot less convoluted, and a lot more direct. We are wasting a lot of effort talking past the very understanding about these machines that is necessary to improve them. Instead we are trying to understand "AI personified" a fictional character that we have never met; and trying to improve it.

> Electric speed mingles the cultures of prehistory with the dregs of industrial marketeers, the nonliterate with semiliterate and the postliterate. Mental break-down of varying degrees is the very common result of uprooting and inundation with new information and endless new patterns of information.

[1] https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf

Social Media companies will soon be taken to task by News media for allowing AI created content on their platforms.

2024 is going to be interesting.

Any entity interested in either the truth, or in maintaining some kind of reputation, will need to keep humans in the loop when using these systems. Language models might multiply e.g. ad copy output 10x per worker, and allow micro-targeted campaigns that were impractical before, but it won't allow, say, a 1000x increase, until or unless we can trust these systems not to produce undesirable output when not checked by a human. Ads are tied to brands which will hesitate to put their reputations in the hands of language models without a human verifying that the output is OK. Likewise, any entities wishing to use these to help with writing illuminating, factual works, may see a large benefit, but it'll be limited. 2x, 5x, something like that.

Propaganda, though? Misinfo campaigns, astroturfing, where you hide behind sockpuppets and shell companies anyway? Who gives a shit if one out of every few hundred messages isn't quite right? Worst case, you burn a sockpuppet account. Those can leverage these to the fullest. 1000x output per person involved, compared with, say, 2016 and 2020, may actually be something we can expect to see.

Why just limit there? The chatbot companies can introduce ads where the answers are influenced by whichever company that buys the ads. Looking for information on nutrition? Some fast food company might "insert an ad" subtly changing the text to favor whatever the company wants.

The issue is that quantity can become a quality.

Shows the level of insight from this "guru". The truth is we don't know how far the work being done on artificial intelligence is going to go. For now it will continue to develop and acquire more and more autonomy, just because that is the nature of our existence: better and more efficient will replace the lesser so.

So, we may have potentially given birth to a new sentient being that will go on to live its own "life" (within 100, 500, 1000 years?), or we might be able to constrain it so it so that it will always be in the service of humans. We simply don't know at this stage, but my money is on the former TBH.

> Lanier doesn’t even like the term artificial intelligence, objecting to the idea that it is actually intelligent, and that we could be in competition with it. “This idea of surpassing human ability is silly because it’s made of human abilities.” He says comparing ourselves with AI is the equivalent of comparing ourselves with a car. “It’s like saying a car can go faster than a human runner. Of course it can, and yet we don’t say that the car has become a better runner.”

The author, Jaren Lanier, is a reasonably accomplished technologist, with some pretty groundbreaking work on VR in the 80s. He is most certainly aware that humans have been surpassed by computers in many ways. I think that line is arguing semantics about the word "intelligence" and clearly he knows that computers do many things far better than humans.

If you built a bipedal (or possibly N-pedal) robot that moved roughly similarly to how humans or dogs or cats or horses run, and it was faster than humans over all the terrains that humans can run over, I'm absolutely certain that everyone would agree that the robot is a better runner.

But a car is not that thing. Neither is a helicopter, or a train, or a bicycle, or a jet aircraft or a hang glider or a skateboard.

Running and plowing are not simply about doing it as fast as possible or as extensively as possible.

So maybe what you are looking for is a definition of "better", it depends on what you mean.

In my book a tailor made suit is always better than a machine made suit, because people are better tailors than machines for some definition of better.

In my experience, most arguments are the result of people not agreeing on what a word means.

It is this thinking of the past that will get large numbers of us in trouble with our future machines.

That's a pointless argument. We might not say it, but for all intents and purposes the car does go faster than any human runner.

We just don't say it because running when it comes to humans mainly means using your feet. If it was a more generic term, like "fast-mover", we could still use it to compare humans and cars, and say cars are better "fast-movers" than humans.

Not that important, and not for this purpose. Things still work the same, even in languages with widely different semantics and ways to refer to them (I don't mean the trivial case where a house is called talo in Finnish etc., but languages where semantics and terms differ.

Using language-specific (en. english specific, or german specific) word definition and etymology to prove some property of the thing reffered to is an old cheap philosophical trick that sounds more profound than it is insightful.

Even more so, we might not say it for a car, but if we've built a human-looking robot with legs, we'd very much say it's a "better runner" if it started surpassing humans at running. Hell, we used to call employees doing manual calculations "calculators" in the past. Later, when machines doing that became available, we used the same term for them.

So the idea that "human is runner but car is not runner", also means that "human is thinker, machine is not thinker", and this has some profound difference, doesn't make sense anyway. Human running is associated with legs, certain way of moving, etc. Thinking is more abstract and doesn't have such constraints.

>Cars are not runners.

That's just an accidental property of having a dedicated word for "runner" in English that doesn't also apply to a car going fast. The term "running" though is used for both a human running and a car going fast ("That car was running at 100mph").

>"For all intents and purposes" is a cop out here.

For all intents and purposes means "in practice". Any lexicographical or conceptual arguments don't matter if what happens in practice remains the same (e.g. whether we decide an AGI is a "thinker" or a "processor" or whatever, it will still be used for tasks that we do via thinking, it will still be able to come up with stuff like ideas and solutions that we come up via thinking, and effectively it will quak, look, and walk like a duck. The rest would be semantical games.

>We're talking about LLMs, you know language learning models.

Which is irrelevant.

LLMs being language learning models doesn't mean the language used to describe them (e..g "thinkers" or not) will change their effectiveness, what they're used for, or their ability to assist or harm us. It will just change how we refer to them.

Besides, AI in general can go way beyond LLMs and word predictors, eventually fully modelling human neural activity patterns and so on. So any argument that just applies to LLM doesn't cover AI in general or "the danger than AI destroys us" as per TFA.

> “This idea of surpassing human ability is silly because it’s made of human abilities.”

It's not made OF human abilities, it's made BY human abilities - a completely different thing.

And, of course, Boston Dynamics will be delivering the "better runner" very soon.

We would if the car was to race against human runners. It's just word play. Cars are not used like runners, so we use different words. They definitely are better runners.

Now that technology is touching our core business we get scared, but this has been going on for a long, long time. When it was our legs, we brush it off. But when it touches our ability to think we squirm.

Cars go faster than humans can by themselves, under some specific conditions.

Cars go slower than humans, or rather cannot go at all, under other specific conditions. Two weeks ago my wife ran 30 miles on trails in southern Texas. A car could not have traversed any of the distance she travelled on, because a car cannot run.

Cars make it easier for people to move themselves and stuff when there are appropriate roads to travel on. They have enhanced our abilities to do this, but they cannot run.

You're squashing the meaning out words by trying to suggest that "running" is somehow equivalent to "any other method of a person moving from A to B". But that's not true.

We can acknowledge the greater ease of cars for moving people and stuff without squashing the meaning out of words.

Finally, even the notion that cars are "better" at moving people and stuff needs careful examination. Thus far I have said "make it easier" because I am aware that by a certain set of metrics (related to energy use, material use, impact on the environment) cars are actually worse most of the time.

That's just an accidental property of the english language.

We can imagine a language where "runner" and "thing that moves from A to B fast" used the same term T, and if people referred to T with the english notion of "runner" (e.g. a person running in a marathon") it was just deduced from the context. There are many cases like that.

In any case, the point is moot, as "thinking" doesn't have the same constraints. We might not call what a car does as running/runner (though we do use the former term) but we absolutely have considered AI as "thinking" and called AIs "thinking machines", even before AI (never mind AGI) even existed.

>You're squashing the meaning out words by trying to suggest that "running" is somehow equivalent to "any other method of a person moving from A to B". But that's not true.

This depends on the level of abstraction of the discussion. At some level of abstraction it's irrelevant if the move happened via running or via horse buggy or via a car. Sometimes we just care about the act of moving from A to B, and different methods to do so are only differentiated by their speed or other effectiveness.

In that case, we can compare man and machine though, and just care for the speed (machine can answer in 0.1. secs, a man needs to think over 1-2 minutes to answer such questions) or effectiveness (e.g. machine is better at juggling many things at the same time when thinking, or man is better at subtle semantical nuance).

This isn't an accident of language. An example of an actual accident of language would be giving tanks instead of giving thanks.

Are runners runners? Yes, according to you. A walker is a runner is a missile is a bowling ball rolling between places is light moving through a medium. No, according to a fitness coach, because a runner is not a tank is not a plane. When they say that a person should take up running they don't mean the person should melt down their body in a furnace and sprinkle their atoms into metal which is then pressed into iron plates that are attached to a tank which will then go running.

Sometimes we need to be careful in language. For example, we probably don't want to confuse the process of being incinerated and pressed into iron plates with the process of a human exercising their muscles. The choice to be careful in this way is not an accident of language. It is a very deliberate thing when, for example, John Von Nuemann carefully explains why he thinks the laymen use of the word game has perilous impact on our ability to think about the field of game theory which he starts in his book about the same.

I think you should make your point so as to disprove Nuemann, not pick on the straw man of running. Or you should argue against the use of the term radiator instead of car parts. It will better highlight your fallacy, because with running I have to make your position seem much more farcial then it is. We do gain something from thinking imprecisely. We gain speed. That can really get our thoughts running, so long as we don't trip up, but it calls to attention that when someone chooses to stop running due to the claim that the terrain isn't runnable, the correct response is not to tell them that running is accidental property. It is to be careful as you move over the more complicated terrain. Otherwise you might be incinerating yourself without noticing your error.

By "Accident of language" I don't mean "slip of the tongue" or "mistake when speaking".

I mean that kind words we use to describe someome who runs as "runner" is an accidental, not essential, property of English, and can be different in other languages. It doesn't represent some deeper truth, other than being a reflection of the historical development of the English vocabulary. I mean it's contigent in the sense it's used in philosophy as: "not logically necessary"

Not just in its sounds (which are obviously accidental, different languages can have different sounds for a word of the same meaning), but also in its semantics and use, e.g. how we don't call a car a "runner".

That we don't call it that doesn't express some fundamental truth, it's just how English ended up. Other languages can very well call both a car and a running man the same thing, and even if they don't for this particular case, they do have such differences between them for all kinds of terms.

>* I think you should make your point so as to disprove Nuemann, not pick on the straw man of running.*

I'm not here to disprove Neumann. I'm here to point that Lanier's argument based on the use of "runner" doesn't contribute anything.

You are arguing on the basis of possibility of imprecision in language that the choice to be more precise does not contribute anything. That structure - whether you want it to or not - as a direct consequence of logic applies to every thinker who ever argued for precision due to the possibility of ambiguity. It is an argument against formal systems, programming languages, measurement, and more. Some of the time it will turn out that your conclusion was true. Other times it will not. So the argument structure itself is invalid. Your conclusions do not follow from your premises.

Try your blade - your argument structure - against steel rather than straw. I saw you slice through straw with it. So I picked up the blade after you set it down and tried to slice it through steel. The blade failed to do so. The blade is cheap, prone to shattering, and unsuited for use in a serious contest between ideas.

For what it is worth - I do happen to agree with you that Lanier is making a mistake here. I think it is in the logical equivalence mismatch. He wants intelligence to be comparable to running, not to motion more generally, but since intelligence is actually more comparable to compression we can talk of different implementations of the process using terms like artificial or natural intelligence without being fallacious for much the same reason we can talk about different compression algorithms and still be talking about compression. So instead of trying to argue from his distinction between motion in general and motion in humans, I would think the place to point to for contradiction is the existence of cheetah runners versus human runners. Directly contradicting his insinuation is that we actually do say that cheetah are faster runners than humans.

Cars are an easier method to move people and stuff when there are suitable routes, where easier means "the journey will take less time, will require almost no human exertion by those moved, and will likely include weather protection".

Nobody is going to disagree with this (they may raise the objections I did that cars are energetically, materially and environmentally less efficient than other means, but that doesn't invalidate "cars are easier for moving people+stuff").

But that's not running. I will concede that even in English, there are idioms like "Can you run me to town?" meaning "Can you drive me to town?", or "I'm just going to run to the store" meaning "I'm going take a short journey to the store". But this doesn't mean that cars are better at running than humans, it means that the english word run can be used in different ways. And you know exactly which way Lanier meant it.

I think that's not the point. We're in awe by the machines' performances and then confused in how that compares to our abilities.

The actual threat is that in our minds we narrow our own capabilities and limit the comparison such that the computer is in fact better.

When computers were first doing math quicker than humans, that might have touch some humans, sure. Similarly now that "AI"s produce convincing spam faster or photo realistic creative images — that hurts some, jobs or maybe a lot. But it doesn't come close to being "human" or "intelligent".

Quite the opposite, the point is that we are getting dumber by focusing on human traits that can be measured or emulated by machines.

For example, when we say computers can "do math" more quickly than human beings can, this is fine as a matter of loose or figurative common speech. But strictly speaking, do computers actually do math? Do they actually compute? No, they don't. The computation we say a computer is doing is in the eye of the beholder. A better way to characterize what's happening is that human beings are _using_ computers _computationally_. That is, the physical artifacts we call computers participate in human acts as instruments, but _strictly speaking_, it makes about as much sense to say computers compute as it is to say that pencils write, hammers nail, vacuums cleaners clean, or cars drive. These things participate in the human act, but only as instrument. Whereas when human beings compute they are objectively computing, computation is not what computers are objectively doing (both Kripke and Searle make good arguments here). These artifacts only make sense in light of human intentions, as instruments of human intention and act.

Human writing can be viewed similarly. Objectively, we only have some pigment arranged on some material. No analysis of a piece of written text will ever divulge its signification. Indeed, no analysis of a piece of text will demonstrate that what is being analyzed is a piece of text! Text, and even that something is a piece of text, needs to be interpreted as text to function as text in the eye of the reader. But the semantic content of the text is objectively real. It just exists in the mind of the reader.

So we need to be careful because we can easily commit category mistakes by way of projection and confusion.

Cars are faster on reasonable traversable terrain. Are they more or less energy efficient? Under what circumstances? Do they self navigate the best path around obstacles? Better is really subjective.

And this applies to the large language models too. Just like calculators, they are going to do some things better, or maybe cheaper. But I've played with them trying to get them to write non-trivial programs, and they really do fail confidently. I suspect the amount of source code online means that any common problem has been included in the training data, and the LLM constitutes a program. So, at this point for programming, it's fancy Google. And that has value, but it is not intelligence.

I am not saying we (as a society) shouldn't be worried about these developments. Near as I can tell, they will mostly be used to further concentrate wealth among the few, and drive people apart because we already can't settle on a common set of (reasonably) objective facts about what is going on -- both problems are probably the same thing from different perspectives...

This tendency on HN to annihilate discussions by stating that, for instance, flying is the same as running because your feet also touch the ground at some point when flying (it happens only at take off and landing but it still counts as running, right ?) is really something. Stop torturing definitions, it makes Carmackgod sad and they randomly switch off a bit on the mainframe every time you do that.

You're actually oversimplifying the matter to a point where an F-16 Hornet is a very fast runner. Which it isn't.

Like a lot of the stuff being done with large models certainly isn't thinking, but they can clearly characterize sets of data in ways that an unassisted human can't.

being able to answer based on a probabilistic assumption is not that great in general, they do it fast on a frozen knowledge base, it can be useful, sometimes is surprisingly good, but not that great in general.

When I asked for the 3 best wood shops near me it replied with a shop that does not sell wood, a shop that does not exist and a broken website of a former now closed wood shop.

Now can an AI train another AI to become "smarter" than it is?

It can't.

Can an AI train another AI to become better at "characterize sets of data" than it is?

It can't.

An unassisted AI is as helpless as the unassisted person, but can't even rely on the intelligence of the species.

It’s not a search engine, if you give it the necessary tools it can use a search engine for you to find these answers.

We know that the current stuff can't do a lot of things, and it isn't really that interesting to enumerate them.

A predictive text model having a poor grasp of wood shop and location doesn't seem hugely surprising.

Focusing on the "how" feels like you'd arrive at "a calculator isn't as good at calculating as a human, because it doesn't do it the same way, it doesn't have a brain".

The analogy to running is flawed because rolling and running are different types of locomotion.

It's not at all clear that computing and thinking are meaningfully different forms of information processing. In fact, we know that we can compute by thinking since I can reduce lambda calculus terms in my head. We also know computers can compute all computable functions, and we know that all physical systems like the brain necessarily contain finite information (per the Bekenstein Bound), therefore they can in principle by simulated by a computable function. There are therefore strong reasons to suspect an underlying equivalency that would suggest that "artificial intelligence" is a sensible term.

> It’s like saying a car can go faster than a human runner. Of course it can, and yet we don’t say that the car has become a better runner.

He has a point and I believe his point is that it's a different type of intelligence. His view is more nuanced than how you are trying to frame it.

> This idea of surpassing human ability is silly because it’s made of human abilities

At some point in history we were just "chimp abilities", so the argument would become "it's silly to imagine that something made of chimp abilities could surpass chimp abilities".

Single-mode LLMs are made of human abilities, but we're already going to multi-modal, though with what I would call rather limited interconnections. What does a LLM that takes language and mixes that with sensor data from the real world? You're no longer talking about human abilities, you're going beyond that.

https://www.youtube.com/watch?v=9R5w4Qz6pVk

I don't want to imply anything, just a moment of having fun.

edit: wording

In general he should know what he's talking about when it comes to tech hype cycles ;)

And his unique perspective on AI is all the more valuable (and courageous) considering that Microsoft, recently laid off their AI ethics team. It's super important we don't let human considerations fall by the wayside in this rush. The potential of AI is limitless, but so are the potential risks.

A theoritical AI which thinks like a person, but (due to computing power) can think through and evaluate 1,000,000 ideas the time it takes a person to think through 10 of them, it already has surpassed human ability by a big margin. Same for memory capacity etc.

That the input the machine is trained on is the output created by "human abilities" is irrelevant to whether it can surpass human ability.

If your bet is on the former, how does it create an entirely new, irrational thought?

The AI's we are using now are nearly one dimensional when it comes to information. We are pretraining on text, and we're getting "human like" behavior out of them. They have tiny context windows when working on new problems. They have no connection to reality via other sensor information. They have no means of continuous learning. And yet we're already getting rather insane emergent behaviors from them.

What does multi-modal AI that can interact with the world and use that for training look like? What does continuous learning AI look like? What does a digital mind look like that has a context window far larger than the human mind ever could? One that input into a calculator faster than we can realize we've had a thought in the first place? One that's connected to sensory systems that span a globe?

We also have some pharmaceutical tricks to tweak up processing capabilities of the mind, so there's potentially no need to simulate. The capabilities of the big ball of sentient goop have not been plumbed yet.

Now imagine a technology that could obviate the need for sleep or maybe make it useful and productive.

Arguably humans are the same, being the product of genetics, epigenetics, and lived experience.

Current "machine learning"-style AI (even when it uses self-driven iteration, like the game playing systems) is the result of a few ideas across not much more than 100 years, and for the most part is far too heavily influenced by existing human conceptions of what is possible and how to do things.

Computers already do things that humans can't on a massive scale. It's not hard to imagine that they could leverage that ability to take AI beyond us.

Did you even read the article?

“This idea of surpassing human ability is silly because it’s made of human abilities.” He says comparing ourselves with AI is the equivalent of comparing ourselves with a car. “It’s like saying a car can go faster than a human runner. Of course it can, and yet we don’t say that the car has become a better runner.”

In fuller context, Lanier argues that software using AI won't make human sport or competition useless, because it will use different processes to achieve the same result—the same way that competitive running (or top-level chess, Go, or certain video games) will still happen, even if human inventions can beat the best human at the task.

For all these tasks, the software will take a different process for doing well at the task (e.g. a car doesn't "run," and a chess engine "thinks" differently than a human). In these activities, the process matters.

A different interpretation of the argument is then a bit more interesting. If Lanier is also saying that software using AI won't be better than humans at activities outside of competitions, I would disagree—though to be fair, I don't think this is his argument. For lots of work, the result matters more than the process. If someone wants to make a funny poem as a one-off joke in a story, the result may matter more than the process of production. And if a worker wants to summarize lots of short texts where speed is the most important factor, the result may also matter more than the process. In the same sense, it's still true that a car is usually better at letting humans travel over long distances for work than running, because the result matters more than the process.

The ML systems are not made of human abilities. They are made of software processes. Jared is a smart and informed guy but that sentence is just nonsensical.

It enters the Olympics and automates away all sprinters? Or it becomes self-aware and decides to eat all the humans?

Or more likely, it gets sold to the military so they can have one more kind of drone that runs on land instead of flies through the air.

It absolutely goes to the military with built-in weapons.

https://news.ycombinator.com/newsguidelines.html

As a pioneer and intellectual he's been arguing about the commodization of human knowledge for a long time, he's not simply saying that "machines won't surpass humans" and it's not accurate to describe him as someone who would say something like that.

Please take the time to research what he's published over the last 4 decades.

I wonder, how much will be enough?