Here's Nathan Labenz on how sketchy the red-teaming process for GPT4 was. Nathan states that OpenAI shut him out soon after he reached out to the board to let them know that GPT4 was a big deal and the board should be paying attention: https://nitter.net/labenz/status/1727327424244023482#m [Based on the thread it seems like he reached out to people outside OpenAI in a way which could have violated a confidentiality agreement -- that could account for the shutout]
My suspicion is that there was a low-level power struggle ongoing on the board for some time, but the straw that broke the camel's back was something like Nathan describes in his thread. To be honest I don't understand why his thread is getting so little play. It seems like a key piece of the puzzle.
In any case, I don't think it would've been right for Helen to say publicly that "we hear GPT-5 is lit but Sam isn't letting us play with it", since "GPT-5 is lit" would be considered confidential information that she shouldn't unilaterally reveal.
So what is Nathan Labenz saying? That GPT-4 is dangerous somehow? It will get many people out of jobs? MS Office got all the typists out of jobs. OCR and Medical Software got all the medical transcriptionists out of jobs. And they created a lot more jobs in the process. GPT-4 is a very powerful tool. It has not a whiff of AGI in it. The whole AGI "scare" seems to be extremely political.
Good planning necessarily reduces the cost of something relative to the unplanned or poorly planned version. If it identifies a non-obvious means of assassination that is surprisingly easy, then it has done something "close enough" to reducing the cost to $0.
This is a piece of software. What would "totally amoral" even mean here? It's an inanimate object, it has no morals, feelings, conscience, etc... He gives it an amoral input, he gets an amoral output.
Amoral literally means "lacking a moral sense; unconcerned with the rightness or wrongness of something." Generally this is considered a problem if you are designing something that might influence the way people act.
In both of your examples, the result of "immoral inputs" is immediate tangible harm. In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.
> In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.
Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator.
An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems.
Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.
Why would anyone give control of air traffic or weapons to AI? That's the key step in AGI, not some tech development. By what social process exactly would we give control of nukes to a chatbot? I can't see it happening.
> Why would anyone give control of air traffic or weapons to AI?
Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept.
> a chatbot.
Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits.
> I can't see it happening.
Because, it already happened.
Turkey is claimed to use completely autonomous drones in a war [1].
South Korea has autonomous sentry guns which defend DMZ [2].
We give hall passes to more than AI. We give passes to humans. We could have a detailed discussion of how to blow up the U.S. Capitol building during the State of the Union address. It is allowed to be a best selling novel or movie. But we freak out if an AI joins the discussion?
You persist in using highly misleading analogies. A military F-16 comes with missiles and other things that are in and of themselves highly destructive, and can be activated at a push of a button. An LLM does not - you'd have to acquire something else capable of killing people first, and wire the LLM into it. The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles.
Speaking of which, it's perfectly legal for a civilian to own a fighter plane such as an F-16 in US and many other countries. You just have to demilitarize it, meaning no weapon pods.
>The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles.
The reason this isn't an issue in practice is because such repurposing would require significant intelligence/electrical engineering skill/etc. The point is that intelligence (the "I" in "AI") will make such tasks far easier.
>Ten-year-old about to play chess for the first time, skeptical that he'll lose to Magnus Carlsen: "Can you explain how he'll defeat me, when we've both got the same pieces, and I move first? Will he use some trick for getting all his pawns to the back row to become Queens?"
The point here is that is giving the user detailed knowledge on how to harm others. This is way different than a gun where you are doing the how (aiming and pulling the trigger).
The guy says he wanted to slow down the progress of AI and GPT suggested a detailed assassination plan with named targets and reasons for each of them. That's the problem.
Thing is, this detailed knowledge is already available and much easier to acquire. There are literally books on Amazon that explain how to jury-rig firearms and bombs etc. Just to give one example: https://www.amazon.com/Improvised-Munitions-Handbook-TM-210/....
When it comes to those "detailed plans", if you actually try something like that, what you get from it is a very broad outline that is pretty much a mish-mash of common sense stuff and cultural tropes (many of which aren't true IRL). Similarly, the "list of targets" that it makes is simply the most prominent people associated with that area in public perception, not necessarily people who are actually key. All of this can be achieved just as well with a few Google searches, and the resulting plan will likely be much better for it.
I've yet to see any example where GPT would come up with something along these lines that is not trivially found on the Internet anyway.
I mean, there's a sense in which my mind is software that's being run by my brain, right? Yet that doesn't absolve me of moral responsibility.
In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.
Completely depends on your morality. I'm pretty sure there are some libertarians out there who think the most basic version of the second amendment includes owning F16 with live weapons.
Sure, but idiots are a thing and the intersection of the sets of libertarians who may believe that and idiots is hopefully empty but it may not be so such outsized power is best dealt with through a chain of command and accountability of sorts.
Sure -- if you're a libertarian who thinks it should be possible to purchase an F16 without a background check, that seems consistent with the position that an amoral GPT-4 should be broadly available.
What kind of background check do you think exists when buying a fighter jet?
It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.
There's been a fair amount of research into hooking up LLMs with the ability to call APIs, browse the web, and even control robots, no? The barrier between planning and doing is not a hard one.
As for cost and risk -- ask GPT-5 how to minimize it. As Nathan said in his thread, it's not about this generation, it's about the next generation of models.
I really don't know how to interpret that link, but I think there is a lot we don't understand which is going on in those billions of parameters. Understanding it fully might be just as hard as understanding the human brain.
I'm concerned that at some point in the training process, we will stumble across a subset of parameters which are both self-aware and self-interested, too. There are a lot of self-interested people in the world. It wouldn't be surprising if the AI learns to do the sort of internal computation that a self-interested person's brain does -- perhaps just to make predictions about the actions of self-interested people, at first. From there it could be a small jump to computations which are able to manipulate the model's training process in order to achieve self-preservation. (Presumably, the data which the model is trained on includes explanations of "gradient descent" and related concepts.)
This might sound far-fetched by the standard of the current model generation. But we're talking about future generations of models here, which almost by definition will exhibit more powerful intelligence and manifest it in new unexpected ways. "The model will be much more powerful, but also unable to understand itself, self-interest, or gradient descent" doesn't quite compute.
The image is OCR'ed and that data is fed back into the context. This is no more interesting or indicative of it passing the mirror test than if you had copy and pasted the previous conversation and asked it what the deal was.
I can think of several ways that AI assistance might radically alters both attack and bodyguard methods. I say "might" because I don't want to move in circles that can give evidenced results for novel approaches in this. And I'm not going to list them for the same reasons I don't want an AI to be capable of listing them: while most of the ideas are probably just Hollywood plot lines, there's a chance some of them might actually work.
Except the reason why we dont all just killed each other yet have nothing to do with risk or cost of killing someone.
And everything LLM can come up with will be exactly the same information you can find in any fiction detective book or TV series about crime. Yeah very very dumb criminal can certainly benefit from it, but he can as well go on 4chan and ask about assassination there. Or on some detective book discussion club or forum.
> Except the reason why we dont all just killed each other yet have nothing to do with risk or cost of killing someone
Most of us don't want to.
Most of those who do, don't know enough to actually do it.
Sometimes such people get into power, and they use new inventions like the then-new-pesticide Zyklon B to industrialise killing.
Last year an AI found 40k novel chemical agents, and because they're novel, the agencies that would normally stop bad actors from getting dangerous substances, would generally not notice the problem.
LLMs can read research papers and write code. A sufficiently capable LLM can recreate that chemical discovery AI.
The only reasons I'm even willing to list this chain, is that the researchers behind that chemical AI have spent most of the intervening time making those agencies aware of the situation, and I expect the agencies to be ready before a future LLM reaches the threshold for reproducing that work.
Everything you say does make sense, except those people who able to get equipment to produce those chemicals and have funding to do something like that - they dont really need AI help here. There are plenty dangerous chemicals already well known to humanity and some dont actually take anything regulated to produce "except" complicated and expensive lab equipment.
Again difficulty of production of poisons and chemicals it's not what prevent mass murdering around the globe.
Complexity and cost are just two of the things that inhibit these attacks.
Three letter agencies knowing who's buying a suspicious quantity from the list of known precursors, that stops quite a lot of the others.
AI in general reduces cost and complexity, that's kind of the point of having it. (For example, a chemistry degree is expensive in both time and money). Right now using an LLM[0] to decide what to get and how to use it is almost certainly more dangerous for the user than anyone else — but this is a moving goal, and the question there has to be "how to we delay this capability for as long as possible, and at the same time how do we prepare to defend against the capability when it does arrive?"
[0] I really hope that includes even GPT-4 before the red-teaming efforts to make it not give detailed instructions for how to cause harm
I dont disagree with his points, but you completely miss the point of my post. People dont need an AI advise to commit crime and kill others. Nonestly humans they're pretty good at it using technology of 1941.
You don't have bunch of cold blood killers going around not just because police is so good and killers are dumb and need AI help. It's because you live in functioning state where society have enough resources so people happy enough to instead go and kill each other in Counter Strike or Fortnite.
I totally agree that AGI could be a dangerous tech, but it's will require autonomity where it can manipulate real world. So far GPT with API access is very far from that point.
If you have ChatGPT API access you can have it write code and bridge that to other external APIs. Without some guard rails an AI is like a toddler with a loaded gun. They don't understand the context of their actions. They can produce dangerous output if asked for it but also if asked for something else entirely.
The danger also doesn't need to be an AI generating code to hack the Gibson. It could also be things like "how do I manipulate someone to do something". Asking an AI for a marketing campaign isn't necessarily amoral. Asking it how to best harass someone into committing self-harm is.
This is where I have issues with OpenAIs stated mission
I want AI to be amoral, or rather I should say I do not want the board of OpenAI, or even the employee of OpenAI choosing what "moral" is and what is "immoral" especially given that OpenAI may be superficially "diverse" in race, gender, etc, but they sure as hell are not politically diverse, and sure has hell do not share a moral philosophy that is aligned with the vast majority of the population of humanity given the vast majority of humanity is religious in someway and I would guess the majority of OpenAI is at best agnostic if not atheist
I do not want a AI Wikipedia.... aka politically biased to only 1 worldview and only useful for basic fact regurgitation like what is the speed of light
It seems that silicon valley is developing not a conscious sentient piece of software rather a conscience, a moral compass is beginning to emerge.
After giving us Facebook, insta, Twitter, and ego-search, influencing many people negatively, suddenly there are moral values being discussed amongst those that decide our collective tech futures.
AI will have even more influence on humankind and some are questioning the morality of money (hint: money had no morals).
Medical transcriptionists out of jobs? As far as I'm aware, medical transcription is still very much the domain of human experts, since getting doctors to cater their audio notes to the whims of software turned out to be impossible (at least in my corner of the EU).
My mom had to do this for her job and apparently some of the docs are so mumbly they have to infer a lot of the words from context and type of procedure but there is a lot of crossover everywhere so it depends a lot on which doc is mumbeling what. And yes you need special training for it (no medical degree though)
1) be surprised by and unprepared for AGI and every step on the path to that goal, or
2) have the developers of every AI examine their work for its potential impact, both when used as intended and when misused, with regard to all the known ways even non-G AI can already go wrong: bugs, making stuff up, reward hacking, domain shift, etc.; or economically speaking how many people will be made unemployed by just a fully [G]eneral self-driving AI? What happens if this is deployed over one year? Do existing LLMs get used by SEO to systematically undermine the assumptions behind Page rank and thus web search?; and culturally: how much economic harm do artists really suffer from Diffusion models? Are harms caused by AI porn unavoidable thanks to human psychology, or artefacts of out milieu that will disappear as people become accustomed to it?
There's also a definition problem for AGI, with a lot of people using a standard I'd reserve for ASI. Also some people think an AGI would have to be conscious, I don't know why.
The best outcome is Fully Automated Luxury Communism, but assuming the best outcome is the median outcome is how actual Communism turned into gulags and secret police.
Did they really create a ton more jobs? The past few rounds of industrialization and automation have coinved with plagues/the black death that massively reduced the population, mass agitation, increasing inequality, and recently a major opioid epidemic in regions devastated by the economic changes of globalization and industrialization. I think these tools are very good and we should develop, I also think it's delusional to think it'll just balance out magically and dangerous to expect our already failed systems to protect people left behind. Doesnt exactly look like they worked any of the previous times!
It's remarkable that the old board is the side characterized as willing to blow up the company, since it was Altman's side who threatened to blow it up. All the old board really did was fire Altman and remove Brockman from the board.
It sounded like they would if they could (for instance trying to sell to Anthropic or instating a "slow it way down" CEO), but they even failed at that. Not an effective board at all.
Exactly. It sounds like those board members themselves were acting in the interest of profit instead of the "benefit all humanity" mission stuff, no different than Altman. If anything then, the only difference between the two groups is one of time horizon. Altman wants to make money from the technology now. The board wants to wait until it's advanced enough to take over the world, and then take over the world with it. For the world's benefit, of course.
