Hacker News new | past | comments | ask | show | jobs | submit login
OpenAI has a new safety team – it's run by Sam Altman (theverge.com)
18 points by rntn 7 months ago | hide | past | favorite | 9 comments



Well, I suppose OpenAI's products will now be safe for Sam Altman and his goals.

I'm not so sure about the rest of us, though.

Anyway, life imitates art. I'm sure Dick Jones was on OCP's safety team, which is how he was able to add Directive 4 to Robocop's programming (https://robocop.fandom.com/wiki/Prime_Directives).


Conflict of Interest.

Shouldn’t such Safety teams be by definition independent?


Yes, and there should be a governance structure that makes sure the CEO is held accountable to the principles of... Oh... Oops. They've already burned that whole structure to the ground.


indeed. and they should have "rip-cord" or "fire alarm" abilities to halt any activity that's potentially dangerous.

cue the old story about how any Toyota line worker could and should slam the stop button if they saw anything unsafe, and even if they stopped the line for a false positive, they shouldn't be punished -- because you want people calling out anything that even seems risky without fear of reprisal.


It's about time OpenAI evaluate their corporate safeguards and remove any barriers to maximizing profits.


The problem is the overloading of the word "safety", not really the approach. If there was some mortal danger stemming from this yeah, maybe more objective oversite is needed. But it's a stunt anyway and there is no danger involved, so whatever.


First, I think you correctly put the word in sarcasm quotes. OpenAI now has a "Safety" team.

But second, there definitely can be risk of blood and treasure in LLM use. Can the LLM reach out and stab the user... No. Can the LLM "hallucinate" and generate advice that when acted on has harmful results. Obviously yes. Now the moment you say "So you shouldn't just act on the advice of the LLM" I have to ask, then what's the point of having one?

I'm using loose phrasing here, so let's be more concrete. When Google's Gemini recommended that a user open the door on the back of the camera to tug the film loose if it wouldn't advance, this was a recommendation that would cause the loss of all photos on the roll of film. Small loss of value. But when a user gets similar "advice" on questions about how to clean something where the advice leads to adverse chemical reactions in the material, or worse, medical advice that leads to injury there is some responsibility on Google's part, or on OpenAI's part for having provided a faulty product.

Part of the issue here is we're attempting to replace search engines with answer engines, but answer engines require understanding and these LLMs just aren't there yet. They're fine for entertainment, but most users out in the world don't think of these systems that way.


I can still get ChatGPT to tell me how to make a bomb so yeah, can't wait to see the safety progress I guess.


Putting the fox in the chicken coop?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: