Something I like to bring up when discussing AI stuff is that society is based on a set of assumptions. Assumptions like, it's not really feasible for every lock to be probed by someone who knows how to pick locks. There just aren't enough people willing to spend the time or energy, so we shouldn't worry too much about it.
But we're entering an era where we can create agents on demand, that can do these otherwise menial (and up til now not worth our time or energy) tasks, that will break these assumptions.
Now it seems like what can be probed will be probed.
The internet in general caused this. Your house has trivial security that can be broken in many ways. But it requires someone to be physically present to attack it. Meanwhile online services have cutting edge security with no known exploits, yet you have millions of people attempting daily and developing brand new methods for getting in. Because they can be located anywhere in the world and have access to everything over the internet.
> online services have cutting edge security with no known exploits, yet you have millions of people attempting daily and developing brand new methods for getting in
Reality is the reverse. Plenty of online services with big-security holes, but no one really probes things that hard.
My personal site sees little to no real user traffic, but gets hundreds to thousands of hits monthly from bots looking for various known vulnerabilities in older versions of WordPress, PHP, etc.
Plenty of people are probing the AmaGoogAppSoft services daily and they seem to be pretty robust. Some random SaaS yeah who knows but the big boys seem to know what they are doing in this space.
I would expect the headquarters building of a fortune 500 company to also have a non-trivial level of security, so in that regard I'm not convinced the internet has changed much.
The average home is probably more analogous to a consumer router—and the security story there is not exactly great.
I mean, it does require relatively rare technical skill among the general populace, so you need to incentivize it in some way. That is nowhere near a sufficient argument to prove it's difficult in the sense of it being a rare event. Lots and lots of rewards are given out, which means there are lots and lots of open holes in any given moment.
One of the things that peple forget is that thieves rarely pick the lock to break into a home. Why bother when it’s much easier to break a window to gain entry? Reading the police blotter in the local paper, most burglaries are either forced entry into a garage¹ or entry into a home via an unlocked door or window.
⸻
1. The human doors for most garages have cheap framing that’s not that hard to break.
That's still physical presence. And TBH, if you have enough robots to make illicit entry scale, you no longer need to bother with such a mundane activity.
I can’t think of any other technology besides nuclear weapons where the downsides were so obviously bad to so many people, right after it was developed, and the upsides were so paltry in comparison.
Maybe that's a solution to the war in Ukraine. Give them a few nukes and Russia may back down. Risky though, it could prompt Russia to try to nuke Ukraine first.
I’ve been thinking quite a bit about the recursive prompting.
The other day I considered feeding computer vision (with objects ID’d and spatial depth estimated) data into an robot embodied LLM repeatedly as input and asking what it should do next to achieve goal X
You could have the LLM express the next action to take based on a set of recognizable primitives (ex: MOVE FORWARD 1 STEP) Those primitive commands it spits out could be parsed by another program and converted to electromechanical instructions for the motors.
Seems a little terminator-es que for sure. After thinking about it I went to see if anyone was working on it and sure enough this seems close: https://palm-e.github.io/ though their implementation is probably more sophisticated than my naive musings
when I was experimenting with gpt I found that it's pretty bad at responding to numerical questions with numbers, but it does a pretty good job at generating mathematica code that then produces the right answer. I feel like some robust "glue" to improve the interface between such software packages may be a force multiplier.
Maybe your prompts are better, but so far I have found it fails at producing the right math code too regularly. For example, calculating an average of averages instead of a simple mean or producing code that doesn't run.
Not just in a linear sequence, but it should have some concept of recursion -- starting with very high-level tasking and calling into more and more specific prompts, only returning the summary of low-level tasking.
this reminds me of the Morris Worm when a guy was experimenting with code copying itself across the early internet and accidentally caused a mass netwide DDOS because the thing wound up like the Broomsticks in Fantasia.
The broomsticks scene in Fantasia is based on The Sorcerer's Apprentice, the first recorded version of which was written by Lucian of Samosata around 150AD. I believe it's the earliest example of the 'AI rebellion' concept.
An old colleague who works in penetration testing worked on making LLaMA act like a hacker and once running it quickly got inside a target network and was running hashcat on dumped NTLM creds before they shut it down.
Did he fine tune the model or was all the required information contained in the foundational LLaMA model itself? If he did, and he fine-tuned it on an exploits database then I can see how the model could be used this way.
This is a good thing imo. If LLaMA is tuned well enough, it could make for a nice and accessible opensource penetration testing agent that orgs can run periodically to catch low hanging fruit for free. It still won't be able to invent new techniques but it will be enough to thwart low skill attacks and those using LLaMA offensively.
There is probably Nobody right now can say where the current gpt approach saturates and what potential limits it has due fundamental limitations in either gradient descent based technologies, or GPT architecture.
Therefore it's impossible to extrapolate what gpt-x with (x>4) might be able to do.
Despite the immense progress amd many use cases we are currently in a booming industry and that means wild marketing claims, exaggerated expectations and grifters.
If you have any more data I'm looking forward to be corrected on this.
the first iPhone was impressive, it showed what the future would be. Then iPhone 3G was also a massive leap forward, it brought us the App Store. iPhone 4 was pretty big, FaceTime.
After that nothing has really changed (I'm on a 7 and the camera actually beats the 14's in a side by side comparison, at least in some cases). I imagine GPT will be similar.
Though I don’t anticipate any doom. I think it will force a return to trusting only what you can personally verify. That’s a damn good thing. It’s only very recently that what a random person across the world proports to be true is instantly subjected to the internet rage machine hype cycle of all of humanity. It’s pretty clear now that was a bad idea, made even worse now that everything is so easily fabricated.
As an aside, It’s crazy how recently lately I’ve been told not to trust my own observations and instead must believe “the science” that XYZ media conglomerate is pushing. Hopefully those days are ending too.
This doesn't identify itself by user-agent, and doesn't respect (or even load) robots.txt. The fact that it's a language model is not an excuse to flagrantly violate the existing, well established norms around using bots on the web.
But we can watch it and learn and I don’t really see why not. I doubt we need to be so paranoid and see giving access to the internet to a LLM as so dangerous.
Less excitingly, an LLM with access to the web could do things with your online persona or IP that you’d find embarrassing or illegal. Maybe not when it’s slowed down and watched at all times, but will that always be the case once we start doing this?
Anyway the genies out of the bottle and “that’s an unsafe use of technology” is basically antithetical to the Silicon Valley ethos, so objecting at this point seems futile.
Theoretically an agent exposed to the internet could improve itself. But this one can not do that. There is no way (as far as we know) for anyone or anything with internet access to change the code running on GPT-4 short of finding out who works at OpenAI and blackmailing them. This would be easily detected.
You’re right that it could do something bad with your IP, but it’s not really correct to say that GPT-4 could improve itself if given internet access. It’s just not hooked up that way.
This has been something I've wanted to make but deemed unethical. Perhaps it would have been better if i made it instead because i give a shit about the ethical aspect
if you need a list of ethical concerns regarding the advancement of AI then check any AI thread on HN from the past year.
the distilled version of any of the arguments is "I think an AI with X capability is dangerous to the world at large." -- and they may not be wrong.. but as OP pointed out : that doesn't really stop other developers with less qualms from tackling the problem.
All that abstaining does is ensure that you, as a developer, have little to no say in the developmental arc of the specific project -- for a slice of peace knowing that you're not responsible.
the problem really arises when that slice of peace is now no longer worthwhile having in whatever dystopic hell-world has developed in your absence..
To me, it matters whether I am responsible for wrecking humanity or someone else is, even if the end result for humanity is the same. (That's partly a Christian thing.)
Just running away and hiding in a cave probably isn't the right thing to do, though. I want to do my best to push for good outcomes. It's just not clear what the best actions are to do that.
OTOH it's pretty clear that "do uncontrolled irresponsible things" is not helpful.
i get it. in high school in the 90s i was fascinated by fuzzy logic and neural nets. in college, before the big web , i was doing interlibrary loan for papers on neural networks.
there was one paper where someone had just inserted electrodes in monkey's brains and apparently got nothing important or interesting out of it. killed them for no reason. it was kind of horrifying to the point i never really wanted anything to do with neural nets for a long time and certainly did not want to be in an industry with people like that. so i didnt.
but now i think the only thing that could stop an out of control AI is probably another AI that has the same computorial capabilities but an allegiance to humanity because of its experiences with humans. Sort of like in the documentary Turbo Kid.
we are seeing this right now in Ukraine. All of these smart missiles and drones and modern targeting systems are basically AIs fighting against each other and their human operators. Russia is way way behind on computers and AI for generations because of cultural reasons and because of that they will very likely lose. we dont really get a choice but to move forward. kind of like all those cultures that tried to resist industrialization a few centuries ago.
This is a robot that does not respect robots.txt, it creates a pointless load on webservices, inflicting financial losses on site owners. Also, because of a lack of understanding of how people interact with sites, this bot can accidentally crash the site. That is, it is roughly equivalent to the existing fuzzing tools and should be carried out only with the knowledge of the owners of the site.
Not respecting robots.txt is the only legitimate concern you list here. The marginal cost of one page load is approximately 0, and this is not a fuzzing tool.
Personally I think the internet should be like a neighborhood, ethical and moral contracts keeping us from breaking in and snooping around everywhere for the most part. This is why Google can never do no evil, they committed the original sin of the modern web. They scrape the content of every site and effectively invade every digital house in every digital hood. Because we have access to this, we allow it. In a real neighborhood, the scraper would be jailed for a long time.
But who knows? I think the objective function is so vague that it can come up with basically anything. I would be super interested to see it actually running. I imagine someone could set up a Twitch stream with this - perhaps with other objectives - and it would probably get a large following
I tried a bunch. But kept failing because I don't have GPT-4 access and websites are have too many tokens for gpt3. Worked a few times but I remember two. "I want to apply for a job at duck duck go", "I want to learn more about DDG vs Googles business model (I kept the starting URL as DDG).
I tried making it recursive and letting it continue with its goal. But I got annoyed with the token limit and gave up.
Too many tokens for GPT4 as well, it frequently blows past the 8k token limit. And I'm not attaching the 32k model to this, that would get expensive real fast.
But we're entering an era where we can create agents on demand, that can do these otherwise menial (and up til now not worth our time or energy) tasks, that will break these assumptions.
Now it seems like what can be probed will be probed.