I can't believe all these qualia questions have not evolved in centuries (or at least, the common discourse arond them hasn't). We all have similar rods and cones in our eyes. We have common kinds of color blindness. What other reasonable conclusion is there but that my red is your red? All the machinery is similar enough.
I suppose it's because people associate so much of who they are to the subjectivity of their experience. If I'm not the only one to see and taste the world as I do, am I even special? (The answer is no, and that there are more important things in life than being special.)
I see that you're not alone in your position clearly, but still, this is such a strange take to me. Do people not seriously not see, nay, instinctively understand the ontological difference between the difference between using code someone no longer understands and deploying code no one ever understood?
I'm not saying the code should be up to any specific standards, just that someone should know what's going on.
I don't actually see the difference. If someone writes the code and understands it but then becomes unavailable, what's the practical difference with no one having understood it?
Someone at some point had a working mental model of the code and a reputation to protect and decided that it was good enough to merge. Someone vetted and hired that person. There's a level of familiarity and history that leads others to extend trust.
The way I see it is that LLMs and humans are not inherently different. They are simply on different segments of a very complex spectrum of sensory input and physical output. Over time this position on the spectrum changes, for both LLMs and humans too.
With this in mind, it's all matter of what are your metrics for "trust". If you are placing trust on a human employee because it was hired, does this mean the trust comes from the hiring process? What if the LLM passed went through that too?
About familiarity and history: we are at the point were many people will start working at a new place were the strangers are the humans, you will actually be more familiar and history with LLM tools than actual humans, so how do you take that into consideration?
Obviously this is a massive simplification and reduction of the problem, but I'm still not convinced humans get a green checkmark of quality and trust just because they are humans and were hired by a company.
> Someone at some point had a working mental model of the code and a reputation to protect
This isn’t always true in absolute terms. In many places and projects it’s about doing a good enough job to be able to ship whatever the boss said and getting paid. That might not even involve understanding everything properly either.
Plenty of people view software development as just a stepping stone to management as well.
With reading enough code it becomes apparent that the code quality that AI generates will often be similar or better to human developers, even if the details and design are sometimes demented.
You could never have the same amount of trust in LLM-generated code as in a human developer, even if you wrote a large amount of tests for it. There will always be one more thing that you didn't think to test. But the many reports of massive security holes in AI coding tools and their products show that nobody even bothers with testing. Neither the vendors nor the users.
No. If that interstitial is working, it's only working due to obscurity, and the moment this system becomes even slightly popular it'll become worthless.
Proof of work is not a viable defense -- it's basically impossible to tune the parameters such that the cost is prohibitive or even meaningful to the scrapers but doesn't become an obstacle to users.
It's pretty much just a check for whether the client can run JavaScript. But that's table stakes to a scraper. Trying to discriminate between a real browser, a real browser running in headless mode, or something trying to fake being a real browser requires far more invasive probing of the browser properties (pretty much indistinguishable from browser fingerprinting) and obfuscating what properties are being collected and checked.
That's already what any commercial bot protection product would be doing. Replicating that kind of product as an on-prem open source project would be challenging.
First, this is an adversarial abuse problem. There is actual value in keeping things hidden, which an open source project can't do. Doing bot detection is already hard enough when you can keep your signals and heuristics secret, doing it in the open would be really hard mode. (And no, "security by obscurity is no security at all" doesn't apply here. If you think it does, it just means you haven't actually worked on adversarial engineering problems.)
Second, it's an endless cat and mouse game. There's no product that's done. There's only a product that's good enough right now, but as the attackers adapt it'll very quickly become worthless. For a product like this to be useful it needs constant work. It's one thing to do that work when you're being paid for it, it's totally another for it to be uncompensated open source work. It'd just chew through volunteers like nobody's business.
Third, you'll very quickly find yourself working only in the gray area of bots that are almost but not quite indistinguishable from humans. When working in that gray area, you need access to fresh data about both bot and real user activities, and you need the ability to run and evaluate a lot of experiments. Not a good fit for on-prem open source.
From what I gather the idea for Anubis isn't to _stop_ bots, it's to make them slow down enough to not bring down servers.
Like they said in the presentation, git(lab/tea) instances have insane amounts of links on every page and the AI crawlers just blindly click everything in nanoseconds, causing massive loads for servers where normally there might be a maybe a few thousand git pulls/pushes a day and a few hundred people clicking on the links at a human pace.
Plus the bots are made to be cheap, fast and uncaring. They'll happily re-fetch 10 year old repositories with zero changes multiple times a week, just to see if they might've changed.
Even a if the bad proof of work requires the bots to slow down their click rate, it's enough. If they somehow manage to bypass it completely, then that's a problem.
Formatting is like a dot on the i; there is 200 other small details that are just completely off putting to me :
- naming conventions (ias are lazy and tent to use generic names with no meaning) such as "Glass" instead of "GlassProduct" ;
- error management convention
But the most troublesome to me is that it is just "pissing" out code and has no after-tough about the problem it is solving or the person it is talking to.
The number of times I have to repeat myself just to get a stubborn answer with no discussion is alarming. It does not benefit my well-being and is annoying to work with except for a bunch of exploratory cases.
I believe LLM are actually the biggest data heist organized. We believe that those models will get better at solving their jobs but the reality is that we are just giving away code, knowledge, ideas at scale, correcting the model for free, and paying to be allowed to do so. And when we watch the 37% minimum hallucination rate, we can more easily understand that the actual tough comes from the human using it.
I'm not comfortable having to argue with a machine and have to explain to it what I'm doing, how, and why - just to get it to spam me with things I have to correct afterwards anyway.
The worst is, all that data is the best insight on everything. How many people ask for X ? How much time did they spend trying to do X ? What were they trying to achieve ? Who are their customers ? etc...
It is supposed to follow that instruction though. When it generates code, I can tell is to use tabs, 2 spaces, etc. and the generated code will use that. It works well with Claude, at least.
The shoe comparison is ridiculous. For let's say 99% of people, shoe requirements are the same (in function), with almost all variations being purely esthetic. There are, let's say, 10 kinds of shoes, or perhaps 100. Make it a thousand, for argument's sake.
Meanwhile, every single business has different workflows and therefore different needs. The most common ones (browsers, etc.) are answered by traditional software. If you can write in detail the business needs as pertaining to workflows - business rules, let's call them - you've effectively made the software already. The only difference being that telling ChatGPT to do something in English and telling the computer to do it in code is that one is non-deterministic.
Software is, primarily, a means to process information, which is to say reality (in a business setting). An AI that can replace software developers can, in effect, replace every job that happens on a computer, in every company on Earth. Apart from Jevon's paradox (which is much more applicable to software than shoes), this shift would be so gargantuan that it's barely worth thinking about, in the same way that it's not worth thinking about a supervolcanic eruption: the consequences would be earth-shattering, and finding employment would be the least of your worries.
reply