> In June, researchers published a study that concluded that one-tenth of the academic papers they examined “were processed with LLMs,” calling into question not just those individual papers but whole networks of citation and reference on which scientific knowledge relies.
> “I don’t think anyone has reliable information about post-2021 language usage by humans,” Speer wrote.
> Derek Sullivan, a cataloguer at a public-library system in Pennsylvania, told me that AI-generated books had begun to cross his desk regularly. Though he first noticed the problem thanks to a recipe book by a nonexistent author that featured “a meal plan that told you to eat straight marinara sauce for lunch,” the slop books he sees often cover highly consequential subjects like living with fibromyalgia or raising children with ADHD.
If it is (apparently) easy for humans to tell when content is AI-generated slop, then it should be possible to develop an AI to distinguish human-created content.
As mentioned, we have heuristics like frequency of the word "delve", and simple techniques such as measuring perplexity. I'd like to see a GAN style approach to this problem. It could potentially help improve the "humanness" of AI-generated content.
> If it is (apparently) easy for humans to tell when content is AI-generated slop
It's actually not. It's rather difficult for humans as well. We can see verbose text that is confused and call it AI, but it could just be a human aswell.
To borrow an older model training method, "Generative adversarial network". If we can distinguish AI from humans... We can use it to improve AI and close the gap.
So, it becomes an arms race that constantly evolves.
I made one of these back in 2005 and it was inevitably gangloaded with questionable content like p*rn, malware, software license keys, copyrighted material, etc.
Just p*rn?
You can consider yourself lucky.
One week after publishing my own public instance of an IMGUR-like, I woke up with +150 emails from my host provider and a final one telling me that they had shutdown my server.
Someone had posted ~500 pictures of "naked kids", a Canadian bot had found them and notified my host provider, who automatically took action.
Everything happened in less than 30 minutes, between the first picture being uploaded and the server being shutdown.
First I tried to restart the server and clean it, but I received new notifications as soon as it was online. So I just restored the last backup before it all started.
And I removed the ability for public users to upload pictures.
I will never ever publish a service allowing user to upload publicly any content.
* Literally every article (with descriptions, author info, post date, etc) displayed on the homepage in chronological order with no pagination, great for ctrl-f, etc.
* Selected authors and category-based navigation available.
* Entire site is very fast and lightweight, text-focused, no superfluous javascript or CSS, good typography, works well on mobile, etc.
This is very nice; I think the same can be achieved with general footnotes (not just source links), though you'd have to be careful to keep them short lest you take up a lot of screen space.
It’s just nice to see folks wanting minimalistic but functional features. I’m ticking quite a few of these on a service I’m working on. You can see my blog running on it https://lmno.lol/alvaro
Not yet officially launched. Happy to share invites to use now (for early adopters).
On HumanEval, I see 90.2 for GPT-4o and 89.0 for DeepSeek v2.5.
- https://blog.getbind.co/2024/09/19/deepseek-2-5-how-does-it-...
- https://paperswithcode.com/sota/code-generation-on-humaneval
reply