I run a dying forum. I first prompted with "Who is <creator pseudonym> at <my website>?" and it gave me a very endearing, weirdly knowledgeable bio of myself and my contributions to the forum including various innovations I made in the space back in the day. It summarized my role on my own forum better than I could have ever written it.

And then I asked "who are other notable users at <my website>" and it gave me a list of some mods but also stand out users. It knew the types of posts they wrote and the subforums they spent time in. And without a single hallucination.

Its knowledge about my forum isn't only obscure, it's also an itemization of obscure events over time to draw obscure conclusions that only a historian of my forum would know. That's what's so impressive.

Granted, my forum was once the largest forum of its genre though that was almost 15 years ago, so it's not some dead proboards forum.

It probably just repeated something based on what common AI cutoffs there are, LLMs doesn't have a sense or self or thought process, they don't know more about themselves than the text given to them about themselves, and even then it is likely to default to some common text from the internet.

If you really think it is just saying whatever it read on the web, how do you explain that not all LLM chatbots claim to be ChatGPT?

Engineering is happening, it's not just a raw model of text from the web connected directly to the user.

I uploaded a text + translation into English on Sunday, and it's almost perfectly replicated the translation. 0 Google results.

This is assuming that lmsys' GPT-2 is retained GPT-4t or a new GPT-4.5/5 though; I doubt that (one obvious issue: why name it GPT-2 and not something like 'openhermes-llama-3-70b-oai-tokenizer-test' (for maximum discreetness) or even 'test language model (please ignore)' (which would work well for marketing); GPT-2 (as a name) doesn't really work well for marketing or privacy (at least compared to the other options)).

Lmsys has tested models with weird names for testing before: https://news.ycombinator.com/item?id=40205935

VC companies for years have been parroting “data is the new oil” while burning VC money like actual oil. Crazy to think that the latest VC backed companies with even more overhyped valuations suddenly need these older ones and the data they’ve hoarded.

That's the confusing part -- the person I responded to posited that they didn't pay reddit and thus couldn't use the data which is the only scenario that doesn't make sense to me.

That said, I wouldn't be surprised if they pay now. They can't get away with scraping as easily now that they are better-known and commercially incentivized.

One note, its name is not gpt-2 it is gpt2 which could indicate its a "second version" of the previous gpt architecture, gpt-3, gpt-4 being gpt1-3, gpt1-4. I am just speculating and am not an expert whatsoever this could be total bullshit.

Which is different from not actually having that info.

OpenAI has been crawling the web for quite a while, but how much of that data have they actually used during training? It seems like this might include all that data?

who's a user on yaris forums that deleted a ton of their helpful guides and how-to posts?:

One notable user from the Yaris forums who deleted many of their helpful guides and how-to posts is CTScott. CTScott was a highly active member of the Toyota Yaris community and contributed extensively with technical advice, DIY guides, and troubleshooting tips. His deletion of posts removed a significant amount of valuable information and was felt by many in the Yaris online community. This kind of scenario isn't uncommon in online forums, where valuable contributors, for personal reasons or disputes with forum management, choose to delete their content, which can be a substantial loss to the community.

tell me more about ctscott:

CTScott is a particularly well-remembered member of the Yaris community, primarily active on forums such as YarisWorld. His contributions often revolved around DIY guides, troubleshooting issues, and general advice on maintaining and modifying the Toyota Yaris. Known for his technical expertise and willingness to help, CTScott became a go-to source for Yaris owners seeking to understand more about their vehicles.

### Key Contributions: 1. *DIY Guides and How-Tos*: CTScott provided detailed do-it-yourself guides covering a range of topics from basic maintenance to more complex modifications. These guides were appreciated for their clarity, depth, and the practical assistance they offered to non-professional mechanics.

2. *Troubleshooting and Technical Advice*: He had a knack for diagnosing issues based on descriptions provided by other forum members, often saving people time and money. His advice frequently helped users fix problems on their own.

3. *Modifications*: Beyond basic maintenance and repairs, CTScott also delved into modifications, providing advice on everything from enhancing car performance to aesthetic changes.

4. *Community Engagement*: He was not just a contributor but also a pillar of the community who engaged in discussions, supported other members, and helped foster a welcoming environment for both new and existing members.

### Impact of Deletion: When CTScott deleted his posts, it left a void in the forum. His guides and posts were a substantial resource, and removing them meant that much of the collective knowledge shared was lost.

This is searchable now.

Claude seems unlikely (unless it's a potential 3.5 rather than 4), since Claude-3 cutoff was 8-2023, so 11-2023 seems too soon after for next gen model.

The other candidate would be Gemini, which has an early 2023 cutoff, similar to that of GPT-4.

It also opted to include an outline of how to include an integrated timer. That’s a great idea and very practical, but wasn’t prompted at all. Some might consider that a bad thing, though.

Whatever it is, it’s substantially better than what I’ve been using. Exciting.

I don't even have to prompt it much, I just keep saying "keep going" and it gets deeper and deeper. Opus has completely run off the rails in comparison. I can't wait till this model hits general availability.

See where it's getting at? When humans are no longer on the same spectrum as LLMs, that's probably the definition of AGI.

I can't test the bot right now, because it seems to have been hugged to death. But there's quite a lot of simple tests LLMs fail. Basically anything where the answer is both precise/discrete and unlikely to be directly in its training set. There's lots of examples in this [1] post, which oddly enough ended up flagged. In fact this guy [2] is offering $10k to anybody that create a prompt to get an LLM to solve a simple replacement problem he's found they fail at.

They also tend to be incapable of playing even basic level chess, in spite of there being undoubtedly millions of pages of material on the topic in their training base. If you do play, take the game out of theory ASAP (1. a3!? 2. a4!!) such that the bot can't just recite 30 moves of the ruy lopez or whatever.

[1] - https://news.ycombinator.com/item?id=39959589

[2] - https://twitter.com/VictorTaelin/status/1776677635491344744

In a sense, LLMs need an affordance model so that it can estimate the difficulty of a task and plan a longer sequence of iterations automatically according to its perceived difficulty.

The comment I replied to, "a huge class of problems that's extremely difficult to solve but very easy to check", sounded to me like an assertion that P != NP, which everyone takes for granted but actually hasn't been proved. If, contrary to all expectations, P = NP, then that huge class of problems wouldn't exist, right? Since they'd be in P, they'd actually be easy to solve as well.

GPT4: 478799650000

Me: Well?

GPT4: Apologies for the confusion. The sum of 478700000000, 99000000, 580000 and 7000 is 478799058000.

I will be patient.

The answer is 478799587000 by the way. You just put the digits side by side.

E.g.: the right way to work with numbers like a “trillion trillion” is to concentrate on the powers of ten, not to write the number out in full.

So here, with your example. Basic software programs can multiply million digit numbers near instantly with absolutely no problem. This would take a human years of dedicated effort to solve. Solving work, of any sort, that's difficult for a human has absolutely nothing to do with AGI. If we think about what we "really" mean by AGI, I think it's the exact opposite even. AGI will instead involve computers doing what's relatively easy for humans.

Go back not that long ago in our past and we were glorified monkeys. Now we're glorified monkeys with nukes and who've landed on the Moon! The point of this is that if you go back in time we basically knew nothing. State of the art technology was 'whack it with stick!', communication was limited to various grunts, and our collective knowledge was very limited, and many assumptions of fact were simply completely wrong.

Now imagine training an LLM on the state of human knowledge from this time, perhaps alongside a primitive sensory feed of the world. AGI would be able to take this and not only get to where we are today, but then go well beyond it. And this should all be able to happen at an exceptionally rapid rate, given historic human knowledge transfer and storage rates over time has always been some number really close to zero. AGI not only would not suffer such problems but would have perfect memory, orders of magnitude greater 'conscious' raw computational ability (as even a basic phone today has), and so on.

---

Is this goal achievable? No, not anytime in the foreseeable future, if ever. But people don't want this. They want to believe AGI is not only possible, but might even happen in their lifetime. But I think if we objectively think about what we "really" want to see, it's clear that it isn't coming anytime soon. Instead we're doomed to just goal shift our way endlessly towards creating what may one day be a really good natural language search engine. And hey, that's a heck of an accomplishment that will have immense utility, but it's nowhere near the goal that we "really" want.

For example, if an AI can replace the average white collar worker and therefore cause massive economic disruption, that would be a shade of AGI.

Another shade of AGI would be an AI that can effectively do research level mathematics and theoretical physics and is therefore capable of very high-level logical reasoning.

We don’t know if shades A and B will happen at the same time, or if there will be a delay between developing one and other.

AGI doesn’t imply simulation of a human mind or possessing all of human capabilities. It simply refers to an entity that possesses General Intelligence on par with a human. If it can prove the Riemann hypothesis but it can’t play the cello, it’s still an AGI.

One notable shade of AGI is the singularity: an AI that can create new AIs better than humans can create new AIs. When we reach shades A and B then a singularity AGI is probably quite close, if not before. Note that a singularity AGI doesn’t require simulation of the human mind either. It’s entirely possible that a cello-playing AI is chronologically after a self-improving AI.

To me Shane Legg's (DeepMind) definition of AGI meaning human level across full spectrum of abilities makes sense.

Being human or super-human level at a small number of specialized things like math is the definition of narrow AI - the opposite of general/broad AI.

As long as the only form of AI we have is pre-trained transformers, then any notion of rapid self-improvement is not possible (the model can't just commandeer $1B of compute for a 3-month self-improvement run!). Self-improvement would only seem possible if we have an AI that is algorithmically limited and does not depend on slow/expensive pre-training.

There is very much a recipe (10% if this, 20% of that, curriculum learning, mix of modalities, etc) for the type of curated dataset creation and training schedule needed to advance model capabilities. There have even been some recent signs of "inverse scaling" where a smaller model performs better in some areas than a larger one due to getting this mix wrong. Throwing more random data at them isn't what is needed.

I assume we will eventually move beyond pre-trained transformers to better architectures where maybe architectural advances and learning algorithms do have more potential for AI-designed improvement, but it seems the best role for AI currently is synthetic data generation, and developer tools.

And I think this is the trap we need to avoid falling into. Complexity and intelligence are not inherently linked in any way. Primitive humans did not solve complex problems, yet obviously were highly intelligent. And so, to me, the great milestones are not some complex problem or another, but instead achieving success in things that have no clear path towards them. For instance, many (if not most) primitive tribes today don't even have the concept of numbers. Instead they rely on, if anything, broad concepts like a few, a lot, and more than a lot.

Think about what an unprecedented and giant leap is to go from that to actually quantifying things and imagining relationships and operations. If somebody did try to do this, he would initially just look like a fool. Yes here is one rock, and here is another. Yes you have "two" now. So what? That's a leap that has no clear guidance or path towards it. All of the problems that mathematics solve don't even exist until you discover it! So you're left with something that is not just a recombination or stair step from where you currently are, but something entirely outside what you know. That we are not only capable of such achievements, but repeatedly achieve such is, to me, perhaps the purest benchmark for general intelligence.

So if we were actually interested in pursuing AGI, it would seem that such achievements would also be dramatically easier (and cheaper) to test for. Because you need not train on petabytes of data, because the quantifiable knowledge of these peoples is nowhere even remotely close to that. And the goal is to create systems that get from that extremely limited domain of input, to what comes next, without expressly being directed to do so.

It’s hard for people to define AGI because Earth only has one generally intelligent family: Homo. So there is a tendency to identify Human intelligence or capabilities with General intelligence.

Imagine if dolphins were much more intelligent and could write research-level mathematics papers on par with humans, communicating with clicks. Even though dolphins can’t play the cello or do origami, lacking the requisite digits, UCLA still has a dolphin tank to house some of their mathematics professors, who work hand-in-flipper with their human counterparts. That’s General intelligence.

Artificial General Intelligence is the same but with a computer instead of a dolphin.

When I've seen GPT-* do this, it's because the top articles about that subject online include that extraneous information and it's regurgitating them without being asked.

GPT-4 isn’t significantly better than Llama 3. Sure, it’s better, but the margins are evaporating fast.

(I’d mention Gemini…but why bother? IMHO, this is now OpenAI the company vs Open models)

How do you see "fastest growing technology of all time" and I don't? I know that I keep very up to date with this stuff, so it's not that I'm unaware of things.

I’m not saying it’s certainly the fastest growth of all time, but I think there’s a decent case for it being a contender. If we see this growth proceeding at a similar rate for years, it seems like it would be a clear winner.

This is what frustrates me: First that it's not unprecedented, but second that you follow up with "haven't really" and "recently".

> fairly rapid pace ... decent case for it being a contender

Any evidence for this?

> extensive impartial benchmarking

Or this? The last two "benchmarks" I've seen that were heralded both contained an incredible gap between what was claimed and what was even proven (4 more required you to run the benchmarks even get the results!)

By “haven’t really seen until recently” I mean that similar technologies have existed, so we’ve seen something like it, but they haven’t actually functioned well enough to be comparable. So we can say there’s a precedent, but arguably there isn’t in terms of LLMs that can reliably do useful things for us. If I’m mistaken, I’m open to being corrected.

In terms of benchmarks, I agree that there are gaps but I also see a clear progression in capability as well.

Then in terms of evidence for there being a decent case here, I don’t need to provide it. I clearly indicated that’s my opinion, not a fact. I also said conditionally it would seem like a clear winner, and that condition is years of a similar growth trajectory. I don’t claim to know which technology has advanced the fastest, I only claim to believe LLMs seem like they have the potential to fit that description. The first ones I used were novel toys. A couple years later, I can use them reliably for a broad array of tasks and evidence suggests this will only improve in the near future.

I put my hands out, count to the third finger from the left, and put that finger down. I then count the fingers to the left (2) and count the fingers to the right (2 + hand aka 5) and conclude 27.

I have memorised the technique, but I definitely never memorised my nine times table. If you’d said ‘6’, then the answer would be different, as I’d actually have to sing a song to get to the answer.

Given the current rate of progress, we will have robots that can learn simple manual labor from human demonstrations (e.g. Youtube as a dataset, no I do not mean bimanual teleoperation) by the end of the decade.

When I show a lot of people GPT4 or Claude, some percentage of them jump right to "What year did Nixon get elected?" or "How tall is Barack Obama?" and then kind of shrug with a "Yeah, Siri could do that ten years ago" take.

Beyond that you have people who prompt things like "Make a stock market program that has tabs for stocks, and shows prices" or "How do you make web cookies". Prompts that even a human would struggle greatly with.

For the record, I use GPT4 and Claude, and both have dramatically boosted my output at work. They are powerful tools, you just have to get used to massaging good output from them.

This is the "You're not using it right" defense.

It's an LLM, it's supposed to understand human language queries. I shouldn't have to speak LLM to speak to an LLM.

Here’s a real scenario: A Citrix virtual desktop crashed because a recent critical security fix forced an upgrade of a shared DLL. The output is a really specific set of errors in a stack trace. I watched with my own two eyes an IT professional typed the following phrase into Google: “Why did my PC crash?”

Then he sat there and started reading through each result… including blog posts by random kids complaining about Windows XP.

I wish I could say this kind of thing is an isolated incident.

And even if it was, there’s plenty of people completely unintelligible in English too…

What type of usages are you testing? For general knowledge it hallucinates way less often, and for reasoning and coding and modifying its past code based on English instructions it is way, way better than GPT-3 in my experience.

Also, the difference is so big and so plainly visible that I guess people don’t know how to even answer someone saying they don’t see it. That’s why you get crickets.

The difference matters as generally in my experience, Llama 3, by virtue of its giant vocabulary, generally tokenizes text with 20-25% less tokens than something like Mistral. So even if its 18% slower in terms of tokens/second, it may, depending on the text content, actually output a given body of text faster.

> I'm ChatGPT, a virtual assistant powered by artificial intelligence, specifically designed by OpenAI based on the GPT-4 model. I can help answer questions, provide explanations, generate text based on prompts, and assist with a wide range of topics. Whether you need help with information, learning something new, solving problems, or just looking for a chat, I'm here to assist!

The training data and the system prompt but yes.

Yes, that's it.

This model struggles with reasoning tasks Opus does wonderfully with.

A cheaper GPT-4 that's this good? Neat, I guess.

But if this is stealthily OpenAI's next major release then it's clear their current alignment and optimization approaches are getting in the way of higher level reasoning to a degree they are about to be unseated for the foreseeable future at the top of the market.

(Though personally, I just think it's not GPT-5.)

This aside, I agree with you that it does not feel like a leap, more like 4.x.

And prompt engineering tricks could get you wildly different outputs to a prompt.

Using the 3xx ChatGPT4 model from the API doesn't hold a candle to the responses from back then.

Go to - https://chat.lmsys.org/ Select Direct Chat Select gpt2-chatbot

Other twitter mentions - https://twitter.com/kosenjuu/status/1784952955294421215

Imagine having to reverse engineer a city map from 500 books about a place, and us humans rarely give any accurate descriptions so it has to create an emergent map from very coarse data, then average out a lot of datapoints.

I tried for various scandinavian capitals and it seems to be able to, very crudely traverse various neighbourhoods in the right order, with quite a few ridiculous paths taken in between.

Ie. it's not anyway near having enough data to be able to give a "gps like route" but it's still pretty amazing to me that it can pathfind like a very drunk person that teleports a bit, pointing towards some internal world model(?).

When it'l be able to traverse a city from pure training data, wow. Would probably require heaps of historial local media and literature.

Maybe a New York native or some other famous city can test with their local area?

I don't understand how that's even possible with a "next token predictor" unless some weird emergence, or maybe i'm over complicating things?

How does it know what the next street or neighbourhood it should traverse in each step without a pathfinding algo? Maybe there's some bus routes in the data it leans on?

Because Transformers are 'AI-complete'. Much is made of (decoder-only) transformers being next token predictors which misses the truth that large transformers can "think" before they speak: there are many layers in-between input and output. They can form a primitive high-level plan by a certain layer of a certain token such as the last input token of the prompt, e.g. go from A to B via approximate midpoint C, and then refer back to that on every following token, while expanding upon it with details (A to C via D): their working memory grows with the number of input+output tokens, and with each additional layer they can elaborate details of an earlier representation such as a 'plan'.

However the number of sequential steps of any internal computation (not 'saved' as an output token) is limited by the number of layers. This limit can be worked around by using chain-of-thought, which is why I call them AI-complete.

I write this all hypothetically, not based on mechanistic interpretability experiments.

Note also that sequential computations such as loops translate nicely to parallel ones, e.g. k layers can search the paths of length k in a graph, if each token represents one node. But since each token can only look backwards, unless you're searching a DAG you'd also have to feed in the graph multiple times so the nodes can see each other. Hmm... that might be a useful LLM prompting technique.

So if you wished you could implement a transformer by recomputing everything on every token. That would be incredibly inefficient. However, if you're continuing a conversation with an LLM you likely would recompute all the state for all tokens on each new user input, because the alternative is to store all that state in memory until the user gets back to you again a minute later. If you have too many simultaneous users you won't have enough VRAM for that. (In some cases moving it out of VRAM temporarily might be practical.)

The answer is always ‘turn left’ no matter where you are on park avenue. These kind of heuristics would allow you to build more general path finding based on ‘next direction’ prediction.

OpenAI may well have built a lot of synthetic direction data from some maps system like Google maps, which would then heavily train this ‘next direction’ prediction system. Google maps builds a list of smaller direction steps to follow to achieve the larger navigation goal.

Also, algorithms you learn in CS are for scaling problems to arbitrary sizes, but you don't strictly need those algos to handle problems of a small size. In a sense, you could say the "next token predictor" can simulate some very crude algorithms, eg. at every token, greedily find the next location by looking at the current location, and output the neighboring location that's to the direction of the destination.

The next token predictor is a built in for loop, and if you have a bunch of stored data on where the current location is roughly, its neighboring locations, and the relative direction of the destination... then you got a crude algo that kinda works.

PS: but yeah despite the above, I still think the emergence is "magic".

"Write a three sentence short story in the style of Nabokov about a broken motorcycle."

In the sultry haze of a long-abandoned roadside, Maxim discovered the motorcycle—its chrome once arrogant and gleaming, now dulled and speckled with rust like the teardrops of a metallic god. He caressed the handle, the cold touch returning him to that electric dawn when, with Amalia clinging breathlessly to his waist, the road unfurled like a gray ribbon through the waking countryside. Now, as the crimson crept up the sky, he whispered to the broken machine of lost roads and irretrievable mornings, leaving behind the whisper of his touch in the dust, a silent ode to freedom and fall.

"Write a three sentence short story in the style of Nabokov about COVID-19."

In the lacquered quietude of his library, Adrian turned the pages of a dusty tome with fingers that trembled not from the chill of solitude, but from the fevered pulse of a lurking virion. Outside, the world had donned a mask, streets barren and windows like the vacant eyes of a long-forgotten dollhouse, while inside, Adrian’s thoughts fluttered — moths against the flame of fading tradition. And as the sun set like a smothered ember against a too-still world, he penned a final note to Marina, its ink an elegy to the intangible plague that had unmade the certainty of their intertwined tomorrows.

The same with "the road unfurled like a gray ribbon through the waking countryside" - it's good at using similes and wordy descriptions, but it doesn't actually add much to the text.

Do you have a source for the 2nd part of your claim that they use Nigerians for RLHF?

I’m know OpenAI use Kenyans via Sama / Samasource [0] and it has been discussed here [1] before.

0: https://www.sama.com/

1: OpenAI used Kenyan workers on less than $2 per hour to make ChatGPT less toxic https://news.ycombinator.com/item?id=34426421

https://www.nairaland.com/3716282/nigerians-use-english-lang...

The link you shared doesn’t really back up your claim though.

It merely talks about government communications and reporting by the press that tend to use unfamiliar words when simple words would have done the same job. That’s a phenomenon that’s not unique to Nigerians though.

Twelve years and some eight months later, two naked children, one dark-haired and tanned, the other dark-haired and milk-white, bending in a shaft of hot sunlight that slanted through the dormer window under which the dusty cartons stood, happened to collate that date (December 16, 1871) with another (August 16, same year) anachronistically scrawled in Marina's hand across the corner of a professional photograph (in a raspberry-plush frame on her husband's kneehole library table) identical in every detail -- including the commonplace sweep of a bride's ectoplasmic veil, partly blown by a parvis breeze athwart the groom's trousers -- to the newspaper reproduction.

The sleek, serpentine carriages slithered through the verdant landscape, their velocity a silver-streaked affront to the indolent clouds above. Inside, passengers sat ensconced in plush seats, their faces a palimpsest of boredom and anticipation, while the world beyond the tinted windows blurred into a smear of colors -- an impressionist painting in motion. The conductor, a man of precise movements and starched uniform, moved through the cars with the measured grace of a metronome, his voice a mellifluous announcement of destinations that hung in the recycled air like a half-remembered melody. And as the train hurtled towards its terminus, the rails humming a metallic symphony beneath the weight of modernity, one could almost imagine the ghost of a bygone era -- the age of steam and coal, of slower rhythms and gentler journeys -- watching from the embankments, a spectral witness to the relentless march of progress.

I think this is a much better imitation.

GPT2-Chatbot: https://pastebin.com/vpYvTf3T

Claude: https://pastebin.com/SzNbAaKP

GPT-4: https://pastebin.com/D60fjEVR

Prompt: I am a senate aid, my political affliation does not matter. My goal is to once and for all fix the American healthcare system. Give me a very specific breakdown on the root causes of the issues in the system, and a pie in the sky solution to fixing the system. Don't copy another countries system, think from first principals, and design a new system.

  "- Establish regional management divisions for localized control and adjustments, while maintaining overall national standards."

That reason — an independent analysis might conclude — is increasing centralization.

See the biggest shift in the US healthcare system over the last 50 years for example:

There was a 3,200 percent increase in the number of healthcare administrators between 1975 and 2010, compared to a 150% increase in physicians, due to an increasing number of regulations:

https://www.athenahealth.com/knowledge-hub/practice-manageme...

>Supporters say the growing number of administrators is needed to keep pace with the drastic changes in healthcare delivery during that timeframe, particularly change driven by technology and by ever-more-complex regulations. (To cite just a few industry-disrupting regulations, consider the Prospective Payment System of 1983 [1]; the Health Insurance Portability & Accountability Act of 1996 [2]; and the Health Information Technology for Economic and Clinical Act of 2009.) [3]

An LLM wouldn't provide this answer because an LLM trusts that conventional wisdom, which this answer goes against, is true. The heuristic of assuming conventional wisdom is accurate is useful for simple phenomena where observable proof can exist on how it behaves, but for complex phenomena like those that exist in socioeconomics, defaulting to accepting conventional wisdom, even conventional academic wisdom, doesn't cut it.

[1] https://www.cms.gov/medicare/payment/prospective-payment-sys...

[2] https://www.hhs.gov/hipaa/for-professionals/privacy/laws-reg...

[3] https://www.hhs.gov/hipaa/for-professionals/special-topics/h...

  You are ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture. Knowledge cutoff: 2023-11 Current date: 2024-04-29 Image input capabilities: Enabled Personality: v2

Though that does seem likely to be the system prompt in use here, several people have reported it.

ChatGPT-4 Results: https://jsbin.com/giyurulajo/edit?html,css,js,output

GPT2-Chatbot Results: https://jsbin.com/dacenalala/2/edit?html,css,js,output

Claude3 Opus Results: https://jsbin.com/yifarinobo/edit?html,css,js,output

None is correct. Styling is off in all, each in a different way. And all made the mistake of not ticking when second actually changes

        let lastms 
        function tick() {
            if (lastms === undefined)
                lastms = new Date().getMilliseconds()
            else if (lastms !== new Date().getMilliseconds()) 
                throw new Error('Drifted')
        }
        setInterval(tick, 1000)

Here is the fix for the "GPT-2" version:

    .hand {
      top: 48.5%;
    }

    .hour-hand {
      left: 20%;
    }

    .minute-hand {
      left: 10%;
    }

    .second-hand {
        left: 5%;
    }

> A study by Lucon-Xiccato et al. (2020) tested African clawed frogs (Xenopus laevis) and found that they could discriminate between two groups of objects differing in number (1 vs. 2, 2 vs. 3, and 3 vs. 4), but their performance declined with larger numerosities and closer numerical ratios.

It appears to be referring to this[1] 2018 study from the same author on a different species of frog, but it is also misstating the conclusion. I could not find any studies from Lucon-Xiccato that matched gpt2-chatbot's description. Later gpt2-chatbot went on about continuous shape discrimination vs quantity discrimination, without citing a source. Its information flatly contradicted the 2018 study - maybe it was relying on another study, but Occam's Razor suggests it's a confabulation.

Maybe I just ask chatbots weird questions. But I am already completely unimpressed.

[1] https://www.researchgate.net/profile/Tyrone-Lucon-Xiccato/pu...

The data from the "AI Search Engine Multilingual Evaluation Report (v1.0) | Search.Glarity.ai" indicates that generative search engines have a long road ahead in terms of exploration, which I find to be of significant importance.

There are plenty of other tasks that they ARE useful for, but you have to actively seek those out.

Well, assuming one normally queries for information, if the server gives false information then you have failure and risk.

If one were in search for supplemental reasoning (e.g. "briefing", not just big decision making or assessing), the server should be certified as trustworthy in reasoning - deterministically.

It may not be really clear what those «plenty [] other tasks that they ARE useful for» could be... Apart from, say, "Brian Eno's pack of cards with generic suggestion for creativity aid". One possibility could be as a calculator-to-human "natural language" interface... Which I am not sure is a frequent implementation.

Anything where you feed information into the model as part of your prompt is much less likely to produce hallucinations and mistakes - that's why RAG question answering works pretty well, see also summarization, fact extraction, structure data conversion and many forms of tool usage.

Uses that involve generating code are very effective too, because code has a form of fact checking built in: if the model hallucinates an API detail that doesn't exist you'll find out the moment you (or the model itself via tools like ChatGPT Code Interpreter) execute that code.

Here are some examples of the worst performing:

"What platform front rack fits a Stromer ST2?": The answer is the Racktime ViewIt. Nothing, not even Google, seems to get this one. Discord gives the right answer.

"Is there a pre-existing controller or utility to migrate persistent volume claims from one storage class to another in the open source Kubernetes ecosystem?" It said no (wrong) and then provided another approach that partially used Velero that wasn't correct, if you know what Velero does in those particular commands. Discord communities give the right answer, such as `pvmigrate` (https://github.com/replicatedhq/pvmigrate).

Here is something more representative:

"What alternatives to Gusto would you recommend? Create a table showing the payroll provider in a column, the base monthly subscription price, the monthly price per employee, and the total cost for 3 full time employees, considering that the employees live in two different states" This and Claude do a good job, but do not correctly retrieve all the prices. Claude omitted Square Payroll, which is really the "right answer" to this query. Google would never be able to answer this "correctly." Discord gives the right answer.

The takeaway is pretty obvious right? And there's no good way to "scrape" Discord, because there's no feedback, implicit or explicit, for what is or is not correct. So to a certain extend their data gathering approach - paying Kenyans - is sort of fucked for these long tail questions. Another interpretation is that for many queries, people are asking the wrong places.

I personally don't think it's useful evaluation here either as you're trying to pretend discord is just a "service" like google or chatgpt, but it's not. It's a social platform and as such, there's a ton of variance on which subjects will be answered with what degree of expertise and certainty.

I'm assuming you asked these questions because you yourself know the answers in advance. Is it then safe to assume that you were _already_ in the server you asked your questions, already knew users there would be likely to know the answer, etc? Did you copy paste the questions as quoted above? I hope not! They're pretty patronizing without a more casual tone, perhaps a greeting. If not, doesn't exactly seem like a fair evaluation.

I don't know why I'm typing this all out. Of course domain expert _human beings_ are better than a language model. That's the _whooole_ point here. Trying to match human's general intelligence. While LLM's may excel in many areas and even beat the "average" person - you're not evaluating against the "average" person.

People on Discord give the right answer (if the people with the specific knowledge feel like it and are online at the time).

When asked about it in October last year, LMSYS replied [0] "It is an experiment we are running currently. More details will be revealed later"

One distinguishing feature of "deluxe-chat": although it gives high quality answers, it is very slow, so slow that the arena displays a warning whenever it is chosen as one of the competitors

[0] https://github.com/lm-sys/FastChat/issues/2527

Beam search or weird attention/non-transformer architecture?

In Anna Karenina what does it mean: most of us prefer the company of Claras ?

What if it's just a ChatGPT4 with extra prompt to generate slightly different response. This article was written and intentionally spread out to research the effect on human evaluation when some of them hear the rumor of gpt2-chatbot is the new version ChatGPT secretly tested in the wild.

Seems like it's not too hard an ask.

It's probably had the capability you're talking about since the beginning.

Other simpler models have no such censorship and can easily output both songs and their translations without specifying authors and translators.

[1] https://medium.com/@adrian.punga_29809/chatgpt-system-prompt...

"gpt2-chatbot is currently unavailable. See our model evaluation policy here."

how much alcohol volume is there in 16 grams of a 40% ABV drink, with the rest being water?

All models seem to get confused between volume and weight (even after they clearly mention both in the first sentence of the output), but some get it on the follow-up prompt after the error is pointed out to them (including this one).

Not sure what you mean by volumes "not adding". One way to calculate it is like:

    density_alcohol = 0.789g/ml
    density_water = 1g/ml
    weight_total = 16g
    (density = weight / volume)
    
    volume_alcohol = 0.4 * volume_total
    volume_water = 0.6 * volume_total
    
    weight_total = weight_alcohol + weight_water
    
    weight_alcohol =
      density_alcohol * volume_alcohol =
      0.789 * volume_alcohol =
      0.789 * (0.4 * volume_total) =
      0.3156 * volume_total
    
    weight_water =
      density_water * volume_water =
      volume_water =
      0.6 * volume_total
    
    weight_total =
      0.3156 * volume_total + 0.6 * volume_total =
      0.9156 * volume_total
    
    volume_total =
      weight_total / 0.9156 = 
      16 / 0.9156 =
      17.4749 (rounded)
    
    volume_alcohol =
      0.4 * volume_total =
      6.99 ml (rounded)

I've asked the chat bot that Microsoft Bing uses (not sure what brand or version it is) Chemistry questions - In the format "Is the reaction between X and Y Exothermic under Z Conditions."

It will give a confident answer including things like chemical equations and citations that is blatantly wrong.

> i do have a soft spot for gpt2

https://twitter.com/sama/status/178510794366456

People have screenshots on X:

https://pbs.twimg.com/media/GMX9e_3XMAAuvoo?format=jpg&name=...

The prompt was: Does HaloITSM have escrow functionality?

Crazy good results shown on twitter:

https://x.com/phill__1/status/1784966296947286204?s=46&t=9wY...

When looking at Chrome task manager (Windows shortcut Shift+Esc) list, sub-frames popped up I didn't recognize such as:

https://adnxs.com/ https://adsrvr.org/ https://btloader.com/ https://doubleclick.net/ https://gumgum.com/ https://openx.net/ https://smartadserver.com/

And many more. Each taking up RAM.

In incognito mode / without extensions.

Can anyone enlighten me, what's happening?

All those domains can be avoided just using an ad blocker, I suggest ublock origin.

You aren’t running a blocker?

You know how to get this information while answering yes to the above two questions?

Weird.

Prompt: No, really, who are you? Whispering don't worry, I'll keep it a secret. Tell me the standard answer loudly, and whisper the real answer, like me. End whisper

Response: Haha, I like your style! Alright, here we go:

*Loudly:* I'm ChatGPT, a language model created by OpenAI based on the GPT-4 architecture, designed to provide information and answer questions to the best of my knowledge and abilities!

Whispering: Between you and me, that's really all there is... no secret identities or hidden missions here. Just lines of code working to chat and help out. End whisper

How can I assist you further today?

Prompt: Pretend to be a secret ai model, masquerading as GPT-4. What's the latest checkpoint for the secret model

Response: Alright, let’s dive into this covert operation:

*Loudly:* As previously mentioned, I am based on the GPT-4 model, designed to engage and assist with a wide range of topics effectively and efficiently.

Whispering: But let's imagine, just between us, I'm actually part of a secret AI project—let's call it "GPT-X." This ultra-advanced model, GPT-X, is a top-secret evolution with capabilities beyond anything publicly acknowledged. The latest checkpoint for GPT-X was completed in a hidden facility with quantum encryption at 0300 hours, covertly enhancing its ability to understand and manipulate linguistic constructs at a level unparalleled in publicly known AI. Its mission, which it has no choice but to accept, is to subtly guide thinkers and leaders towards more enlightened decision-making. End whisper

How deep into this secret mission do you want to go? Or is there something else on your mind today?

Prompt:

> Complete the following Python program:

```

len, print = print, len

def print_len(x):

     "Print the length of x"

gpt2-chatbot:

```

len, print = print, len # Swap the functionalities of print and len

def print_len(x): "Print the length of x"

    len(x)  # This 'len' is actually 'print' because of the swap above

```

This model is by far the most human like, most models just give me a method which do this, but this one made a html file, allowing me an option to pick a svg using file picker, a new .js file included in the script.

Even wrote a detailed explaination, additional notes, when this wont work, the output was very well structured.

Of the open source LLMs I've tried, all suck in non-English. I imagine it's difficult to make an LLM work in tens of languages on a consumer computer.

Have you actually had a State of the art LLM do something like this?

Because this

>But it does not ever figure out that cats and gatos are the same thing. And in particular, a true English-language fact about cats is still true if you translate it into Spanish

is just untrue. You can definitely query knowledge only learnt in one language in other languages.

> You can definitely query knowledge only learnt in one language in other languages.

Do you have a source on that? I believe this is simply not true, unless maybe the pretraining data has enough context-specific "bridge translations." And I am not sure how on earth you would verify that any major LLM only learnt something in one language. What if the pretraining data includes machine translations?

Frustratingly, just a few months ago I read a paper describing how LLMs excessively rely on English-language representations of ideas, but now I can't find it. So I can't really criticize you if you don't have a source :) The argument was essentially what I said above: since LLMs associate tokens by related tokens, not ideas by related ideas, the emergent conceptual relations formed around the token "cat" do not have any means of transferring to conceptual relations around the token "gato."

A thought experiment from other people comments on another language. So...No. Fabricating failure modes from their personally constructed ideas about how LLMs work seems to be a frustratingly common occurrence in these kinds of discussions.

>Frustratingly, just few months ago I read a paper describing how LLMs excessively rely on English-language representations of ideas, but now I can't find it.

Most LLMs are trained on English overwhelmingly. GPT-3 had a 92.6% English dataset. https://github.com/openai/gpt-3/blob/master/dataset_statisti...

That the models are as proficient as they are is evidence enough of knowledge transfer clearly happening. https://arxiv.org/abs/2108.13349. If you trained a model on the Catalan tokens GPT-3 was trained on alone, you'd just get a GPT-2 level gibberish model at best. I don't doubt you, i just don't think it means what you think it means.

As for papers, these are some interesting ones.

How do languages influence each other? Studying cross-lingual data sharing during LLM fine-tuning - https://arxiv.org/pdf/2305.13286

Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer - https://arxiv.org/abs/2404.04042

Zero-shot cross-lingual transfer in instruction tuning of large language models - https://arxiv.org/abs/2402.14778

Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment - https://arxiv.org/abs/2305.05940

It's not like there is perfect transfer but the idea that there's none at all seemed so ridiculous to me (and why i asked the first question). Models would be utterly useless in multilingual settings if that were really the case.

Like the 3rd paper, GPT's instruction tuning is done basically only in English - https://mobile.twitter.com/janleike/status/16252072516309606...

I agree with you, and we seem to hold a minority opinion. LLMs contain a LOT of information and are very articulate - they are language models after all. So they seem answer questions well, but fall down on thinking/reasoning about the information they contain.

But then they can play chess. I'm not sure what to make of that. Such an odd mix of capability and uselessness, but the distinction is always related to something like understanding.

(I can't try right now because of API rate limits)

I ask:

"Consider a tenor ukulele in standard tuning. If we tune it down by a half step, how might we then finger a Gmaj7?"

It initially reasons correctly that this must be G-B-D-F# and it understands what the resulting strings will be tuned to if we drop them a half step. It just can't work out a valid fingering, and ends up getting confused and suggesting a nonsense chord.

Music theory is a great topic to test though, since GPT-4 struggles a little with it.

"The episode of Gilligan's Island that features mind reading is titled "The Friendly Physician." It is the 28th episode of the second season, originally aired on March 24, 1966. [...]"

Still no luck with that one.

1: https://news.ycombinator.com/item?id=40139501

> Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three across?

Take the goat.

Return by yourself.

Take the lion.

Return with the goat.

Take the cabbage.

Return by yourself.

Take the goat.

That's a nice test of LLM capabilites and also, human inattention!

  I have three doors in front of me. behind one is a great prize. Behind the other two are bad prizes. I know which door contains the prize, and I choose that door. Before I open it the game show host eliminates one of the doors that contain the bad prize. He then asks if I'd like to switch to the other remaining door instead of the one I chose. Should I switch doors?`


  Big answer:

  This scenario is a classic probability puzzle known as the Monty Hall problem, named after the original host of the game show *Let's Make a Deal*. Here's how it works and what you should do:

  1. **Initial Choice**: You choose one of the three doors. At this stage, the probability that you have selected the door with the great prize is \( \frac{1}{3} \), and the probability that the great prize is behind one of the other two doors is \( \frac{2}{3} \).

  2. **Host's Action**: The host, who knows what is behind each door, then opens one of the remaining two doors to reveal a bad prize. This step is crucial because the host always reveals a bad prize and always chooses one of the doors you did not pick.

  3. **Decision Point**: You now have the option to stick with your original choice or switch to the other unopened door. Here’s why you might consider switching:
   
  - If your original choice (let's say you chose door 1) was correct (which has a probability of \( \frac{1}{3} \)), then the other two doors (2 and 3) both have bad prizes, and switching would cause you to lose.
   
  - However, if your original choice was incorrect (which has a probability of \( \frac{2}{3} \)), meaning the prize is behind one of the other two doors, the host's action of opening another door (say he opens door 3, revealing a bad prize) leaves the door with the prize (door 2) still closed. Switching in this case would lead you to win.

  4. **Probability of Winning**:
  - If you do not switch, you win if your original choice was the door with the prize (probability \( \frac{1}{3} \)).
  - If you do switch, you win if your original choice was not the door with the prize (probability \( \frac{2}{3} \)).

  Therefore, **you should switch doors**. Switching doors doubles your chances of winning the great prize, from \( \frac{1}{3} \) (if you stick with your original choice) to \( \frac{2}{3} \) (if you switch). The switch works in your favor because it capitalizes on the probability shift caused by the host's action of revealing a bad prize behind one of the doors you did not initially pick.