People keep saying "it will get there eventually," or some variety thereof, and I just gotta keep reminding - that is an as-yet unproven claim. It may never get there! Just because we've seen some pretty astounding leaps in capabilities so far, _does not mean_ that we will continue to see them, nor that we'll ever hit the fabled land of "no more mistakes," or hell, even "no more obvious mistakes."
I'm not saying I think it won't (though I _suspect_ it won't), I'm just saying we don't have any actual proof that it _will,_ we're all just running on assumptions right now.
To me, I see LLMs as the same type of revolution that dragon naturally speaking was to voice to text.
A huge leap forward over existing models, but we've spent the last two (three?) decades trying to close the remaining gap left by dragon in the voice to text problem space, and haven't much progress to show.
I think LLMs are likely to be like that. They are a huge jump over previous models of NLP, but I don't see them improving enough to matter to indicate they'll ever make it to AGI
The limitation lies in Transformer architecture itself, therefore AGI was never a possibiblity.
It's wonderful, but not miraculous; And that's okay.
At this point big techs are just milking the hype of what already reached it's boundaries.
I dunno I'm confident enough to say transformer based architectures have really reached their boundaries quite yet but I would agree the hype milks a bit past where it seems transformers alone will take things.
> but we've spent the last two (three?) decades trying to close the remaining gap left by dragon in the voice to text problem space, and haven't much progress to show.
Who says it's possible to close the gap? Humans are certainly not capable of doing perfect speech-to-text. You can sit someone down with a song recording and the ability to replay it as much as they want and there's no guarantee they'll ever be able to tell you what the lyrics are.
I'm not saying I think it won't (though I _suspect_ it won't), I'm just saying we don't have any actual proof that it _will,_ we're all just running on assumptions right now.